yrb-lite-actioncable 0.1.0.beta2 → 0.1.0.beta3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG-actioncable.md +19 -4
- data/README.md +100 -146
- data/lib/yrb_lite/action_cable/sync.rb +111 -361
- data/lib/yrb_lite/action_cable/version.rb +1 -1
- metadata +31 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2a29ee03d2c3eb1dd68e3b6be4ffc8d57f7c244c6ace2b4edcc715467f525d60
|
|
4
|
+
data.tar.gz: c80b92042a593af7bc9cf22233032af3ca783c507a7a2df2cce052501aadfcd2
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 2687a09e9f83d54240f6b9fc6c40858a30864b8912e2dc8b146a70bc95f5069933efe3ed5643fada439ecdafeb0c392c8edfaff147b38ab67dd206378db9f373
|
|
7
|
+
data.tar.gz: 39f410248b09f295d83a8621e7a7cdada6661fa9aa2b81fc967b75838a878a84583f1e3ec7e6dcf9f300ed840f32eca62c8b4d1cc13ab874efb2fc753cb47523
|
data/CHANGELOG-actioncable.md
CHANGED
|
@@ -6,6 +6,22 @@ this project aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [0.1.0.beta3] - 2026-06-22
|
|
10
|
+
|
|
11
|
+
### Changed
|
|
12
|
+
|
|
13
|
+
- Document streams and awareness streams are separate under AnyCable. Document
|
|
14
|
+
streams are normal server-relayed streams; awareness streams are subscribed
|
|
15
|
+
with `whisper: true` and carry only ephemeral presence frames.
|
|
16
|
+
- The channel accepts and emits the canonical document envelope,
|
|
17
|
+
`{ "update" => "<base64 frame>" }`. Accepted document updates carrying an
|
|
18
|
+
`"id"` are acknowledged with `{ "ack" => id }`.
|
|
19
|
+
- Removed the backend switch. `YrbLite::ActionCable::Sync` is always
|
|
20
|
+
store-backed and fails closed unless both `on_load` and `on_change` are
|
|
21
|
+
configured, so acknowledgements always mean the update is durably recorded.
|
|
22
|
+
- Requires `yrb-lite >= 0.1.0.beta6` (uses the new `update_advances?` and
|
|
23
|
+
wire client-id frame validation).
|
|
24
|
+
|
|
9
25
|
## [0.1.0.beta2] - 2026-06-20
|
|
10
26
|
|
|
11
27
|
### Added
|
|
@@ -26,10 +42,9 @@ this project aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
26
42
|
`YrbLite::Sync` through 0.1.0.beta4). Provides `YrbLite::ActionCable::Sync`, an
|
|
27
43
|
ActionCable channel concern implementing the y-websocket sync protocol and
|
|
28
44
|
awareness/presence over ActionCable and AnyCable: record-before-distribute
|
|
29
|
-
auditing (`on_change`), persistence hooks
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
awareness, and protocol primitives.
|
|
45
|
+
auditing (`on_change`), persistence hooks, presence reaping, idle-document
|
|
46
|
+
eviction, and multi-process replica sync. Depends on `yrb-lite`
|
|
47
|
+
(>= 0.1.0.beta5) for the CRDT documents, awareness, and protocol primitives.
|
|
33
48
|
- `on_change` recorders run in the channel instance's context (carried over from
|
|
34
49
|
`yrb-lite` 0.1.0.beta4), so a recorder can call the channel's own methods
|
|
35
50
|
directly.
|
data/README.md
CHANGED
|
@@ -9,26 +9,27 @@ documents.
|
|
|
9
9
|
|
|
10
10
|
```ruby
|
|
11
11
|
class DocumentChannel < ApplicationCable::Channel
|
|
12
|
-
include YrbLite::Sync
|
|
12
|
+
include YrbLite::ActionCable::Sync
|
|
13
13
|
|
|
14
14
|
def subscribed = sync_for(params[:id])
|
|
15
15
|
def receive(data) = sync_receive(data)
|
|
16
|
-
def unsubscribed =
|
|
16
|
+
def unsubscribed = sync_unsubscribed(params[:id])
|
|
17
17
|
end
|
|
18
18
|
```
|
|
19
19
|
|
|
20
|
-
On the browser, use the
|
|
21
|
-
|
|
20
|
+
On the browser, use the `yrb-lite-client` `ActionCableProvider`. Tiptap,
|
|
21
|
+
ProseMirror, and BlockNote all sync through the `Y.Doc` you pass in and the
|
|
22
|
+
provider's Awareness instance, unless you supply your own.
|
|
22
23
|
|
|
23
24
|
## What you get
|
|
24
25
|
|
|
25
|
-
- Thread-safe `Doc` and `Awareness`. You can share them
|
|
26
|
-
|
|
26
|
+
- Thread-safe Ruby wrappers for `Doc` and `Awareness`. You can share them
|
|
27
|
+
across Puma threads; native CRDT work runs with the GVL released.
|
|
27
28
|
- The y-websocket protocol (document sync plus awareness/presence) as a
|
|
28
29
|
one-include ActionCable concern.
|
|
29
|
-
-
|
|
30
|
-
-
|
|
31
|
-
goes out to anyone.
|
|
30
|
+
- Store-backed ActionCable/AnyCable delivery for multi-process deployments.
|
|
31
|
+
- Authoritative record-before-distribute semantics: each document change is
|
|
32
|
+
recorded durably before it goes out to anyone.
|
|
32
33
|
|
|
33
34
|
What it doesn't do: auth, read-only connections, rate limiting, webhooks,
|
|
34
35
|
metrics. Hocuspocus ships extensions for those; here you'd build them with
|
|
@@ -36,21 +37,26 @@ Rails.
|
|
|
36
37
|
|
|
37
38
|
## Testing
|
|
38
39
|
|
|
39
|
-
Ruby and Rust unit tests cover the core
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
The benchmark
|
|
43
|
-
welcome.
|
|
40
|
+
Ruby and Rust unit tests cover the core. CI also runs the npm client tests and a
|
|
41
|
+
Rails demo smoke slice against the real ActionCable stack. The demo includes
|
|
42
|
+
heavier local suites for hostile input, crash recovery, multi-browser editing,
|
|
43
|
+
AnyCable, and load testing. The benchmark note below is from a single laptop.
|
|
44
|
+
Issues and PRs are welcome.
|
|
44
45
|
|
|
45
46
|
## Install
|
|
46
47
|
|
|
47
48
|
```ruby
|
|
49
|
+
# Core CRDT + protocol primitives:
|
|
48
50
|
gem "yrb-lite"
|
|
51
|
+
|
|
52
|
+
# For the Rails/ActionCable server concern (YrbLite::ActionCable::Sync):
|
|
53
|
+
gem "yrb-lite-actioncable"
|
|
49
54
|
```
|
|
50
55
|
|
|
51
|
-
Requires Ruby 3.4 or newer.
|
|
52
|
-
3.4 and 4.0
|
|
53
|
-
|
|
56
|
+
Requires Ruby 3.4 or newer. The release workflow builds precompiled gems for
|
|
57
|
+
Ruby 3.4 and 4.0 across the supported Ruby platforms, with native smoke tests
|
|
58
|
+
on Linux x86_64 and macOS arm64. Installing from a matching platform gem needs
|
|
59
|
+
no Rust; a source build needs [Rust](https://rustup.rs).
|
|
54
60
|
|
|
55
61
|
To work on the gem itself:
|
|
56
62
|
|
|
@@ -128,44 +134,43 @@ send_to_peer(response) unless response.empty?
|
|
|
128
134
|
|
|
129
135
|
### ActionCable Integration
|
|
130
136
|
|
|
131
|
-
`YrbLite::Sync`
|
|
132
|
-
protocol (document sync +
|
|
137
|
+
`YrbLite::ActionCable::Sync` (from the `yrb-lite-actioncable` gem) is a channel
|
|
138
|
+
concern that implements the full y-websocket protocol (document sync +
|
|
139
|
+
awareness/presence) over ActionCable:
|
|
133
140
|
|
|
134
141
|
```ruby
|
|
135
142
|
# app/channels/document_channel.rb
|
|
136
143
|
class DocumentChannel < ApplicationCable::Channel
|
|
137
|
-
include YrbLite::Sync
|
|
144
|
+
include YrbLite::ActionCable::Sync
|
|
138
145
|
|
|
139
|
-
#
|
|
140
|
-
|
|
141
|
-
# on_save { |key, update| Document.find_by(key: key)&.update!(content: update) }
|
|
146
|
+
on_load { |key| MyStore.load(key) } # source of truth
|
|
147
|
+
on_change { |key, update| MyStore.append(key, update) } # durable record
|
|
142
148
|
|
|
143
149
|
def subscribed
|
|
144
150
|
sync_for params[:id]
|
|
145
151
|
end
|
|
146
152
|
|
|
147
153
|
def receive(data)
|
|
148
|
-
sync_receive(data)
|
|
154
|
+
sync_receive(data, params[:id])
|
|
149
155
|
end
|
|
150
156
|
|
|
151
157
|
def unsubscribed
|
|
152
|
-
|
|
158
|
+
sync_unsubscribed(params[:id])
|
|
153
159
|
end
|
|
154
160
|
end
|
|
155
161
|
```
|
|
156
162
|
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
163
|
+
The concern is store-backed. A handshake is answered from `on_load`; document
|
|
164
|
+
changes are checked against that durable state, recorded through `on_change`,
|
|
165
|
+
then broadcast. Nothing authoritative is kept in ActionCable process memory, so
|
|
166
|
+
AnyCable RPC workers, Puma workers, and separate dynos can all handle messages
|
|
167
|
+
for the same document as long as they share the same store and cable adapter.
|
|
162
168
|
|
|
163
|
-
`
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
it, the in-memory copy is the only one and stays put.
|
|
169
|
+
`on_load` and `on_change` are required. If either is missing, the channel fails
|
|
170
|
+
closed before it can acknowledge or broadcast edits. Presence is ephemeral:
|
|
171
|
+
awareness frames are relayed, and `yrb-lite-client` sends a best-effort
|
|
172
|
+
presence-removal frame on disconnect/pagehide, with the client-side awareness
|
|
173
|
+
timeout as the fallback for abrupt disconnects.
|
|
169
174
|
|
|
170
175
|
Incoming frames are validated as a single well-formed protocol message before
|
|
171
176
|
anything processes or relays them. Malformed, truncated, multi-message,
|
|
@@ -182,37 +187,19 @@ Broadcasts cross processes through the Action Cable adapter, so it needs to be a
|
|
|
182
187
|
real one (`redis` or `solid_cable`, not `async`). With that in place, a change
|
|
183
188
|
on one process reaches clients on all of them.
|
|
184
189
|
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
tells a process to skip its own.
|
|
190
|
-
|
|
191
|
-
A cold process (no copy yet) rebuilds from the durable store through `on_load`.
|
|
192
|
-
In authoritative mode the store is always current, since changes are recorded
|
|
193
|
-
before they go out. Record-before-distribute therefore holds across processes:
|
|
194
|
-
whichever process receives a change records it to the shared store before
|
|
195
|
-
anyone, anywhere, sees it.
|
|
190
|
+
Every process rebuilds document state from the durable store through `on_load`.
|
|
191
|
+
Because changes are recorded before broadcast, record-before-distribute holds
|
|
192
|
+
across processes: whichever process receives a change records it to the shared
|
|
193
|
+
store before anyone, anywhere, sees it.
|
|
196
194
|
|
|
197
195
|
`bun multiprocess.mjs` in the demo runs clients across two processes and checks
|
|
198
|
-
|
|
199
|
-
shared log.
|
|
200
|
-
|
|
201
|
-
##### AnyCable (`sync_backend :store`)
|
|
196
|
+
convergence, fresh reads on both, presence across processes, and one shared log.
|
|
202
197
|
|
|
203
|
-
|
|
204
|
-
block running in Ruby for each broadcast. AnyCable breaks both assumptions.
|
|
205
|
-
anycable-go delivers broadcasts outside Ruby, so the block never runs. Each RPC
|
|
206
|
-
gets a fresh channel instance, which means ivars set in `subscribed` are gone by
|
|
207
|
-
`receive`. And there's no fixed worker-to-document mapping to lean on.
|
|
208
|
-
|
|
209
|
-
`sync_backend :store` is the path for that: stateless per message, no warm
|
|
210
|
-
copy.
|
|
198
|
+
##### AnyCable
|
|
211
199
|
|
|
212
200
|
```ruby
|
|
213
201
|
class DocumentChannel < ApplicationCable::Channel
|
|
214
|
-
include YrbLite::Sync
|
|
215
|
-
sync_backend :store
|
|
202
|
+
include YrbLite::ActionCable::Sync
|
|
216
203
|
|
|
217
204
|
on_load { |key| MyStore.load(key) } # required: source of truth
|
|
218
205
|
on_change { |key, update| MyStore.append(key, update) } # required: record
|
|
@@ -227,6 +214,10 @@ end
|
|
|
227
214
|
- A handshake (SyncStep1) is answered from the store. Changes are recorded, then
|
|
228
215
|
broadcast. Nothing is held in Ruby between calls, so any worker can handle any
|
|
229
216
|
message.
|
|
217
|
+
- Document frames use the normal server path. Awareness/presence uses a
|
|
218
|
+
separate awareness stream with AnyCable `whisper: true`, so cursor traffic can
|
|
219
|
+
take the low-latency client-to-client path without bypassing document
|
|
220
|
+
durability.
|
|
230
221
|
- Pass `params[:id]` into `sync_receive`/`sync_unsubscribed` so the document key
|
|
231
222
|
survives AnyCable's per-command instances.
|
|
232
223
|
- The sender gets its own updates echoed back (no Ruby callback to filter them).
|
|
@@ -234,34 +225,31 @@ end
|
|
|
234
225
|
|
|
235
226
|
The demo checks this against a real anycable-go + RPC server
|
|
236
227
|
(`frontend/anycable_probe.mjs`, `anycable_concurrent.mjs`): liveness, the
|
|
237
|
-
|
|
228
|
+
yrb-lite client provider, cross-process reads, and concurrent convergence.
|
|
238
229
|
|
|
239
230
|
The wire format is the standard y-protocols binary messages, base64-encoded in
|
|
240
|
-
the ActionCable envelope.
|
|
241
|
-
`{ "update" => ... }
|
|
242
|
-
message per frame, so the off-the-shelf provider works with no custom client
|
|
243
|
-
code:
|
|
231
|
+
the ActionCable envelope. yrb-lite uses one canonical document envelope,
|
|
232
|
+
`{ "update" => ... }`, and sends one message per frame.
|
|
244
233
|
|
|
245
234
|
```js
|
|
246
|
-
import { createConsumer } from "@
|
|
247
|
-
import {
|
|
235
|
+
import { createConsumer } from "@anycable/web"
|
|
236
|
+
import { ActionCableProvider } from "yrb-lite-client"
|
|
248
237
|
|
|
249
|
-
const provider = new
|
|
238
|
+
const provider = new ActionCableProvider(ydoc, createConsumer(), "DocumentChannel", { id: docId })
|
|
239
|
+
provider.connect()
|
|
250
240
|
```
|
|
251
241
|
|
|
252
242
|
[`examples/actioncable-demo`](examples/actioncable-demo) is a full Rails + Tiptap
|
|
253
|
-
app using
|
|
243
|
+
app using the yrb-lite provider, with end-to-end tests.
|
|
254
244
|
|
|
255
|
-
####
|
|
245
|
+
#### Record Before Distribute
|
|
256
246
|
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
auditing or to guarantee nothing is distributed until it's stored, register an
|
|
260
|
-
`on_change` recorder:
|
|
247
|
+
Every document change is durably recorded before anyone else sees it. Register
|
|
248
|
+
an `on_change` recorder:
|
|
261
249
|
|
|
262
250
|
```ruby
|
|
263
251
|
class DocumentChannel < ApplicationCable::Channel
|
|
264
|
-
include YrbLite::Sync
|
|
252
|
+
include YrbLite::ActionCable::Sync
|
|
265
253
|
|
|
266
254
|
on_change do |key, update|
|
|
267
255
|
# Synchronous, durable write. `update` is the exact CRDT delta.
|
|
@@ -269,78 +257,43 @@ class DocumentChannel < ApplicationCable::Channel
|
|
|
269
257
|
end
|
|
270
258
|
|
|
271
259
|
def subscribed = sync_for(params[:id])
|
|
272
|
-
def receive(data) = sync_receive(data)
|
|
273
|
-
def unsubscribed =
|
|
260
|
+
def receive(data) = sync_receive(data, params[:id])
|
|
261
|
+
def unsubscribed = sync_unsubscribed(params[:id])
|
|
274
262
|
end
|
|
275
263
|
```
|
|
276
264
|
|
|
277
265
|
With `on_change` registered, a change is recorded before it goes anywhere. The
|
|
278
266
|
recorder writes the raw CRDT delta synchronously; only then is the change
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
it's applied. That's what makes the log authoritative. Replay the deltas onto a
|
|
282
|
-
fresh `Y.Doc` and you get the document back exactly.
|
|
267
|
+
broadcast. Replay the deltas onto a fresh `Y.Doc` and you get the document back
|
|
268
|
+
exactly.
|
|
283
269
|
|
|
284
270
|
If the recorder raises (say the store is down), the change is rejected: not
|
|
285
271
|
applied, not sent to anyone. The cost is a synchronous durable write per change,
|
|
286
272
|
which serializes that document's writes. Other documents use other locks and run
|
|
287
273
|
in parallel.
|
|
288
274
|
|
|
289
|
-
`on_change`
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
`on_change` to an fsync'd append-only log and checks, end to end, that the log
|
|
293
|
-
alone rebuilds the document.
|
|
275
|
+
The demo wires `on_change` to a durable Postgres-backed log by default, with an
|
|
276
|
+
fsync'd file log available via `STORE_KIND=file`, and checks end to end that the
|
|
277
|
+
log alone rebuilds the document.
|
|
294
278
|
|
|
295
279
|
#### Reliable delivery (acks)
|
|
296
280
|
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
state everyone *has*; they don't recover an update that never arrived.
|
|
302
|
-
|
|
303
|
-
yrb-lite closes that gap with an opt-in, client-driven acknowledgement. If an
|
|
304
|
-
incoming frame carries an `"id"`, the server replies `{ "ack": <id> }` once the
|
|
305
|
-
update has been **accepted** -- recorded in audit mode, applied in fast mode. A
|
|
306
|
-
causally-gapped update is not acked (it gets a resync instead), so the client
|
|
307
|
-
knows it hasn't landed yet.
|
|
281
|
+
yrb-lite document delivery is ack-tracked. Browser document updates carry an
|
|
282
|
+
`"id"`, and the server replies `{ "ack": <id> }` once the update has been
|
|
283
|
+
**durably recorded**. A causally-gapped update is not acked; the server sends a
|
|
284
|
+
resync request, and the client keeps the update queued until it lands.
|
|
308
285
|
|
|
309
286
|
```
|
|
310
|
-
client -> server { "
|
|
287
|
+
client -> server { "update": "<base64 update>", "id": 42 }
|
|
311
288
|
server -> client { "ack": 42 } # update accepted; safe to forget
|
|
312
289
|
```
|
|
313
290
|
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
This is entirely **self-gating**: stock clients send no `"id"`, so they never get
|
|
322
|
-
acks and behave exactly as before. Only a client that opts in by tagging its
|
|
323
|
-
frames participates.
|
|
324
|
-
|
|
325
|
-
Two client examples ship in the demo:
|
|
326
|
-
|
|
327
|
-
- [`frontend/reliable.mjs`](examples/actioncable-demo/frontend/reliable.mjs) — a
|
|
328
|
-
minimal reference client showing the raw mechanism (tag with an id, retain,
|
|
329
|
-
retransmit on a timer, drain on ack), with an end-to-end test that loses an
|
|
330
|
-
update mid-flight and recovers it purely by retransmit.
|
|
331
|
-
- [`frontend/provider/reliable_actioncable_provider.mjs`](examples/actioncable-demo/frontend/provider/reliable_actioncable_provider.mjs)
|
|
332
|
-
— the standard `@y-rb/actioncable` `WebsocketProvider`, vendored and augmented
|
|
333
|
-
for production use. It's a drop-in replacement that speaks the same protocol
|
|
334
|
-
and envelope, and adds reliability with **sync-since-last-ack** framing: rather
|
|
335
|
-
than retransmitting updates one by one, it keeps the unacknowledged local
|
|
336
|
-
updates in a queue and sends their *merge* as a single causally-complete delta,
|
|
337
|
-
with the id being the highest sequence in the batch (so one `{ ack: id }`
|
|
338
|
-
cumulatively confirms everything up to it). Because the whole unacked tail goes
|
|
339
|
-
as one self-contained delta, the server never sees an internal gap and never
|
|
340
|
-
has to round-trip a resync for a lost middle update — the next edit, or the
|
|
341
|
-
next timer tick, carries it. Awareness stays fire-and-forget; against a server
|
|
342
|
-
that doesn't implement acks it warns once and falls back to plain delivery; and
|
|
343
|
-
`reliable: false` opts out entirely. The demo's editor uses this provider.
|
|
291
|
+
`yrb-lite-client`'s `ActionCableProvider` handles this automatically. It keeps
|
|
292
|
+
the unacknowledged local document tail in a queue and sends the merged tail as a
|
|
293
|
+
single causally-complete delta. The id is the highest sequence in the batch, so
|
|
294
|
+
one `{ ack: id }` cumulatively confirms everything up to it. Because CRDT apply
|
|
295
|
+
is idempotent, a resend that already landed is a harmless no-op that just
|
|
296
|
+
re-acks. Awareness stays ephemeral and is not acked.
|
|
344
297
|
|
|
345
298
|
### User Awareness/Presence
|
|
346
299
|
|
|
@@ -376,24 +329,24 @@ message = awareness.encode_update(update_bytes)
|
|
|
376
329
|
|
|
377
330
|
## Thread Safety
|
|
378
331
|
|
|
379
|
-
|
|
380
|
-
`
|
|
381
|
-
|
|
332
|
+
`Doc` and `Awareness` are safe to share across Ruby threads. A `Doc` or
|
|
333
|
+
`Awareness` can be used concurrently from Puma workers, ActionCable connection
|
|
334
|
+
threads, or background jobs without external locking.
|
|
382
335
|
|
|
383
336
|
That comes from how the underlying types work, not from locking on top:
|
|
384
337
|
|
|
385
338
|
- `yrs::Doc` is `Send + Sync`. Every operation takes the document's internal
|
|
386
339
|
RwLock with blocking semantics (`read_blocking`/`write_blocking`), so
|
|
387
340
|
concurrent access serializes instead of erroring or corrupting state.
|
|
388
|
-
- `yrs::sync::Awareness` is
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
Each native method opens and closes its transaction
|
|
393
|
-
|
|
394
|
-
-
|
|
395
|
-
a yrs upgrade regressed
|
|
396
|
-
turning thread-unsafe.
|
|
341
|
+
- `yrs::sync::Awareness` is `Send` but not `Sync` in the current yrs version,
|
|
342
|
+
so the Ruby wrapper stores it in a `Mutex`. The mutex is always acquired
|
|
343
|
+
inside the no-GVL native section and released before Ruby runs again.
|
|
344
|
+
- The extension uses no `RefCell`-style runtime borrows that could panic under
|
|
345
|
+
re-entrancy. Each native method opens and closes its transaction or mutex
|
|
346
|
+
guard inside one call.
|
|
347
|
+
- Static assertions in `lib.rs` prove `Doc` and `Mutex<Awareness>` are
|
|
348
|
+
`Send + Sync`. If a yrs upgrade regressed either wrapper's thread-safety, the
|
|
349
|
+
gem would fail to compile instead of quietly turning thread-unsafe.
|
|
397
350
|
|
|
398
351
|
`test/thread_safety_test.rb` runs shared docs, the full sync handshake, fan-in
|
|
399
352
|
sync, and awareness state across 8 threads at once, and checks the interleaving
|
|
@@ -414,11 +367,12 @@ A slow operation also can't stall the VM. A thread applying a large update holds
|
|
|
414
367
|
the doc's write lock without holding the GVL, so other Ruby threads keep running
|
|
415
368
|
instead of queuing behind it.
|
|
416
369
|
|
|
417
|
-
Each method has the same shape: copy
|
|
418
|
-
yrs work
|
|
419
|
-
the GVL back, then build Ruby objects. No Ruby API is touched
|
|
420
|
-
|
|
421
|
-
deadlock. Panics in native code are caught and re-raised as Ruby
|
|
370
|
+
Each method has the same shape: copy Ruby byte strings first, drop the GVL, do
|
|
371
|
+
the yrs work while taking and releasing native locks entirely inside the
|
|
372
|
+
closure, take the GVL back, then build Ruby objects. No Ruby API is touched
|
|
373
|
+
without the GVL, and no native lock is held while reacquiring it, so the lock
|
|
374
|
+
order can't deadlock. Panics in native code are caught and re-raised as Ruby
|
|
375
|
+
exceptions.
|
|
422
376
|
|
|
423
377
|
## Message Type Constants
|
|
424
378
|
|
|
@@ -11,16 +11,15 @@ module YrbLite::ActionCable # rubocop:disable Style/ClassAndModuleChildren
|
|
|
11
11
|
# (and awareness/presence) with browser clients. Messages are the standard
|
|
12
12
|
# y-protocols binary messages, base64-encoded in a JSON envelope:
|
|
13
13
|
#
|
|
14
|
-
# { "
|
|
15
|
-
# { "
|
|
14
|
+
# { "update" => "<base64 bytes>", "id" => 42 } # client -> server
|
|
15
|
+
# { "update" => "...", "origin" => "<id>" } # server -> subscribers
|
|
16
|
+
# { "ack" => 42 } # server -> sender
|
|
16
17
|
#
|
|
17
18
|
# Example:
|
|
18
19
|
# class DocumentChannel < ApplicationCable::Channel
|
|
19
20
|
# include YrbLite::ActionCable::Sync
|
|
20
21
|
#
|
|
21
22
|
# on_load { |key| Document.find_by(key: key)&.content }
|
|
22
|
-
# on_save { |key, update| Document.find_by(key: key)&.update!(content: update) }
|
|
23
|
-
#
|
|
24
23
|
# # on_change blocks run in the channel instance's context, so instance
|
|
25
24
|
# # methods (current_user, params, ...) are available without plumbing:
|
|
26
25
|
# on_change { |key, update| Document.record!(key, update, by: current_user) }
|
|
@@ -34,13 +33,13 @@ module YrbLite::ActionCable # rubocop:disable Style/ClassAndModuleChildren
|
|
|
34
33
|
# end
|
|
35
34
|
#
|
|
36
35
|
# def unsubscribed
|
|
37
|
-
#
|
|
36
|
+
# sync_unsubscribed
|
|
38
37
|
# end
|
|
39
38
|
# end
|
|
40
39
|
#
|
|
41
|
-
# The
|
|
42
|
-
#
|
|
43
|
-
#
|
|
40
|
+
# The concern is store-backed and fail-closed: every document update is
|
|
41
|
+
# validated against `on_load`, recorded through `on_change`, then broadcast.
|
|
42
|
+
# No authoritative document state is kept in ActionCable process memory.
|
|
44
43
|
module Sync
|
|
45
44
|
# Validated frame kinds from Awareness#message_kind. A frame only gets a
|
|
46
45
|
# non-DROP kind if it is exactly one well-formed message; anything
|
|
@@ -52,6 +51,11 @@ module YrbLite::ActionCable # rubocop:disable Style/ClassAndModuleChildren
|
|
|
52
51
|
MSG_KIND_AWARENESS = 3
|
|
53
52
|
MSG_KIND_AWARENESS_QUERY = 4
|
|
54
53
|
|
|
54
|
+
# Default incoming-frame size cap (decoded bytes). Generous enough for a
|
|
55
|
+
# large initial SyncStep2, small enough to bound a single message's
|
|
56
|
+
# allocation/parse cost. Override per channel with `max_frame_bytes`.
|
|
57
|
+
DEFAULT_MAX_FRAME_BYTES = 8 * 1024 * 1024
|
|
58
|
+
|
|
55
59
|
def self.included(base)
|
|
56
60
|
base.extend(ClassMethods)
|
|
57
61
|
end
|
|
@@ -61,272 +65,118 @@ module YrbLite::ActionCable # rubocop:disable Style/ClassAndModuleChildren
|
|
|
61
65
|
# return a binary Y.js update (or nil for a fresh document).
|
|
62
66
|
def on_load(callable = nil, &block)
|
|
63
67
|
@on_load = callable || block if callable || block
|
|
64
|
-
@on_load
|
|
65
|
-
end
|
|
68
|
+
return @on_load if defined?(@on_load) && @on_load
|
|
66
69
|
|
|
67
|
-
|
|
68
|
-
# message that modified the document.
|
|
69
|
-
def on_save(callable = nil, &block)
|
|
70
|
-
@on_save = callable || block if callable || block
|
|
71
|
-
@on_save
|
|
70
|
+
superclass.respond_to?(:on_load) ? superclass.on_load : nil
|
|
72
71
|
end
|
|
73
72
|
|
|
74
73
|
# Record every document change durably before it is applied or
|
|
75
|
-
# distributed (
|
|
76
|
-
#
|
|
77
|
-
#
|
|
78
|
-
# the change is rejected: neither applied to the shared document nor
|
|
79
|
-
# broadcast to other subscribers.
|
|
74
|
+
# distributed. Called synchronously with (key, update), where update is
|
|
75
|
+
# the exact CRDT delta. If the block raises, the change is rejected:
|
|
76
|
+
# neither acknowledged nor broadcast to other subscribers.
|
|
80
77
|
#
|
|
81
78
|
# A block recorder runs in the *channel instance's* context, so it can
|
|
82
79
|
# call the channel's own methods (current_user, params, a per-connection
|
|
83
80
|
# Current.* accessor) directly, with no thread-local plumbing. (A non-Proc
|
|
84
81
|
# callable is invoked with #call instead, since it carries its own
|
|
85
|
-
# context.) on_change always fires from within sync_receive
|
|
86
|
-
# on_load/on_save, which can run context-free in the shared registry.
|
|
87
|
-
#
|
|
88
|
-
# Registering an on_change switches that channel onto the strict path
|
|
89
|
-
# (record, apply, broadcast). Without it, the default fast path applies
|
|
90
|
-
# and broadcasts, with an optional on_save snapshot.
|
|
82
|
+
# context.) on_change always fires from within sync_receive.
|
|
91
83
|
def on_change(callable = nil, &block)
|
|
92
84
|
@on_change = callable || block if callable || block
|
|
93
|
-
@on_change
|
|
85
|
+
return @on_change if defined?(@on_change) && @on_change
|
|
86
|
+
|
|
87
|
+
superclass.respond_to?(:on_change) ? superclass.on_change : nil
|
|
94
88
|
end
|
|
95
89
|
|
|
96
|
-
#
|
|
97
|
-
#
|
|
98
|
-
#
|
|
99
|
-
#
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
# affinity) and across processes. Requires `on_load` and `on_change`.
|
|
106
|
-
def sync_backend(mode = nil)
|
|
107
|
-
@sync_backend = mode if mode
|
|
108
|
-
@sync_backend || :memory
|
|
90
|
+
# Maximum size, in decoded bytes, of an incoming document/awareness frame.
|
|
91
|
+
# Oversized frames are dropped before base64 decode and before native
|
|
92
|
+
# parsing, so a client can't force huge allocations/CPU (a DoS vector).
|
|
93
|
+
# Defaults to DEFAULT_MAX_FRAME_BYTES; set to nil to disable the cap.
|
|
94
|
+
def max_frame_bytes(bytes = :__unset__)
|
|
95
|
+
@max_frame_bytes = bytes unless bytes == :__unset__
|
|
96
|
+
return @max_frame_bytes if defined?(@max_frame_bytes)
|
|
97
|
+
|
|
98
|
+
superclass.respond_to?(:max_frame_bytes) ? superclass.max_frame_bytes : DEFAULT_MAX_FRAME_BYTES
|
|
109
99
|
end
|
|
110
100
|
end
|
|
111
101
|
|
|
112
102
|
# Call from `subscribed`. Streams broadcasts for this document and
|
|
113
|
-
# transmits the server's opening handshake (SyncStep1
|
|
103
|
+
# transmits the server's opening handshake (SyncStep1 from the store).
|
|
114
104
|
def sync_for(key)
|
|
115
105
|
@sync_key = key.to_s
|
|
116
106
|
@sync_origin = SecureRandom.hex(8)
|
|
117
|
-
|
|
107
|
+
sync_require_store_recorder!
|
|
118
108
|
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
awareness = sync_awareness
|
|
123
|
-
|
|
124
|
-
sync_stream sync_stream_name, coder: ActiveSupport::JSON do |payload|
|
|
125
|
-
sync_on_broadcast(payload)
|
|
126
|
-
end
|
|
127
|
-
|
|
128
|
-
# Opening handshake: SyncStep1 then the current awareness, each as its
|
|
129
|
-
# own single-message frame, so providers that parse one message per frame
|
|
130
|
-
# (e.g. @y-rb/actioncable) handle both. The client replies SyncStep2 to
|
|
131
|
-
# the SyncStep1, delivering its state to the server.
|
|
132
|
-
sync_transmit(awareness.sync_step1)
|
|
133
|
-
sync_transmit(awareness.encode_awareness_update)
|
|
109
|
+
sync_stream sync_stream_name
|
|
110
|
+
sync_stream sync_awareness_stream_name, whisper: true if respond_to?(:whispers_to)
|
|
111
|
+
sync_transmit(sync_load_doc.sync_step1)
|
|
134
112
|
end
|
|
135
113
|
|
|
136
114
|
# Call from `receive`. Applies the client's message, replies directly
|
|
137
115
|
# when the protocol calls for it, and relays document/awareness changes
|
|
138
116
|
# to the other subscribers.
|
|
139
117
|
#
|
|
140
|
-
#
|
|
141
|
-
#
|
|
142
|
-
#
|
|
143
|
-
#
|
|
144
|
-
# Reliable delivery (opt-in, client-driven): if the frame carries an "id",
|
|
145
|
-
# the server replies `{ "ack" => id }` once the update has been accepted
|
|
146
|
-
# (recorded in audit mode, applied in fast mode). A causally-gapped update
|
|
147
|
-
# is not acked -- it gets a resync instead -- so an ack-aware client knows
|
|
148
|
-
# to retransmit until the update lands. Stock clients send no "id", never
|
|
149
|
-
# get acks, and are completely unaffected.
|
|
118
|
+
# Reliable delivery: document updates carry an "id", and the server replies
|
|
119
|
+
# `{ "ack" => id }` once the update has been durably recorded. A
|
|
120
|
+
# causally-gapped update is not acked -- it gets a resync instead -- so the
|
|
121
|
+
# client retransmits until the update lands.
|
|
150
122
|
def sync_receive(data, key = nil)
|
|
151
123
|
# Pass `key` (params[:id]) when your transport doesn't keep the channel
|
|
152
124
|
# instance alive across actions. Under AnyCable each RPC command gets a
|
|
153
125
|
# fresh channel, so instance variables set in `subscribed` are gone here.
|
|
154
126
|
@sync_key = key.to_s if key
|
|
155
127
|
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
m = data.is_a?(Hash) ? (data["m"] || data["update"]) : nil
|
|
159
|
-
return unless m.is_a?(String)
|
|
128
|
+
encoded = data.is_a?(Hash) ? data["update"] : nil
|
|
129
|
+
return unless encoded.is_a?(String)
|
|
160
130
|
|
|
161
131
|
# Optional client-supplied id for reliable delivery (see sync_send_ack).
|
|
162
132
|
id = data.is_a?(Hash) ? data["id"] : nil
|
|
163
133
|
|
|
134
|
+
# Frame-size cap: drop oversized frames before decoding (the encoded form
|
|
135
|
+
# is ~4/3 the decoded size) and again after, so a client can't force large
|
|
136
|
+
# base64 decodes / native parses / merges. A dropped frame is never acked.
|
|
137
|
+
cap = self.class.max_frame_bytes
|
|
138
|
+
return if cap && encoded.bytesize > (cap * 4 / 3) + 4
|
|
139
|
+
|
|
164
140
|
begin
|
|
165
|
-
bytes = Base64.strict_decode64(
|
|
141
|
+
bytes = Base64.strict_decode64(encoded)
|
|
166
142
|
rescue ArgumentError
|
|
167
143
|
return # not valid base64; ignore the frame and keep the connection
|
|
168
144
|
end
|
|
169
145
|
|
|
170
|
-
|
|
146
|
+
return if cap && bytes.bytesize > cap
|
|
147
|
+
|
|
148
|
+
sync_send_ack(id, sync_dispatch(encoded, bytes))
|
|
171
149
|
end
|
|
172
150
|
|
|
173
151
|
# Route a decoded frame to the backend/path that handles it and return the
|
|
174
152
|
# outcome symbol (:recorded/:applied/:gap/:noop) used by the reliable-
|
|
175
153
|
# delivery ack. A dropped frame returns nil (never acked).
|
|
176
154
|
def sync_dispatch(encoded, bytes)
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
awareness = sync_awareness
|
|
180
|
-
kind = awareness.message_kind(bytes)
|
|
181
|
-
# Malformed / truncated / multi-message / unknown frames are dropped
|
|
182
|
-
# before they can be processed or relayed to other clients.
|
|
183
|
-
return if kind == MSG_KIND_DROP
|
|
184
|
-
|
|
185
|
-
sync_track_clients(awareness, bytes) if kind == MSG_KIND_AWARENESS
|
|
186
|
-
|
|
187
|
-
if kind == MSG_KIND_UPDATE && self.class.on_change
|
|
188
|
-
sync_apply_authoritative(awareness, encoded, bytes)
|
|
189
|
-
else
|
|
190
|
-
sync_apply_fast(awareness, encoded, bytes, kind)
|
|
191
|
-
end
|
|
192
|
-
end
|
|
193
|
-
|
|
194
|
-
# Call from `unsubscribed`. Clears the presence states this connection
|
|
195
|
-
# introduced and tells the other subscribers to drop those cursors, so a
|
|
196
|
-
# closed tab or dropped socket doesn't leave a ghost cursor behind until
|
|
197
|
-
# the client-side timeout reaps it.
|
|
198
|
-
def sync_clear_presence
|
|
199
|
-
return if @sync_clients.nil? || @sync_clients.empty?
|
|
200
|
-
|
|
201
|
-
removal = sync_awareness.remove_clients(@sync_clients)
|
|
202
|
-
@sync_clients = []
|
|
203
|
-
return if removal.empty?
|
|
204
|
-
|
|
205
|
-
sync_distribute(Base64.strict_encode64(removal))
|
|
155
|
+
sync_receive_store_backed(encoded, bytes)
|
|
206
156
|
end
|
|
207
157
|
|
|
208
|
-
#
|
|
209
|
-
#
|
|
210
|
-
# memory (only when an `on_load` is configured to bring it back; otherwise
|
|
211
|
-
# the in-memory document is the only copy and is kept). Prevents a
|
|
212
|
-
# long-running server from accumulating every document it has ever served.
|
|
158
|
+
# Kept as the ActionCable lifecycle hook target. There is no cached document
|
|
159
|
+
# or server-owned presence state to clean up in the store-backed design.
|
|
213
160
|
def sync_unsubscribed(key = nil)
|
|
214
161
|
@sync_key = key.to_s if key
|
|
215
|
-
return if self.class.sync_backend == :store # nothing cached per process
|
|
216
|
-
|
|
217
|
-
sync_clear_presence
|
|
218
|
-
saver = self.class.on_save
|
|
219
|
-
Sync.release(@sync_key, evictable: !self.class.on_load.nil?) do |awareness|
|
|
220
|
-
saver&.call(@sync_key, awareness.encode_state_as_update)
|
|
221
|
-
end
|
|
222
|
-
end
|
|
223
|
-
|
|
224
|
-
# The shared Awareness (document + presence) for this channel's key.
|
|
225
|
-
# Also useful for server-side reads, e.g.:
|
|
226
|
-
# sync_awareness.encode_state_as_update
|
|
227
|
-
def sync_awareness
|
|
228
|
-
Sync.awareness_for(@sync_key, self.class.on_load)
|
|
229
162
|
end
|
|
230
163
|
|
|
231
164
|
private
|
|
232
165
|
|
|
233
|
-
# Default path: apply the message, answer direct requests, relay
|
|
234
|
-
# state-changing messages to the other subscribers. Routing comes from the
|
|
235
|
-
# native `kind` (from Awareness#message_kind) rather than peeking at bytes.
|
|
236
|
-
# Document changes (SyncStep2, Update) and awareness get relayed; requests
|
|
237
|
-
# (SyncStep1, awareness-query) are answered above and not relayed. An
|
|
238
|
-
# optional on_save snapshot is taken after a document change.
|
|
239
|
-
#
|
|
240
|
-
# Returns an outcome symbol for the reliable-delivery ack: :applied when a
|
|
241
|
-
# document update was integrated and relayed, :gap when it was rejected for
|
|
242
|
-
# a resync, :noop for everything else (requests, awareness, empty updates).
|
|
243
|
-
def sync_apply_fast(awareness, encoded, bytes, kind)
|
|
244
|
-
# A document update that isn't causally ready (an earlier one was lost in
|
|
245
|
-
# transit) would relay an un-integrable change to peers and stall the
|
|
246
|
-
# replica. Drop it and ask the client to resync instead, which re-delivers
|
|
247
|
-
# the missing piece. See sync_apply_authoritative for the durable variant.
|
|
248
|
-
if kind == MSG_KIND_UPDATE
|
|
249
|
-
update = awareness.update_from_message(bytes)
|
|
250
|
-
# A no-op message (e.g. the empty SyncStep2 in an opening handshake)
|
|
251
|
-
# carries no change, so there's nothing to relay, persist, or ack.
|
|
252
|
-
return :noop unless update
|
|
253
|
-
|
|
254
|
-
unless awareness.update_ready?(update)
|
|
255
|
-
sync_request_resync(awareness)
|
|
256
|
-
return :gap
|
|
257
|
-
end
|
|
258
|
-
end
|
|
259
|
-
|
|
260
|
-
response = awareness.handle(bytes)
|
|
261
|
-
sync_transmit(response) unless response.empty?
|
|
262
|
-
|
|
263
|
-
return :noop unless [MSG_KIND_UPDATE, MSG_KIND_AWARENESS].include?(kind)
|
|
264
|
-
|
|
265
|
-
sync_distribute(encoded)
|
|
266
|
-
return :noop unless kind == MSG_KIND_UPDATE
|
|
267
|
-
|
|
268
|
-
sync_persist
|
|
269
|
-
:applied
|
|
270
|
-
end
|
|
271
|
-
|
|
272
|
-
# Authoritative path: record the change durably, then apply it to the
|
|
273
|
-
# shared document, then distribute it. The sequence runs under a
|
|
274
|
-
# per-document lock so changes are recorded in a single total order that
|
|
275
|
-
# matches the order they're applied, and nothing is distributed (or applied)
|
|
276
|
-
# before it has been recorded. If the recorder raises, the change is
|
|
277
|
-
# rejected (not applied, not broadcast) and the exception propagates, so the
|
|
278
|
-
# channel can surface it and the client can resync.
|
|
279
|
-
#
|
|
280
|
-
# Before recording, the update must be causally ready: every dependency it
|
|
281
|
-
# references must already be in the doc. If an earlier update was lost in
|
|
282
|
-
# transit, or its record failed, a later update arrives with a gap. Recording
|
|
283
|
-
# it would write a permanently-pending entry to the log -- one that can never
|
|
284
|
-
# be replayed until the missing update shows up. Such an update is rejected
|
|
285
|
-
# (not recorded, not applied, not relayed) and the client is asked to resync,
|
|
286
|
-
# which re-delivers the missing range as one causally-complete delta.
|
|
287
|
-
def sync_apply_authoritative(awareness, encoded, bytes)
|
|
288
|
-
recorder = self.class.on_change
|
|
289
|
-
|
|
290
|
-
outcome = Sync.lock_for(@sync_key).synchronize do
|
|
291
|
-
update = awareness.update_from_message(bytes)
|
|
292
|
-
# A no-op message (e.g. the empty SyncStep2 in a client's opening
|
|
293
|
-
# handshake) carries no change, so there's nothing to record or relay.
|
|
294
|
-
next :noop unless update
|
|
295
|
-
next :gap unless awareness.update_ready?(update)
|
|
296
|
-
|
|
297
|
-
sync_record_change(recorder, update) # durable write; raise to reject
|
|
298
|
-
awareness.apply_update(update) # only recorded changes reach the doc
|
|
299
|
-
sync_distribute(encoded) # ...and only then the wire
|
|
300
|
-
:recorded
|
|
301
|
-
end
|
|
302
|
-
|
|
303
|
-
case outcome
|
|
304
|
-
when :recorded then sync_persist
|
|
305
|
-
when :gap then sync_request_resync(awareness)
|
|
306
|
-
end
|
|
307
|
-
|
|
308
|
-
# Surface the outcome for the reliable-delivery ack: :recorded means the
|
|
309
|
-
# update is durably written (and will be acked); :gap triggered a resync
|
|
310
|
-
# (no ack); :noop carried no change.
|
|
311
|
-
outcome
|
|
312
|
-
end
|
|
313
|
-
|
|
314
166
|
# Ask this connection's client to resync: re-send SyncStep1 carrying the
|
|
315
167
|
# server's current (gap-free) state vector. The client replies SyncStep2
|
|
316
168
|
# with everything the server is missing, delivered as one causally-complete
|
|
317
169
|
# delta -- which heals the gap that triggered the resync.
|
|
318
|
-
def sync_request_resync(
|
|
319
|
-
sync_transmit(
|
|
170
|
+
def sync_request_resync(doc)
|
|
171
|
+
sync_transmit(doc.sync_step1)
|
|
320
172
|
end
|
|
321
173
|
|
|
322
174
|
# Reliable delivery: acknowledge an accepted update back to the sending
|
|
323
175
|
# connection. An ack-aware client tags each outgoing update with an "id"
|
|
324
176
|
# and retains it until the matching `{ "ack" => id }` returns, retransmitting
|
|
325
|
-
# on a timer or reconnect; idempotent CRDT apply makes resends free.
|
|
326
|
-
# only
|
|
327
|
-
#
|
|
328
|
-
# mode. A gapped update gets no ack (it got a resync), so the client keeps
|
|
329
|
-
# retransmitting until the missing range lands and the update can integrate.
|
|
177
|
+
# on a timer or reconnect; idempotent CRDT apply makes resends free. Acks
|
|
178
|
+
# are sent only after the update has been durably recorded, or when a retry
|
|
179
|
+
# is already present in the durable store.
|
|
330
180
|
def sync_send_ack(id, outcome)
|
|
331
181
|
return if id.nil?
|
|
332
182
|
return unless %i[recorded applied].include?(outcome)
|
|
@@ -336,11 +186,9 @@ module YrbLite::ActionCable # rubocop:disable Style/ClassAndModuleChildren
|
|
|
336
186
|
transmit({ "ack" => id })
|
|
337
187
|
end
|
|
338
188
|
|
|
339
|
-
# Single broadcast point
|
|
340
|
-
#
|
|
341
|
-
#
|
|
342
|
-
# identifies the sending process (other processes apply it to their own
|
|
343
|
-
# replica; see sync_on_broadcast).
|
|
189
|
+
# Single broadcast point so relay semantics live in one place and tests can
|
|
190
|
+
# observe distribution. Store-backed streams intentionally echo to the
|
|
191
|
+
# sender; applying the same CRDT update twice is a no-op.
|
|
344
192
|
def sync_distribute(encoded)
|
|
345
193
|
ActionCable.server.broadcast(
|
|
346
194
|
sync_stream_name,
|
|
@@ -348,70 +196,39 @@ module YrbLite::ActionCable # rubocop:disable Style/ClassAndModuleChildren
|
|
|
348
196
|
)
|
|
349
197
|
end
|
|
350
198
|
|
|
351
|
-
# Transmit raw protocol bytes to this connection
|
|
199
|
+
# Transmit raw protocol bytes to this connection.
|
|
352
200
|
def sync_transmit(bytes)
|
|
353
201
|
transmit(sync_envelope(Base64.strict_encode64(bytes)))
|
|
354
202
|
end
|
|
355
203
|
|
|
356
|
-
# Build an outgoing envelope.
|
|
357
|
-
# (yrb-lite's own clients) and "update" (the @y-rb/actioncable provider),
|
|
358
|
-
# so either client works against the same server.
|
|
204
|
+
# Build an outgoing envelope.
|
|
359
205
|
def sync_envelope(encoded, extra = {})
|
|
360
|
-
{ "
|
|
206
|
+
{ "update" => encoded }.merge(extra)
|
|
361
207
|
end
|
|
362
208
|
|
|
363
|
-
#
|
|
364
|
-
#
|
|
365
|
-
#
|
|
366
|
-
#
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
209
|
+
# This concern acks updates as *durably recorded*, so it MUST have both a
|
|
210
|
+
# loader (to rebuild the doc and detect causal gaps) and a recorder (to
|
|
211
|
+
# actually persist before acking). Fail closed rather than silently acking
|
|
212
|
+
# and broadcasting updates that were never stored -- which a cold load or
|
|
213
|
+
# reconnect would then lose.
|
|
214
|
+
def sync_require_store_recorder!
|
|
215
|
+
missing = []
|
|
216
|
+
missing << :on_load unless self.class.on_load
|
|
217
|
+
missing << :on_change unless self.class.on_change
|
|
218
|
+
return if missing.empty?
|
|
219
|
+
|
|
220
|
+
raise YrbLite::Error,
|
|
221
|
+
"YrbLite::ActionCable::Sync requires #{missing.join(" and ")}. Updates are acked as " \
|
|
222
|
+
"durably recorded; without a loader and recorder, an ack would claim a persistence " \
|
|
223
|
+
"that never happened, and a cold load would lose the edit."
|
|
370
224
|
end
|
|
371
225
|
|
|
372
|
-
#
|
|
373
|
-
#
|
|
374
|
-
#
|
|
375
|
-
#
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
return unless encoded.is_a?(String)
|
|
379
|
-
|
|
380
|
-
begin
|
|
381
|
-
bytes = Base64.strict_decode64(encoded)
|
|
382
|
-
rescue ArgumentError
|
|
383
|
-
return
|
|
384
|
-
end
|
|
385
|
-
|
|
386
|
-
awareness = sync_awareness
|
|
387
|
-
case awareness.message_kind(bytes)
|
|
388
|
-
when MSG_KIND_UPDATE
|
|
389
|
-
update = awareness.update_from_message(bytes)
|
|
390
|
-
awareness.apply_update(update) if update
|
|
391
|
-
when MSG_KIND_AWARENESS
|
|
392
|
-
awareness.handle(bytes)
|
|
393
|
-
end
|
|
394
|
-
end
|
|
395
|
-
|
|
396
|
-
# -- Store-backed (AnyCable-native) path --------------------------------
|
|
397
|
-
|
|
398
|
-
# Subscribe without a custom block, so AnyCable (which delivers broadcasts
|
|
399
|
-
# outside Ruby) relays them directly. Send the opening SyncStep1 built from
|
|
400
|
-
# the durable store. No warm replica is kept.
|
|
401
|
-
def sync_for_store_backed
|
|
402
|
-
sync_stream sync_stream_name
|
|
403
|
-
sync_transmit(sync_load_doc.sync_step1)
|
|
404
|
-
end
|
|
405
|
-
|
|
406
|
-
# Subscribe to the document's broadcast stream. Under AnyCable (which adds a
|
|
407
|
-
# `whispers_to` method) this also enables client-to-client whispering on the
|
|
408
|
-
# stream, so a client that whispers awareness (any AnyCable consumer can)
|
|
409
|
-
# reaches other subscribers with no server round-trip. On plain ActionCable
|
|
410
|
-
# the option is omitted -- there's no whisper support -- so presence is
|
|
411
|
-
# server-relayed. It's automatic either way; nothing to configure.
|
|
412
|
-
def sync_stream(name, **opts, &)
|
|
413
|
-
opts[:whisper] = true if respond_to?(:whispers_to)
|
|
414
|
-
stream_from(name, **opts, &)
|
|
226
|
+
# Subscribe to a broadcast stream. The document stream is never whisper-
|
|
227
|
+
# enabled; when AnyCable is present we separately subscribe an awareness
|
|
228
|
+
# stream with `whisper: true`, so the client-to-client path is scoped to
|
|
229
|
+
# ephemeral presence instead of the durable document stream.
|
|
230
|
+
def sync_stream(name, **, &)
|
|
231
|
+
stream_from(name, **, &)
|
|
415
232
|
end
|
|
416
233
|
|
|
417
234
|
# Stateless per message: no warm replica, no assumptions about which process
|
|
@@ -424,6 +241,8 @@ module YrbLite::ActionCable # rubocop:disable Style/ClassAndModuleChildren
|
|
|
424
241
|
# document update was durably recorded and relayed, :gap when it was
|
|
425
242
|
# rejected for a resync, :noop for everything else.
|
|
426
243
|
def sync_receive_store_backed(encoded, bytes)
|
|
244
|
+
sync_require_store_recorder!
|
|
245
|
+
|
|
427
246
|
case Sync.codec.message_kind(bytes)
|
|
428
247
|
when MSG_KIND_SYNC_STEP1
|
|
429
248
|
result = sync_load_doc.handle_sync_message(bytes)
|
|
@@ -433,23 +252,24 @@ module YrbLite::ActionCable # rubocop:disable Style/ClassAndModuleChildren
|
|
|
433
252
|
update = Sync.codec.update_from_message(bytes)
|
|
434
253
|
return :noop unless update
|
|
435
254
|
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
255
|
+
Sync.lock_for(@sync_key).synchronize do
|
|
256
|
+
# To tell whether this update is causally ready we rebuild the doc
|
|
257
|
+
# from the store and check against it. That's an O(history) load per
|
|
258
|
+
# update, mitigated by snapshotting in the app's load path if needed.
|
|
259
|
+
doc = sync_load_doc
|
|
260
|
+
unless doc.update_ready?(update)
|
|
261
|
+
sync_request_resync(doc)
|
|
262
|
+
next :gap
|
|
263
|
+
end
|
|
264
|
+
|
|
265
|
+
# Lost-ack retry: the durable store already contains this update.
|
|
266
|
+
# Ack it without recording or relaying again.
|
|
267
|
+
next :applied unless doc.update_advances?(update)
|
|
447
268
|
|
|
448
|
-
|
|
449
|
-
|
|
269
|
+
sync_record_change(self.class.on_change, update) # record before relay
|
|
270
|
+
sync_distribute(encoded)
|
|
271
|
+
:recorded
|
|
450
272
|
end
|
|
451
|
-
sync_distribute(encoded)
|
|
452
|
-
:recorded
|
|
453
273
|
when MSG_KIND_AWARENESS
|
|
454
274
|
sync_distribute(encoded)
|
|
455
275
|
:noop
|
|
@@ -466,23 +286,12 @@ module YrbLite::ActionCable # rubocop:disable Style/ClassAndModuleChildren
|
|
|
466
286
|
doc
|
|
467
287
|
end
|
|
468
288
|
|
|
469
|
-
# Record the awareness client IDs carried by an incoming message (already
|
|
470
|
-
# known to be an awareness frame) so we can clear them when this connection
|
|
471
|
-
# closes.
|
|
472
|
-
def sync_track_clients(awareness, bytes)
|
|
473
|
-
awareness.awareness_client_ids(bytes).each do |id|
|
|
474
|
-
@sync_clients << id unless @sync_clients.include?(id)
|
|
475
|
-
end
|
|
476
|
-
end
|
|
477
|
-
|
|
478
289
|
def sync_stream_name
|
|
479
290
|
"yrb_lite:#{@sync_key}"
|
|
480
291
|
end
|
|
481
292
|
|
|
482
|
-
def
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
saver.call(@sync_key, sync_awareness.encode_state_as_update)
|
|
293
|
+
def sync_awareness_stream_name
|
|
294
|
+
"#{sync_stream_name}:awareness"
|
|
486
295
|
end
|
|
487
296
|
|
|
488
297
|
# Invoke the on_change recorder. A block/proc runs in this channel instance's
|
|
@@ -493,11 +302,9 @@ module YrbLite::ActionCable # rubocop:disable Style/ClassAndModuleChildren
|
|
|
493
302
|
recorder.is_a?(Proc) ? instance_exec(*args, &recorder) : recorder.call(*args)
|
|
494
303
|
end
|
|
495
304
|
|
|
496
|
-
# -- Shared
|
|
305
|
+
# -- Shared process state ----------------------------------------------
|
|
497
306
|
|
|
498
|
-
@registry = {}
|
|
499
307
|
@locks = {}
|
|
500
|
-
@subscribers = Hash.new(0)
|
|
501
308
|
@registry_mutex = Mutex.new
|
|
502
309
|
|
|
503
310
|
class << self
|
|
@@ -515,76 +322,19 @@ module YrbLite::ActionCable # rubocop:disable Style/ClassAndModuleChildren
|
|
|
515
322
|
@codec ||= YrbLite::Awareness.new
|
|
516
323
|
end
|
|
517
324
|
|
|
518
|
-
#
|
|
519
|
-
#
|
|
520
|
-
# subscribers can never observe two documents for one key; all
|
|
521
|
-
# subsequent operations run lock-free on the thread-safe native types.
|
|
522
|
-
def awareness_for(key, loader = nil)
|
|
523
|
-
@registry_mutex.synchronize do
|
|
524
|
-
@registry[key] ||= begin
|
|
525
|
-
awareness = YrbLite::Awareness.new
|
|
526
|
-
if loader && (state = loader.call(key))
|
|
527
|
-
awareness.apply_update(state)
|
|
528
|
-
end
|
|
529
|
-
awareness
|
|
530
|
-
end
|
|
531
|
-
end
|
|
532
|
-
end
|
|
533
|
-
|
|
534
|
-
# Per-document mutex serializing the authoritative record -> apply ->
|
|
535
|
-
# broadcast section, so a document's audit log is a single total order.
|
|
325
|
+
# Per-document mutex serializing load -> record -> broadcast within this
|
|
326
|
+
# process. The durable store remains the cross-process source of truth.
|
|
536
327
|
# Only briefly holds the registry mutex to fetch/create the lock; the
|
|
537
328
|
# durable write itself runs while holding only this per-key lock.
|
|
538
329
|
def lock_for(key)
|
|
539
330
|
@registry_mutex.synchronize { @locks[key] ||= Mutex.new }
|
|
540
331
|
end
|
|
541
332
|
|
|
542
|
-
#
|
|
543
|
-
def subscribe(key)
|
|
544
|
-
@registry_mutex.synchronize { @subscribers[key] += 1 }
|
|
545
|
-
end
|
|
546
|
-
|
|
547
|
-
# Drop a subscriber. When the last one leaves and the document is
|
|
548
|
-
# evictable (there's an on_load to bring it back, so unloading can't lose
|
|
549
|
-
# data), persist it via the given block and unload it from memory, so a
|
|
550
|
-
# long-running server doesn't accumulate every document and lock it has
|
|
551
|
-
# ever seen. Returns true if the document was evicted.
|
|
552
|
-
#
|
|
553
|
-
# The persist runs outside the registry lock (it may do I/O), and we
|
|
554
|
-
# re-check the subscriber count afterward: if someone reconnected while
|
|
555
|
-
# we were saving, eviction is aborted and the warm document is kept.
|
|
556
|
-
def release(key, evictable:)
|
|
557
|
-
awareness = @registry_mutex.synchronize do
|
|
558
|
-
@subscribers[key] -= 1 if @subscribers[key].positive?
|
|
559
|
-
next nil unless @subscribers[key].zero?
|
|
560
|
-
|
|
561
|
-
@subscribers.delete(key)
|
|
562
|
-
evictable ? @registry[key] : nil
|
|
563
|
-
end
|
|
564
|
-
return false unless awareness
|
|
565
|
-
|
|
566
|
-
yield awareness if block_given?
|
|
567
|
-
|
|
568
|
-
@registry_mutex.synchronize do
|
|
569
|
-
# A subscriber may have returned during the persist above.
|
|
570
|
-
next false unless @subscribers[key].zero?
|
|
571
|
-
|
|
572
|
-
@subscribers.delete(key)
|
|
573
|
-
@locks.delete(key)
|
|
574
|
-
!@registry.delete(key).nil?
|
|
575
|
-
end
|
|
576
|
-
end
|
|
577
|
-
|
|
578
|
-
def registry
|
|
579
|
-
@registry_mutex.synchronize { @registry.dup }
|
|
580
|
-
end
|
|
581
|
-
|
|
582
|
-
# Clear all documents (useful for testing).
|
|
333
|
+
# Clear process-local locks and codec (useful for testing).
|
|
583
334
|
def reset!
|
|
584
335
|
@registry_mutex.synchronize do
|
|
585
|
-
@registry = {}
|
|
586
336
|
@locks = {}
|
|
587
|
-
@
|
|
337
|
+
@codec = nil
|
|
588
338
|
end
|
|
589
339
|
end
|
|
590
340
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: yrb-lite-actioncable
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.0.
|
|
4
|
+
version: 0.1.0.beta3
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- JP Camara
|
|
@@ -29,14 +29,42 @@ dependencies:
|
|
|
29
29
|
requirements:
|
|
30
30
|
- - ">="
|
|
31
31
|
- !ruby/object:Gem::Version
|
|
32
|
-
version: 0.1.0.
|
|
32
|
+
version: 0.1.0.beta6
|
|
33
33
|
type: :runtime
|
|
34
34
|
prerelease: false
|
|
35
35
|
version_requirements: !ruby/object:Gem::Requirement
|
|
36
36
|
requirements:
|
|
37
37
|
- - ">="
|
|
38
38
|
- !ruby/object:Gem::Version
|
|
39
|
-
version: 0.1.0.
|
|
39
|
+
version: 0.1.0.beta6
|
|
40
|
+
- !ruby/object:Gem::Dependency
|
|
41
|
+
name: actioncable
|
|
42
|
+
requirement: !ruby/object:Gem::Requirement
|
|
43
|
+
requirements:
|
|
44
|
+
- - ">="
|
|
45
|
+
- !ruby/object:Gem::Version
|
|
46
|
+
version: '7.0'
|
|
47
|
+
type: :runtime
|
|
48
|
+
prerelease: false
|
|
49
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
50
|
+
requirements:
|
|
51
|
+
- - ">="
|
|
52
|
+
- !ruby/object:Gem::Version
|
|
53
|
+
version: '7.0'
|
|
54
|
+
- !ruby/object:Gem::Dependency
|
|
55
|
+
name: activesupport
|
|
56
|
+
requirement: !ruby/object:Gem::Requirement
|
|
57
|
+
requirements:
|
|
58
|
+
- - ">="
|
|
59
|
+
- !ruby/object:Gem::Version
|
|
60
|
+
version: '7.0'
|
|
61
|
+
type: :runtime
|
|
62
|
+
prerelease: false
|
|
63
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
64
|
+
requirements:
|
|
65
|
+
- - ">="
|
|
66
|
+
- !ruby/object:Gem::Version
|
|
67
|
+
version: '7.0'
|
|
40
68
|
- !ruby/object:Gem::Dependency
|
|
41
69
|
name: minitest
|
|
42
70
|
requirement: !ruby/object:Gem::Requirement
|