yrb-lite 0.1.0.beta4-aarch64-linux → 0.1.0.beta6-aarch64-linux

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 811a029459e6baa58dd666044e5109b00f9492b3cbca4e9ea19009e4460398ba
4
- data.tar.gz: 38f3b38b56f1da1fc5b03c33e771ee14c89a92d45156b00c342d30ba60800663
3
+ metadata.gz: 2730b0d6a2bc31a36de1f842d7345659788d4c45968a895dfbbf7855ee6be21d
4
+ data.tar.gz: 9731717390fc3e45c983f20dd51429c214a6aaca85cb75b0e375a34390a785e0
5
5
  SHA512:
6
- metadata.gz: 000b629de89198de2cf68d674acb297208d82b6e55760dc7a7252750b6c0868f7e18fda227793168b0904a80dd36b8d597d8ae335e53143b527e992b4cadaec9
7
- data.tar.gz: 974cb16418ca4454daf682fbd0b45b5dad6d413a3db7e978d28d4481b7984473485ce29cc04070c458f62a4756d7a1da3c6525170a6128b922093aae876ab206
6
+ metadata.gz: '0913343e81dd6564a2ce23f058ec63cadb23917a105a86e430d859df77856054a8c0e4c02051a28a0dc8903c916f8c7ea60dd051ae8cbc022064d28539610680'
7
+ data.tar.gz: fb98a8a3e81ec173f812618dce6075a84f6516e4c7991c650c35835958465312b5b0380176a1e5628fdf302de8faa0ec05d3df64b1f78533c96820825ec4d2bc
data/CHANGELOG.md CHANGED
@@ -6,6 +6,53 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [0.1.0.beta6] - 2026-06-22
10
+
11
+ (yrb-lite core gem. The `yrb-lite-client` npm package ships these client changes as 0.1.2.)
12
+
13
+ ### Added
14
+
15
+ - `yrb-lite-client`, the TypeScript client package for the yrb-lite
16
+ ActionCable/AnyCable protocol. It provides `ActionCableProvider`,
17
+ `YProtocolSession`, and the standalone `ReliableSync` delivery core.
18
+
19
+ ### Changed
20
+
21
+ - Document delivery is ack-tracked by default in `yrb-lite-client`: document
22
+ frames use `{ update, id }`, acknowledgements use `{ ack }`, and pending
23
+ document updates stay queued until acked.
24
+ - The ActionCable protocol surface uses a single canonical document envelope:
25
+ `{ "update" => "<base64 frame>" }`.
26
+ - AnyCable awareness/presence uses an awareness-only whisper envelope,
27
+ `{ awareness: "<base64 awareness frame>" }`, while document frames stay on
28
+ the server persistence/ack path.
29
+
30
+ ### Fixed
31
+
32
+ - Incoming protocol frames are validated before mutating documents or awareness
33
+ state, including trailing-byte rejection on the TypeScript client.
34
+ - Native/Rust protocol entry points reject wire client IDs that are unsafe for
35
+ JavaScript clients.
36
+ - `lib0` is declared as a direct runtime dependency of `yrb-lite-client`.
37
+
38
+ ## [0.1.0.beta5] - 2026-06-18
39
+
40
+ ### Changed
41
+
42
+ - **Breaking:** the ActionCable integration has been extracted into a separate
43
+ gem, [`yrb-lite-actioncable`](https://rubygems.org/gems/yrb-lite-actioncable).
44
+ `yrb-lite` is now a standalone y-crdt wrapper: CRDT documents, awareness, and
45
+ the y-websocket sync protocol primitives, with no Rails/ActionCable coupling
46
+ (mirrors the `y-rb` / `yrb-actioncable` split). The `base64` runtime
47
+ dependency moved with it.
48
+
49
+ ### Migration
50
+
51
+ - Using `YrbLite::Sync`? Add `gem "yrb-lite-actioncable"` and change
52
+ `include YrbLite::Sync` to `include YrbLite::ActionCable::Sync`. The concern's
53
+ API is otherwise unchanged. If you only use `YrbLite::Doc`/`YrbLite::Awareness`,
54
+ nothing changes.
55
+
9
56
  ## [0.1.0.beta4] - 2026-06-18
10
57
 
11
58
  ### Changed
@@ -15,9 +62,7 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
15
62
  `current_user`, `params`, request/connection-scoped accessors -- directly,
16
63
  instead of plumbing them in through a thread-local. A non-Proc callable (an
17
64
  object responding to `#call`) is still invoked with `#call` and its own
18
- context. `on_load`/`on_save` are unchanged: they can run in the shared
19
- document registry during a cold load or eviction, where no connection
20
- instance exists, so they remain key-only. Existing block recorders that use
65
+ context. Existing block recorders that use
21
66
  only the `(key, update)` arguments and lexically-scoped constants are
22
67
  unaffected; the only behavioral change is `self` inside the block.
23
68
 
@@ -49,15 +94,13 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
49
94
 
50
95
  - Reliable delivery (opt-in, client-driven). A client may tag a document update
51
96
  with an `"id"`; the server replies `{ "ack": <id> }` once the update has been
52
- accepted (recorded in audit mode, applied in fast mode). This lets an
97
+ durably recorded. This lets an
53
98
  ack-aware client retain and retransmit an update until delivery is confirmed,
54
- so an edit can't be silently lost on a flaky connection. Stock clients send no
55
- `"id"`, never get acks, and behave exactly as before.
56
- - A vendored, ack-aware `@y-rb/actioncable` provider in the demo
57
- (`reliable_actioncable_provider.mjs`) that adds reliable delivery with
58
- "sync-since-last-ack" framing (the unacknowledged tail is sent as one merged,
59
- causally-complete delta), plus a minimal reference client and an intensive
60
- message-loss stress test.
99
+ so an edit can't be silently lost on a flaky connection. Clients that omit
100
+ `"id"` are still accepted, but their delivery is not ack-tracked.
101
+ - Demo coverage for reliable delivery with "sync-since-last-ack" framing (the
102
+ unacknowledged tail is sent as one merged, causally-complete delta), plus a
103
+ minimal reference client and an intensive message-loss stress test.
61
104
 
62
105
  ### Fixed
63
106
 
@@ -77,21 +120,19 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77
120
  native extension). The GVL is released during CRDT work so docs can run in
78
121
  parallel on MRI.
79
122
  - `YrbLite::Sync` ActionCable channel concern implementing the y-websocket
80
- protocol (document sync plus awareness/presence). It's wire-compatible with
81
- the [`@y-rb/actioncable`](https://www.npmjs.com/package/@y-rb/actioncable)
82
- browser provider, and accepts its `{ update: ... }` envelope and `{ m: ... }`.
123
+ protocol (document sync plus awareness/presence).
83
124
  - A "record-before-distribute" mode via an `on_change` hook, so every change is
84
125
  recorded durably before it's applied or relayed.
85
126
  - Presence cleanup on disconnect, and idle-document eviction.
86
- - Two backends: `sync_backend :memory` (default, classic ActionCable) and
87
- `sync_backend :store` (stateless, AnyCable-ready, multi-process).
127
+ - Store-backed ActionCable delivery for AnyCable and multi-process use.
88
128
  - Hardening against bad input: malformed or multi-message frames are dropped
89
129
  before processing or relay, and native panics are contained at the FFI
90
130
  boundary.
91
131
  - Precompiled native gems for common platforms (no Rust toolchain needed to
92
132
  install) via the cross-gem workflow.
93
133
 
94
- [Unreleased]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta4...main
134
+ [Unreleased]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta5...main
135
+ [0.1.0.beta5]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta4...v0.1.0.beta5
95
136
  [0.1.0.beta4]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta3...v0.1.0.beta4
96
137
  [0.1.0.beta3]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta2...v0.1.0.beta3
97
138
  [0.1.0.beta2]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta1...v0.1.0.beta2
data/README.md CHANGED
@@ -9,26 +9,27 @@ documents.
9
9
 
10
10
  ```ruby
11
11
  class DocumentChannel < ApplicationCable::Channel
12
- include YrbLite::Sync
12
+ include YrbLite::ActionCable::Sync
13
13
 
14
14
  def subscribed = sync_for(params[:id])
15
15
  def receive(data) = sync_receive(data)
16
- def unsubscribed = sync_clear_presence
16
+ def unsubscribed = sync_unsubscribed(params[:id])
17
17
  end
18
18
  ```
19
19
 
20
- On the browser, use the [`@y-rb/actioncable`](https://www.npmjs.com/package/@y-rb/actioncable)
21
- provider as-is. Tiptap, ProseMirror, and BlockNote all sync through it.
20
+ On the browser, use the `yrb-lite-client` `ActionCableProvider`. Tiptap,
21
+ ProseMirror, and BlockNote all sync through the `Y.Doc` you pass in and the
22
+ provider's Awareness instance, unless you supply your own.
22
23
 
23
24
  ## What you get
24
25
 
25
- - Thread-safe `Doc` and `Awareness`. You can share them across Puma threads,
26
- and the GVL is released while yrs does the actual work.
26
+ - Thread-safe Ruby wrappers for `Doc` and `Awareness`. You can share them
27
+ across Puma threads; native CRDT work runs with the GVL released.
27
28
  - The y-websocket protocol (document sync plus awareness/presence) as a
28
29
  one-include ActionCable concern.
29
- - A store-backed mode for AnyCable and multi-process deployments.
30
- - An optional authoritative mode that records each change durably before it
31
- goes out to anyone.
30
+ - Store-backed ActionCable/AnyCable delivery for multi-process deployments.
31
+ - Authoritative record-before-distribute semantics: each document change is
32
+ recorded durably before it goes out to anyone.
32
33
 
33
34
  What it doesn't do: auth, read-only connections, rate limiting, webhooks,
34
35
  metrics. Hocuspocus ships extensions for those; here you'd build them with
@@ -36,21 +37,26 @@ Rails.
36
37
 
37
38
  ## Testing
38
39
 
39
- Ruby and Rust unit tests cover the core, and an end-to-end suite runs the real
40
- stack: it fuzzes the protocol, throws garbage and chaos at the server, kills the
41
- server mid-write to check crash recovery, and drives real browsers under load.
42
- The benchmark numbers below are from a single laptop. Issues and PRs are
43
- welcome.
40
+ Ruby and Rust unit tests cover the core. CI also runs the npm client tests and a
41
+ Rails demo smoke slice against the real ActionCable stack. The demo includes
42
+ heavier local suites for hostile input, crash recovery, multi-browser editing,
43
+ AnyCable, and load testing. The benchmark note below is from a single laptop.
44
+ Issues and PRs are welcome.
44
45
 
45
46
  ## Install
46
47
 
47
48
  ```ruby
49
+ # Core CRDT + protocol primitives:
48
50
  gem "yrb-lite"
51
+
52
+ # For the Rails/ActionCable server concern (YrbLite::ActionCable::Sync):
53
+ gem "yrb-lite-actioncable"
49
54
  ```
50
55
 
51
- Requires Ruby 3.4 or newer. Precompiled gems ship for Linux and macOS on Ruby
52
- 3.4 and 4.0, so installing there needs no Rust. Other platforms (and any other
53
- Ruby version) build from source, which needs [Rust](https://rustup.rs).
56
+ Requires Ruby 3.4 or newer. The release workflow builds precompiled gems for
57
+ Ruby 3.4 and 4.0 across the supported Ruby platforms, with native smoke tests
58
+ on Linux x86_64 and macOS arm64. Installing from a matching platform gem needs
59
+ no Rust; a source build needs [Rust](https://rustup.rs).
54
60
 
55
61
  To work on the gem itself:
56
62
 
@@ -128,44 +134,43 @@ send_to_peer(response) unless response.empty?
128
134
 
129
135
  ### ActionCable Integration
130
136
 
131
- `YrbLite::Sync` is a channel concern that implements the full y-websocket
132
- protocol (document sync + awareness/presence) over ActionCable:
137
+ `YrbLite::ActionCable::Sync` (from the `yrb-lite-actioncable` gem) is a channel
138
+ concern that implements the full y-websocket protocol (document sync +
139
+ awareness/presence) over ActionCable:
133
140
 
134
141
  ```ruby
135
142
  # app/channels/document_channel.rb
136
143
  class DocumentChannel < ApplicationCable::Channel
137
- include YrbLite::Sync
144
+ include YrbLite::ActionCable::Sync
138
145
 
139
- # Optional persistence:
140
- # on_load { |key| Document.find_by(key: key)&.content }
141
- # on_save { |key, update| Document.find_by(key: key)&.update!(content: update) }
146
+ on_load { |key| MyStore.load(key) } # source of truth
147
+ on_change { |key, update| MyStore.append(key, update) } # durable record
142
148
 
143
149
  def subscribed
144
150
  sync_for params[:id]
145
151
  end
146
152
 
147
153
  def receive(data)
148
- sync_receive(data)
154
+ sync_receive(data, params[:id])
149
155
  end
150
156
 
151
157
  def unsubscribed
152
- sync_clear_presence
158
+ sync_unsubscribed(params[:id])
153
159
  end
154
160
  end
155
161
  ```
156
162
 
157
- One `YrbLite::Awareness` is shared per document key. Creating it is
158
- mutex-serialized; after that everything runs lock-free on the thread-safe
159
- native types. The concern answers SyncStep1 directly, relays document and
160
- awareness changes to the other subscribers (not back to the sender), and calls
161
- `on_save` after any message that changed the document.
163
+ The concern is store-backed. A handshake is answered from `on_load`; document
164
+ changes are checked against that durable state, recorded through `on_change`,
165
+ then broadcast. Nothing authoritative is kept in ActionCable process memory, so
166
+ AnyCable RPC workers, Puma workers, and separate dynos can all handle messages
167
+ for the same document as long as they share the same store and cable adapter.
162
168
 
163
- `sync_unsubscribed` clears the connection's presence, so a closed tab doesn't
164
- leave a stale cursor hanging until the client-side timeout. It also unloads the
165
- document from memory once the last subscriber disconnects, which keeps the
166
- process from holding onto every document it ever served. That unload only
167
- happens when `on_load` is set and the document can be reloaded later; without
168
- it, the in-memory copy is the only one and stays put.
169
+ `on_load` and `on_change` are required. If either is missing, the channel fails
170
+ closed before it can acknowledge or broadcast edits. Presence is ephemeral:
171
+ awareness frames are relayed, and `yrb-lite-client` sends a best-effort
172
+ presence-removal frame on disconnect/pagehide, with the client-side awareness
173
+ timeout as the fallback for abrupt disconnects.
169
174
 
170
175
  Incoming frames are validated as a single well-formed protocol message before
171
176
  anything processes or relays them. Malformed, truncated, multi-message,
@@ -182,37 +187,19 @@ Broadcasts cross processes through the Action Cable adapter, so it needs to be a
182
187
  real one (`redis` or `solid_cable`, not `async`). With that in place, a change
183
188
  on one process reaches clients on all of them.
184
189
 
185
- Each process also keeps its own copy of the document and applies broadcasts from
186
- the others. The merge is an ordinary CRDT apply, idempotent and
187
- order-independent, which keeps server reads and new-client handshakes current on
188
- every process. Each broadcast carries a per-process id (`Sync.process_id`) that
189
- tells a process to skip its own.
190
-
191
- A cold process (no copy yet) rebuilds from the durable store through `on_load`.
192
- In authoritative mode the store is always current, since changes are recorded
193
- before they go out. Record-before-distribute therefore holds across processes:
194
- whichever process receives a change records it to the shared store before
195
- anyone, anywhere, sees it.
190
+ Every process rebuilds document state from the durable store through `on_load`.
191
+ Because changes are recorded before broadcast, record-before-distribute holds
192
+ across processes: whichever process receives a change records it to the shared
193
+ store before anyone, anywhere, sees it.
196
194
 
197
195
  `bun multiprocess.mjs` in the demo runs clients across two processes and checks
198
- the lot: convergence, fresh copies on both, presence across processes, and one
199
- shared log.
200
-
201
- ##### AnyCable (`sync_backend :store`)
196
+ convergence, fresh reads on both, presence across processes, and one shared log.
202
197
 
203
- The default backend keeps that warm in-memory copy and relies on a `stream_from`
204
- block running in Ruby for each broadcast. AnyCable breaks both assumptions.
205
- anycable-go delivers broadcasts outside Ruby, so the block never runs. Each RPC
206
- gets a fresh channel instance, which means ivars set in `subscribed` are gone by
207
- `receive`. And there's no fixed worker-to-document mapping to lean on.
208
-
209
- `sync_backend :store` is the path for that: stateless per message, no warm
210
- copy.
198
+ ##### AnyCable
211
199
 
212
200
  ```ruby
213
201
  class DocumentChannel < ApplicationCable::Channel
214
- include YrbLite::Sync
215
- sync_backend :store
202
+ include YrbLite::ActionCable::Sync
216
203
 
217
204
  on_load { |key| MyStore.load(key) } # required: source of truth
218
205
  on_change { |key, update| MyStore.append(key, update) } # required: record
@@ -227,6 +214,10 @@ end
227
214
  - A handshake (SyncStep1) is answered from the store. Changes are recorded, then
228
215
  broadcast. Nothing is held in Ruby between calls, so any worker can handle any
229
216
  message.
217
+ - Document frames use the normal server path. Awareness/presence uses a
218
+ separate awareness stream with AnyCable `whisper: true`, so cursor traffic can
219
+ take the low-latency client-to-client path without bypassing document
220
+ durability.
230
221
  - Pass `params[:id]` into `sync_receive`/`sync_unsubscribed` so the document key
231
222
  survives AnyCable's per-command instances.
232
223
  - The sender gets its own updates echoed back (no Ruby callback to filter them).
@@ -234,34 +225,31 @@ end
234
225
 
235
226
  The demo checks this against a real anycable-go + RPC server
236
227
  (`frontend/anycable_probe.mjs`, `anycable_concurrent.mjs`): liveness, the
237
- `@y-rb/actioncable` provider, cross-process reads, and concurrent convergence.
228
+ yrb-lite client provider, cross-process reads, and concurrent convergence.
238
229
 
239
230
  The wire format is the standard y-protocols binary messages, base64-encoded in
240
- the ActionCable envelope. The server accepts the `@y-rb/actioncable` provider's
241
- `{ "update" => ... }` envelope (and its own `{ "m" => ... }`) and sends one
242
- message per frame, so the off-the-shelf provider works with no custom client
243
- code:
231
+ the ActionCable envelope. yrb-lite uses one canonical document envelope,
232
+ `{ "update" => ... }`, and sends one message per frame.
244
233
 
245
234
  ```js
246
- import { createConsumer } from "@rails/actioncable"
247
- import { WebsocketProvider } from "@y-rb/actioncable"
235
+ import { createConsumer } from "@anycable/web"
236
+ import { ActionCableProvider } from "yrb-lite-client"
248
237
 
249
- const provider = new WebsocketProvider(ydoc, createConsumer(), "DocumentChannel", { id: docId })
238
+ const provider = new ActionCableProvider(ydoc, createConsumer(), "DocumentChannel", { id: docId })
239
+ provider.connect()
250
240
  ```
251
241
 
252
242
  [`examples/actioncable-demo`](examples/actioncable-demo) is a full Rails + Tiptap
253
- app using that provider, with end-to-end tests.
243
+ app using the yrb-lite provider, with end-to-end tests.
254
244
 
255
- #### Authoritative audit mode (record before distribute)
245
+ #### Record Before Distribute
256
246
 
257
- By default a change is applied and broadcast immediately (the fast path). If you
258
- need to durably record every change before anyone else sees it, whether for
259
- auditing or to guarantee nothing is distributed until it's stored, register an
260
- `on_change` recorder:
247
+ Every document change is durably recorded before anyone else sees it. Register
248
+ an `on_change` recorder:
261
249
 
262
250
  ```ruby
263
251
  class DocumentChannel < ApplicationCable::Channel
264
- include YrbLite::Sync
252
+ include YrbLite::ActionCable::Sync
265
253
 
266
254
  on_change do |key, update|
267
255
  # Synchronous, durable write. `update` is the exact CRDT delta.
@@ -269,78 +257,43 @@ class DocumentChannel < ApplicationCable::Channel
269
257
  end
270
258
 
271
259
  def subscribed = sync_for(params[:id])
272
- def receive(data) = sync_receive(data)
273
- def unsubscribed = sync_clear_presence
260
+ def receive(data) = sync_receive(data, params[:id])
261
+ def unsubscribed = sync_unsubscribed(params[:id])
274
262
  end
275
263
  ```
276
264
 
277
265
  With `on_change` registered, a change is recorded before it goes anywhere. The
278
266
  recorder writes the raw CRDT delta synchronously; only then is the change
279
- applied to the shared document and broadcast. The whole sequence runs under a
280
- per-document lock, so every change to a document is recorded in the same order
281
- it's applied. That's what makes the log authoritative. Replay the deltas onto a
282
- fresh `Y.Doc` and you get the document back exactly.
267
+ broadcast. Replay the deltas onto a fresh `Y.Doc` and you get the document back
268
+ exactly.
283
269
 
284
270
  If the recorder raises (say the store is down), the change is rejected: not
285
271
  applied, not sent to anyone. The cost is a synchronous durable write per change,
286
272
  which serializes that document's writes. Other documents use other locks and run
287
273
  in parallel.
288
274
 
289
- `on_change` and `on_save` are separate. `on_save` snapshots the whole document
290
- when it gets a chance; `on_change` is the per-change log. The demo's `AUDIT=1`
291
- mode (in [`examples/actioncable-demo`](examples/actioncable-demo)) wires
292
- `on_change` to an fsync'd append-only log and checks, end to end, that the log
293
- alone rebuilds the document.
275
+ The demo wires `on_change` to a durable Postgres-backed log by default, with an
276
+ fsync'd file log available via `STORE_KIND=file`, and checks end to end that the
277
+ log alone rebuilds the document.
294
278
 
295
279
  #### Reliable delivery (acks)
296
280
 
297
- The y-websocket protocol is fire-and-forget. If a client's update is lost in
298
- transit (a flaky socket, a send that never lands) and the client makes no
299
- further edits, the server stays idle and never asks anyone to resync, so that
300
- edit is gone -- even though the client believes it was saved. CRDTs converge the
301
- state everyone *has*; they don't recover an update that never arrived.
302
-
303
- yrb-lite closes that gap with an opt-in, client-driven acknowledgement. If an
304
- incoming frame carries an `"id"`, the server replies `{ "ack": <id> }` once the
305
- update has been **accepted** -- recorded in audit mode, applied in fast mode. A
306
- causally-gapped update is not acked (it gets a resync instead), so the client
307
- knows it hasn't landed yet.
281
+ yrb-lite document delivery is ack-tracked. Browser document updates carry an
282
+ `"id"`, and the server replies `{ "ack": <id> }` once the update has been
283
+ **durably recorded**. A causally-gapped update is not acked; the server sends a
284
+ resync request, and the client keeps the update queued until it lands.
308
285
 
309
286
  ```
310
- client -> server { "m": "<base64 update>", "id": 42 }
287
+ client -> server { "update": "<base64 update>", "id": 42 }
311
288
  server -> client { "ack": 42 } # update accepted; safe to forget
312
289
  ```
313
290
 
314
- That's the whole server side. A reliable client tags each outgoing update with
315
- an incrementing id, keeps it in a pending buffer, and retransmits on a timer (and
316
- on reconnect) until the matching ack returns. Because CRDT apply is idempotent, a
317
- resend that already landed is a harmless no-op that just re-acks. An update lost
318
- in transit is recovered by the client's own retransmit -- no reconnect required,
319
- and no dependence on a later edit happening to trigger a resync.
320
-
321
- This is entirely **self-gating**: stock clients send no `"id"`, so they never get
322
- acks and behave exactly as before. Only a client that opts in by tagging its
323
- frames participates.
324
-
325
- Two client examples ship in the demo:
326
-
327
- - [`frontend/reliable.mjs`](examples/actioncable-demo/frontend/reliable.mjs) — a
328
- minimal reference client showing the raw mechanism (tag with an id, retain,
329
- retransmit on a timer, drain on ack), with an end-to-end test that loses an
330
- update mid-flight and recovers it purely by retransmit.
331
- - [`frontend/provider/reliable_actioncable_provider.mjs`](examples/actioncable-demo/frontend/provider/reliable_actioncable_provider.mjs)
332
- — the standard `@y-rb/actioncable` `WebsocketProvider`, vendored and augmented
333
- for production use. It's a drop-in replacement that speaks the same protocol
334
- and envelope, and adds reliability with **sync-since-last-ack** framing: rather
335
- than retransmitting updates one by one, it keeps the unacknowledged local
336
- updates in a queue and sends their *merge* as a single causally-complete delta,
337
- with the id being the highest sequence in the batch (so one `{ ack: id }`
338
- cumulatively confirms everything up to it). Because the whole unacked tail goes
339
- as one self-contained delta, the server never sees an internal gap and never
340
- has to round-trip a resync for a lost middle update — the next edit, or the
341
- next timer tick, carries it. Awareness stays fire-and-forget; against a server
342
- that doesn't implement acks it warns once and falls back to plain delivery; and
343
- `reliable: false` opts out entirely. The demo's editor uses this provider.
291
+ `yrb-lite-client`'s `ActionCableProvider` handles this automatically. It keeps
292
+ the unacknowledged local document tail in a queue and sends the merged tail as a
293
+ single causally-complete delta. The id is the highest sequence in the batch, so
294
+ one `{ ack: id }` cumulatively confirms everything up to it. Because CRDT apply
295
+ is idempotent, a resend that already landed is a harmless no-op that just
296
+ re-acks. Awareness stays ephemeral and is not acked.
344
297
 
345
298
  ### User Awareness/Presence
346
299
 
@@ -376,24 +329,24 @@ message = awareness.encode_update(update_bytes)
376
329
 
377
330
  ## Thread Safety
378
331
 
379
- Unlike the official `y-rb` gem, yrb-lite is safe to share across Ruby threads. A
380
- `Doc` or `Awareness` can be used concurrently from Puma workers, ActionCable
381
- connection threads, or background jobs without external locking.
332
+ `Doc` and `Awareness` are safe to share across Ruby threads. A `Doc` or
333
+ `Awareness` can be used concurrently from Puma workers, ActionCable connection
334
+ threads, or background jobs without external locking.
382
335
 
383
336
  That comes from how the underlying types work, not from locking on top:
384
337
 
385
338
  - `yrs::Doc` is `Send + Sync`. Every operation takes the document's internal
386
339
  RwLock with blocking semantics (`read_blocking`/`write_blocking`), so
387
340
  concurrent access serializes instead of erroring or corrupting state.
388
- - `yrs::sync::Awareness` is built for multi-threaded servers: client states
389
- live in a `DashMap` and the whole API is `&self`.
390
- - The extension adds no interior-mutability tricks. There's no `RefCell`, where
391
- a re-entrant borrow would panic and take the Ruby process down with it.
392
- Each native method opens and closes its transaction in one call, so no lock
393
- or borrow outlives a call and there's nothing to deadlock on.
394
- - A `Send + Sync` static assertion for both wrapped types lives in `lib.rs`. If
395
- a yrs upgrade regressed this, the gem would fail to compile instead of quietly
396
- turning thread-unsafe.
341
+ - `yrs::sync::Awareness` is `Send` but not `Sync` in the current yrs version,
342
+ so the Ruby wrapper stores it in a `Mutex`. The mutex is always acquired
343
+ inside the no-GVL native section and released before Ruby runs again.
344
+ - The extension uses no `RefCell`-style runtime borrows that could panic under
345
+ re-entrancy. Each native method opens and closes its transaction or mutex
346
+ guard inside one call.
347
+ - Static assertions in `lib.rs` prove `Doc` and `Mutex<Awareness>` are
348
+ `Send + Sync`. If a yrs upgrade regressed either wrapper's thread-safety, the
349
+ gem would fail to compile instead of quietly turning thread-unsafe.
397
350
 
398
351
  `test/thread_safety_test.rb` runs shared docs, the full sync handshake, fan-in
399
352
  sync, and awareness state across 8 threads at once, and checks the interleaving
@@ -414,11 +367,12 @@ A slow operation also can't stall the VM. A thread applying a large update holds
414
367
  the doc's write lock without holding the GVL, so other Ruby threads keep running
415
368
  instead of queuing behind it.
416
369
 
417
- Each method has the same shape: copy the Ruby byte string, drop the GVL, do the
418
- yrs work (taking and releasing the doc lock entirely inside the closure), take
419
- the GVL back, then build Ruby objects. No Ruby API is touched without the GVL,
420
- and the doc lock is never held across a GVL boundary, so the lock order can't
421
- deadlock. Panics in native code are caught and re-raised as Ruby exceptions.
370
+ Each method has the same shape: copy Ruby byte strings first, drop the GVL, do
371
+ the yrs work while taking and releasing native locks entirely inside the
372
+ closure, take the GVL back, then build Ruby objects. No Ruby API is touched
373
+ without the GVL, and no native lock is held while reacquiring it, so the lock
374
+ order can't deadlock. Panics in native code are caught and re-raised as Ruby
375
+ exceptions.
422
376
 
423
377
  ## Message Type Constants
424
378
 
Binary file
Binary file
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module YrbLite
4
- VERSION = "0.1.0.beta4"
4
+ VERSION = "0.1.0.beta6"
5
5
  end
data/lib/yrb_lite.rb CHANGED
@@ -13,8 +13,8 @@ rescue LoadError
13
13
  end
14
14
 
15
15
  module YrbLite
16
- # Error class is defined in Rust extension
17
-
18
- # Autoload Sync module - only loaded when ActionCable is available
19
- autoload :Sync, "yrb_lite/sync"
16
+ # Error class is defined in the Rust extension.
17
+ #
18
+ # The ActionCable integration (YrbLite::ActionCable::Sync) lives in the
19
+ # separate `yrb-lite-actioncable` gem; require "yrb_lite/action_cable".
20
20
  end
metadata CHANGED
@@ -1,29 +1,15 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: yrb-lite
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0.beta4
4
+ version: 0.1.0.beta6
5
5
  platform: aarch64-linux
6
6
  authors:
7
7
  - JP Camara
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2026-06-18 00:00:00.000000000 Z
11
+ date: 2026-06-22 00:00:00.000000000 Z
12
12
  dependencies:
13
- - !ruby/object:Gem::Dependency
14
- name: base64
15
- requirement: !ruby/object:Gem::Requirement
16
- requirements:
17
- - - "~>"
18
- - !ruby/object:Gem::Version
19
- version: '0.2'
20
- type: :runtime
21
- prerelease: false
22
- version_requirements: !ruby/object:Gem::Requirement
23
- requirements:
24
- - - "~>"
25
- - !ruby/object:Gem::Version
26
- version: '0.2'
27
13
  - !ruby/object:Gem::Dependency
28
14
  name: minitest
29
15
  requirement: !ruby/object:Gem::Requirement
@@ -66,10 +52,10 @@ dependencies:
66
52
  - - "~>"
67
53
  - !ruby/object:Gem::Version
68
54
  version: '1.2'
69
- description: yrb-lite is a thread-safe Ruby binding over the Rust y-crdt (yrs) library
70
- plus an ActionCable concern implementing the full y-websocket sync protocol and
71
- awareness. It lets a Rails app be the collaboration server for Y.js editors (Tiptap,
72
- ProseMirror, BlockNote) with no Node sidecar.
55
+ description: 'yrb-lite is a thread-safe Ruby binding over the Rust y-crdt (yrs) library:
56
+ CRDT documents, awareness/presence, and the y-websocket sync protocol primitives,
57
+ with the GVL released during native work so documents sync in parallel. The ActionCable/Rails
58
+ integration lives in the companion yrb-lite-actioncable gem.'
73
59
  email:
74
60
  - johnpcamara@gmail.com
75
61
  executables: []
@@ -83,7 +69,6 @@ files:
83
69
  - lib/yrb_lite.rb
84
70
  - lib/yrb_lite/3.4/yrb_lite.so
85
71
  - lib/yrb_lite/4.0/yrb_lite.so
86
- - lib/yrb_lite/sync.rb
87
72
  - lib/yrb_lite/version.rb
88
73
  homepage: https://github.com/jpcamara/yrb-lite
89
74
  licenses:
@@ -114,6 +99,6 @@ requirements: []
114
99
  rubygems_version: 3.5.23
115
100
  signing_key:
116
101
  specification_version: 4
117
- summary: Thread-safe Ruby bindings for y-crdt (Y.js) with the y-websocket sync protocol
118
- for ActionCable
102
+ summary: 'Thread-safe Ruby bindings for y-crdt (Y.js): documents, awareness, and the
103
+ y-websocket sync protocol'
119
104
  test_files: []
data/lib/yrb_lite/sync.rb DELETED
@@ -1,580 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- require "base64"
4
- require "securerandom"
5
-
6
- module YrbLite
7
- # y-websocket protocol over ActionCable.
8
- #
9
- # Include this module in an ActionCable channel to sync Y.js documents
10
- # (and awareness/presence) with browser clients. Messages are the standard
11
- # y-protocols binary messages, base64-encoded in a JSON envelope:
12
- #
13
- # { "m" => "<base64 bytes>" } # client -> server
14
- # { "m" => "...", "origin" => "<id>" } # server -> subscribers
15
- #
16
- # Example:
17
- # class DocumentChannel < ApplicationCable::Channel
18
- # include YrbLite::Sync
19
- #
20
- # on_load { |key| Document.find_by(key: key)&.content }
21
- # on_save { |key, update| Document.find_by(key: key)&.update!(content: update) }
22
- #
23
- # # on_change blocks run in the channel instance's context, so instance
24
- # # methods (current_user, params, ...) are available without plumbing:
25
- # on_change { |key, update| Document.record!(key, update, by: current_user) }
26
- #
27
- # def subscribed
28
- # sync_for params[:id]
29
- # end
30
- #
31
- # def receive(data)
32
- # sync_receive(data)
33
- # end
34
- #
35
- # def unsubscribed
36
- # sync_clear_presence
37
- # end
38
- # end
39
- #
40
- # The shared YrbLite::Awareness instances are safe to use from ActionCable's
41
- # worker thread pool: the native types are Send + Sync and every operation
42
- # releases the GVL, so concurrent clients sync in parallel.
43
- module Sync
44
- # Validated frame kinds from Awareness#message_kind. A frame only gets a
45
- # non-DROP kind if it is exactly one well-formed message; anything
46
- # malformed, truncated, multi-message, or unknown is dropped before it can
47
- # be processed or relayed.
48
- MSG_KIND_DROP = 0
49
- MSG_KIND_SYNC_STEP1 = 1
50
- MSG_KIND_UPDATE = 2
51
- MSG_KIND_AWARENESS = 3
52
- MSG_KIND_AWARENESS_QUERY = 4
53
-
54
- def self.included(base)
55
- base.extend(ClassMethods)
56
- end
57
-
58
- module ClassMethods
59
- # Load persisted document state. Called once per key with (key);
60
- # return a binary Y.js update (or nil for a fresh document).
61
- def on_load(callable = nil, &block)
62
- @on_load = callable || block if callable || block
63
- @on_load
64
- end
65
-
66
- # Persist document state. Called with (key, update) after every
67
- # message that modified the document.
68
- def on_save(callable = nil, &block)
69
- @on_save = callable || block if callable || block
70
- @on_save
71
- end
72
-
73
- # Record every document change durably before it is applied or
74
- # distributed (authoritative audit mode). Called synchronously with
75
- # (key, update), where update is the exact CRDT delta, serialized per
76
- # document so the recorded order is the apply order. If the block raises,
77
- # the change is rejected: neither applied to the shared document nor
78
- # broadcast to other subscribers.
79
- #
80
- # A block recorder runs in the *channel instance's* context, so it can
81
- # call the channel's own methods (current_user, params, a per-connection
82
- # Current.* accessor) directly, with no thread-local plumbing. (A non-Proc
83
- # callable is invoked with #call instead, since it carries its own
84
- # context.) on_change always fires from within sync_receive, unlike
85
- # on_load/on_save, which can run context-free in the shared registry.
86
- #
87
- # Registering an on_change switches that channel onto the strict path
88
- # (record, apply, broadcast). Without it, the default fast path applies
89
- # and broadcasts, with an optional on_save snapshot.
90
- def on_change(callable = nil, &block)
91
- @on_change = callable || block if callable || block
92
- @on_change
93
- end
94
-
95
- # Select the document backend:
96
- # :memory (default): keep a warm in-memory replica per process and keep
97
- # it current via a custom stream_from callback. Fast, but it assumes
98
- # classic ActionCable (the callback runs in Ruby) and
99
- # process<->document affinity.
100
- # :store: stateless per message, with no warm replica and no custom
101
- # stream callback. Handshakes and reads are served from the durable
102
- # store (`on_load`); changes are recorded (`on_change`) and relayed.
103
- # Works under AnyCable (broadcasts handled outside Ruby, no worker
104
- # affinity) and across processes. Requires `on_load` and `on_change`.
105
- def sync_backend(mode = nil)
106
- @sync_backend = mode if mode
107
- @sync_backend || :memory
108
- end
109
- end
110
-
111
- # Call from `subscribed`. Streams broadcasts for this document and
112
- # transmits the server's opening handshake (SyncStep1 + awareness).
113
- def sync_for(key)
114
- @sync_key = key.to_s
115
- @sync_origin = SecureRandom.hex(8)
116
- @sync_clients = [] # awareness client IDs seen on this connection
117
-
118
- return sync_for_store_backed if self.class.sync_backend == :store
119
-
120
- Sync.subscribe(@sync_key)
121
- awareness = sync_awareness
122
-
123
- stream_from sync_stream_name, coder: ActiveSupport::JSON do |payload|
124
- sync_on_broadcast(payload)
125
- end
126
-
127
- # Opening handshake: SyncStep1 then the current awareness, each as its
128
- # own single-message frame, so providers that parse one message per frame
129
- # (e.g. @y-rb/actioncable) handle both. The client replies SyncStep2 to
130
- # the SyncStep1, delivering its state to the server.
131
- sync_transmit(awareness.sync_step1)
132
- sync_transmit(awareness.encode_awareness_update)
133
- end
134
-
135
- # Call from `receive`. Applies the client's message, replies directly
136
- # when the protocol calls for it, and relays document/awareness changes
137
- # to the other subscribers.
138
- #
139
- # If an `on_change` recorder is registered, document changes take the
140
- # strict authoritative path (record -> apply -> broadcast, serialized per
141
- # document); otherwise the fast path is used.
142
- #
143
- # Reliable delivery (opt-in, client-driven): if the frame carries an "id",
144
- # the server replies `{ "ack" => id }` once the update has been accepted
145
- # (recorded in audit mode, applied in fast mode). A causally-gapped update
146
- # is not acked -- it gets a resync instead -- so an ack-aware client knows
147
- # to retransmit until the update lands. Stock clients send no "id", never
148
- # get acks, and are completely unaffected.
149
- def sync_receive(data, key = nil)
150
- # Pass `key` (params[:id]) when your transport doesn't keep the channel
151
- # instance alive across actions. Under AnyCable each RPC command gets a
152
- # fresh channel, so instance variables set in `subscribed` are gone here.
153
- @sync_key = key.to_s if key
154
-
155
- # Accept both envelope keys: "m" (yrb-lite's own clients) and "update"
156
- # (the @y-rb/actioncable browser provider).
157
- m = data.is_a?(Hash) ? (data["m"] || data["update"]) : nil
158
- return unless m.is_a?(String)
159
-
160
- # Optional client-supplied id for reliable delivery (see sync_send_ack).
161
- id = data.is_a?(Hash) ? data["id"] : nil
162
-
163
- begin
164
- bytes = Base64.strict_decode64(m)
165
- rescue ArgumentError
166
- return # not valid base64; ignore the frame and keep the connection
167
- end
168
-
169
- sync_send_ack(id, sync_dispatch(m, bytes))
170
- end
171
-
172
- # Route a decoded frame to the backend/path that handles it and return the
173
- # outcome symbol (:recorded/:applied/:gap/:noop) used by the reliable-
174
- # delivery ack. A dropped frame returns nil (never acked).
175
- def sync_dispatch(encoded, bytes)
176
- return sync_receive_store_backed(encoded, bytes) if self.class.sync_backend == :store
177
-
178
- awareness = sync_awareness
179
- kind = awareness.message_kind(bytes)
180
- # Malformed / truncated / multi-message / unknown frames are dropped
181
- # before they can be processed or relayed to other clients.
182
- return if kind == MSG_KIND_DROP
183
-
184
- sync_track_clients(awareness, bytes) if kind == MSG_KIND_AWARENESS
185
-
186
- if kind == MSG_KIND_UPDATE && self.class.on_change
187
- sync_apply_authoritative(awareness, encoded, bytes)
188
- else
189
- sync_apply_fast(awareness, encoded, bytes, kind)
190
- end
191
- end
192
-
193
- # Call from `unsubscribed`. Clears the presence states this connection
194
- # introduced and tells the other subscribers to drop those cursors, so a
195
- # closed tab or dropped socket doesn't leave a ghost cursor behind until
196
- # the client-side timeout reaps it.
197
- def sync_clear_presence
198
- return if @sync_clients.nil? || @sync_clients.empty?
199
-
200
- removal = sync_awareness.remove_clients(@sync_clients)
201
- @sync_clients = []
202
- return if removal.empty?
203
-
204
- sync_distribute(Base64.strict_encode64(removal))
205
- end
206
-
207
- # Call from `unsubscribed`. Clears this connection's presence and, when the
208
- # last subscriber for the document leaves, persists and unloads it from
209
- # memory (only when an `on_load` is configured to bring it back; otherwise
210
- # the in-memory document is the only copy and is kept). Prevents a
211
- # long-running server from accumulating every document it has ever served.
212
- def sync_unsubscribed(key = nil)
213
- @sync_key = key.to_s if key
214
- return if self.class.sync_backend == :store # nothing cached per process
215
-
216
- sync_clear_presence
217
- saver = self.class.on_save
218
- Sync.release(@sync_key, evictable: !self.class.on_load.nil?) do |awareness|
219
- saver&.call(@sync_key, awareness.encode_state_as_update)
220
- end
221
- end
222
-
223
- # The shared Awareness (document + presence) for this channel's key.
224
- # Also useful for server-side reads, e.g.:
225
- # sync_awareness.encode_state_as_update
226
- def sync_awareness
227
- Sync.awareness_for(@sync_key, self.class.on_load)
228
- end
229
-
230
- private
231
-
232
- # Default path: apply the message, answer direct requests, relay
233
- # state-changing messages to the other subscribers. Routing comes from the
234
- # native `kind` (from Awareness#message_kind) rather than peeking at bytes.
235
- # Document changes (SyncStep2, Update) and awareness get relayed; requests
236
- # (SyncStep1, awareness-query) are answered above and not relayed. An
237
- # optional on_save snapshot is taken after a document change.
238
- #
239
- # Returns an outcome symbol for the reliable-delivery ack: :applied when a
240
- # document update was integrated and relayed, :gap when it was rejected for
241
- # a resync, :noop for everything else (requests, awareness, empty updates).
242
- def sync_apply_fast(awareness, encoded, bytes, kind)
243
- # A document update that isn't causally ready (an earlier one was lost in
244
- # transit) would relay an un-integrable change to peers and stall the
245
- # replica. Drop it and ask the client to resync instead, which re-delivers
246
- # the missing piece. See sync_apply_authoritative for the durable variant.
247
- if kind == MSG_KIND_UPDATE
248
- update = awareness.update_from_message(bytes)
249
- # A no-op message (e.g. the empty SyncStep2 in an opening handshake)
250
- # carries no change, so there's nothing to relay, persist, or ack.
251
- return :noop unless update
252
-
253
- unless awareness.update_ready?(update)
254
- sync_request_resync(awareness)
255
- return :gap
256
- end
257
- end
258
-
259
- response = awareness.handle(bytes)
260
- sync_transmit(response) unless response.empty?
261
-
262
- return :noop unless [MSG_KIND_UPDATE, MSG_KIND_AWARENESS].include?(kind)
263
-
264
- sync_distribute(encoded)
265
- return :noop unless kind == MSG_KIND_UPDATE
266
-
267
- sync_persist
268
- :applied
269
- end
270
-
271
- # Authoritative path: record the change durably, then apply it to the
272
- # shared document, then distribute it. The sequence runs under a
273
- # per-document lock so changes are recorded in a single total order that
274
- # matches the order they're applied, and nothing is distributed (or applied)
275
- # before it has been recorded. If the recorder raises, the change is
276
- # rejected (not applied, not broadcast) and the exception propagates, so the
277
- # channel can surface it and the client can resync.
278
- #
279
- # Before recording, the update must be causally ready: every dependency it
280
- # references must already be in the doc. If an earlier update was lost in
281
- # transit, or its record failed, a later update arrives with a gap. Recording
282
- # it would write a permanently-pending entry to the log -- one that can never
283
- # be replayed until the missing update shows up. Such an update is rejected
284
- # (not recorded, not applied, not relayed) and the client is asked to resync,
285
- # which re-delivers the missing range as one causally-complete delta.
286
- def sync_apply_authoritative(awareness, encoded, bytes)
287
- recorder = self.class.on_change
288
-
289
- outcome = Sync.lock_for(@sync_key).synchronize do
290
- update = awareness.update_from_message(bytes)
291
- # A no-op message (e.g. the empty SyncStep2 in a client's opening
292
- # handshake) carries no change, so there's nothing to record or relay.
293
- next :noop unless update
294
- next :gap unless awareness.update_ready?(update)
295
-
296
- sync_record_change(recorder, update) # durable write; raise to reject
297
- awareness.apply_update(update) # only recorded changes reach the doc
298
- sync_distribute(encoded) # ...and only then the wire
299
- :recorded
300
- end
301
-
302
- case outcome
303
- when :recorded then sync_persist
304
- when :gap then sync_request_resync(awareness)
305
- end
306
-
307
- # Surface the outcome for the reliable-delivery ack: :recorded means the
308
- # update is durably written (and will be acked); :gap triggered a resync
309
- # (no ack); :noop carried no change.
310
- outcome
311
- end
312
-
313
- # Ask this connection's client to resync: re-send SyncStep1 carrying the
314
- # server's current (gap-free) state vector. The client replies SyncStep2
315
- # with everything the server is missing, delivered as one causally-complete
316
- # delta -- which heals the gap that triggered the resync.
317
- def sync_request_resync(awareness)
318
- sync_transmit(awareness.sync_step1)
319
- end
320
-
321
- # Reliable delivery: acknowledge an accepted update back to the sending
322
- # connection. An ack-aware client tags each outgoing update with an "id"
323
- # and retains it until the matching `{ "ack" => id }` returns, retransmitting
324
- # on a timer or reconnect; idempotent CRDT apply makes resends free. We ack
325
- # only when the client supplied an id (so stock clients are unaffected) and
326
- # the update was actually accepted -- recorded in audit mode, applied in fast
327
- # mode. A gapped update gets no ack (it got a resync), so the client keeps
328
- # retransmitting until the missing range lands and the update can integrate.
329
- def sync_send_ack(id, outcome)
330
- return if id.nil?
331
- return unless %i[recorded applied].include?(outcome)
332
-
333
- # Braces are load-bearing: a bare hash would bind to transmit's `via:`
334
- # keyword instead of its positional data argument.
335
- transmit({ "ack" => id })
336
- end
337
-
338
- # Single broadcast point for both paths (and presence removal), so the
339
- # relay semantics live in one place and tests can observe distribution.
340
- # `origin` identifies the sending connection (don't echo to it); `pid`
341
- # identifies the sending process (other processes apply it to their own
342
- # replica; see sync_on_broadcast).
343
- def sync_distribute(encoded)
344
- ActionCable.server.broadcast(
345
- sync_stream_name,
346
- sync_envelope(encoded, "origin" => @sync_origin, "pid" => Sync.process_id)
347
- )
348
- end
349
-
350
- # Transmit raw protocol bytes to this connection (base64, dual-key).
351
- def sync_transmit(bytes)
352
- transmit(sync_envelope(Base64.strict_encode64(bytes)))
353
- end
354
-
355
- # Build an outgoing envelope. We send the payload under both keys: "m"
356
- # (yrb-lite's own clients) and "update" (the @y-rb/actioncable provider),
357
- # so either client works against the same server.
358
- def sync_envelope(encoded, extra = {})
359
- { "m" => encoded, "update" => encoded }.merge(extra)
360
- end
361
-
362
- # Handle a broadcast delivered by the cable adapter. With a multi-process
363
- # adapter (Redis, solid_cable), it may have come from another server
364
- # process. Keep this process's in-memory replica current with changes that
365
- # originated elsewhere, then relay to this connection's browser.
366
- def sync_on_broadcast(payload)
367
- sync_apply_remote(payload["m"]) if payload["pid"] != Sync.process_id
368
- transmit(payload) unless payload["origin"] == @sync_origin
369
- end
370
-
371
- # Apply a change that originated on another process to this process's
372
- # replica, without re-recording it (the origin process already recorded it
373
- # before broadcasting). The CRDT merge is idempotent and commutative, so a
374
- # cold replica converges regardless of ordering, and applying from several
375
- # local connections is harmless.
376
- def sync_apply_remote(encoded)
377
- return unless encoded.is_a?(String)
378
-
379
- begin
380
- bytes = Base64.strict_decode64(encoded)
381
- rescue ArgumentError
382
- return
383
- end
384
-
385
- awareness = sync_awareness
386
- case awareness.message_kind(bytes)
387
- when MSG_KIND_UPDATE
388
- update = awareness.update_from_message(bytes)
389
- awareness.apply_update(update) if update
390
- when MSG_KIND_AWARENESS
391
- awareness.handle(bytes)
392
- end
393
- end
394
-
395
- # -- Store-backed (AnyCable-native) path --------------------------------
396
-
397
- # Subscribe without a custom block, so AnyCable (which delivers broadcasts
398
- # outside Ruby) relays them directly. Send the opening SyncStep1 built from
399
- # the durable store. No warm replica is kept.
400
- def sync_for_store_backed
401
- stream_from sync_stream_name
402
- sync_transmit(sync_load_doc.sync_step1)
403
- end
404
-
405
- # Stateless per message: no warm replica, no assumptions about which process
406
- # owns a document. A client's SyncStep1 is answered from the store, document
407
- # changes are recorded durably before relay and then broadcast, and
408
- # awareness is relayed best-effort. Echoing back to the sender is harmless,
409
- # since the CRDT apply is idempotent.
410
- #
411
- # Returns an outcome symbol for the reliable-delivery ack: :recorded when a
412
- # document update was durably recorded and relayed, :gap when it was
413
- # rejected for a resync, :noop for everything else.
414
- def sync_receive_store_backed(encoded, bytes)
415
- case Sync.codec.message_kind(bytes)
416
- when MSG_KIND_SYNC_STEP1
417
- result = sync_load_doc.handle_sync_message(bytes)
418
- sync_transmit(result[2]) if result
419
- :noop
420
- when MSG_KIND_UPDATE
421
- update = Sync.codec.update_from_message(bytes)
422
- return :noop unless update
423
-
424
- # Store mode keeps no warm replica, so to tell whether this update is
425
- # causally ready we rebuild the doc from the store and check against it.
426
- # That's an O(history) load per update (mitigated by snapshotting the
427
- # store on the load path). A gappy update -- an earlier one was lost or
428
- # its record failed -- is rejected and the client asked to resync,
429
- # rather than written to the log as a permanently-pending entry.
430
- doc = sync_load_doc
431
- unless doc.update_ready?(update)
432
- sync_transmit(doc.sync_step1)
433
- return :gap
434
- end
435
-
436
- if (recorder = self.class.on_change)
437
- sync_record_change(recorder, update) # record before relay
438
- end
439
- sync_distribute(encoded)
440
- :recorded
441
- when MSG_KIND_AWARENESS
442
- sync_distribute(encoded)
443
- :noop
444
- else
445
- :noop
446
- end
447
- end
448
-
449
- # Build a fresh document from the durable store (on_load).
450
- def sync_load_doc
451
- doc = YrbLite::Doc.new
452
- state = self.class.on_load&.call(@sync_key)
453
- doc.apply_update(state) if state
454
- doc
455
- end
456
-
457
- # Record the awareness client IDs carried by an incoming message (already
458
- # known to be an awareness frame) so we can clear them when this connection
459
- # closes.
460
- def sync_track_clients(awareness, bytes)
461
- awareness.awareness_client_ids(bytes).each do |id|
462
- @sync_clients << id unless @sync_clients.include?(id)
463
- end
464
- end
465
-
466
- def sync_stream_name
467
- "yrb_lite:#{@sync_key}"
468
- end
469
-
470
- def sync_persist
471
- return unless (saver = self.class.on_save)
472
-
473
- saver.call(@sync_key, sync_awareness.encode_state_as_update)
474
- end
475
-
476
- # Invoke the on_change recorder. A block/proc runs in this channel instance's
477
- # context (instance_exec) so it can reach the channel's own methods; a
478
- # non-Proc callable is invoked with #call, since it carries its own context.
479
- def sync_record_change(recorder, update)
480
- args = [@sync_key, update]
481
- recorder.is_a?(Proc) ? instance_exec(*args, &recorder) : recorder.call(*args)
482
- end
483
-
484
- # -- Shared document registry ------------------------------------------
485
-
486
- @registry = {}
487
- @locks = {}
488
- @subscribers = Hash.new(0)
489
- @registry_mutex = Mutex.new
490
-
491
- class << self
492
- # A stable id for this server process, stamped on every broadcast so
493
- # other processes know to apply it to their replica and this process
494
- # knows to skip its own. Survives for the life of the process.
495
- def process_id
496
- @process_id ||= SecureRandom.hex(8)
497
- end
498
-
499
- # A shared, stateless decoder for the store-backed path. message_kind and
500
- # update_from_message only read their argument (they don't touch the
501
- # instance's document), so one shared instance is safe across threads.
502
- def codec
503
- @codec ||= YrbLite::Awareness.new
504
- end
505
-
506
- # Get or create the shared Awareness for a key. Creation (including
507
- # the on_load callback) is serialized under a mutex so concurrent
508
- # subscribers can never observe two documents for one key; all
509
- # subsequent operations run lock-free on the thread-safe native types.
510
- def awareness_for(key, loader = nil)
511
- @registry_mutex.synchronize do
512
- @registry[key] ||= begin
513
- awareness = YrbLite::Awareness.new
514
- if loader && (state = loader.call(key))
515
- awareness.apply_update(state)
516
- end
517
- awareness
518
- end
519
- end
520
- end
521
-
522
- # Per-document mutex serializing the authoritative record -> apply ->
523
- # broadcast section, so a document's audit log is a single total order.
524
- # Only briefly holds the registry mutex to fetch/create the lock; the
525
- # durable write itself runs while holding only this per-key lock.
526
- def lock_for(key)
527
- @registry_mutex.synchronize { @locks[key] ||= Mutex.new }
528
- end
529
-
530
- # Count a new subscriber for a document.
531
- def subscribe(key)
532
- @registry_mutex.synchronize { @subscribers[key] += 1 }
533
- end
534
-
535
- # Drop a subscriber. When the last one leaves and the document is
536
- # evictable (there's an on_load to bring it back, so unloading can't lose
537
- # data), persist it via the given block and unload it from memory, so a
538
- # long-running server doesn't accumulate every document and lock it has
539
- # ever seen. Returns true if the document was evicted.
540
- #
541
- # The persist runs outside the registry lock (it may do I/O), and we
542
- # re-check the subscriber count afterward: if someone reconnected while
543
- # we were saving, eviction is aborted and the warm document is kept.
544
- def release(key, evictable:)
545
- awareness = @registry_mutex.synchronize do
546
- @subscribers[key] -= 1 if @subscribers[key].positive?
547
- next nil unless @subscribers[key].zero?
548
-
549
- @subscribers.delete(key)
550
- evictable ? @registry[key] : nil
551
- end
552
- return false unless awareness
553
-
554
- yield awareness if block_given?
555
-
556
- @registry_mutex.synchronize do
557
- # A subscriber may have returned during the persist above.
558
- next false unless @subscribers[key].zero?
559
-
560
- @subscribers.delete(key)
561
- @locks.delete(key)
562
- !@registry.delete(key).nil?
563
- end
564
- end
565
-
566
- def registry
567
- @registry_mutex.synchronize { @registry.dup }
568
- end
569
-
570
- # Clear all documents (useful for testing).
571
- def reset!
572
- @registry_mutex.synchronize do
573
- @registry = {}
574
- @locks = {}
575
- @subscribers = Hash.new(0)
576
- end
577
- end
578
- end
579
- end
580
- end