yrb-lite 0.1.0.beta1 → 0.1.0.beta2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 73c6ed02c103b7647be2d6052ff660490088bff3e1dba3a82105f8fb42ffecab
-  data.tar.gz: c308cb9c4e426992b1cbc31b2a870dc3ab3863c3a0217949b7744c4bc4c48d91
+  metadata.gz: e81bc59c4870f1c86a73f06fadeaf00fe60b46f0677823d2a6de20190761632e
+  data.tar.gz: bf98a231bccd388df0068af60a336c110fdc043daf3cc6cfa56208a4de6352db
 SHA512:
-  metadata.gz: 8eb77739b34c860f961a850a70ed39f778f0711d83fda715c8447af1c7d81625472697142ce5aaf8b8a339c408992a6ada2d8a176cf03b1145afc62bd5f7aca5
-  data.tar.gz: 8886214b02d90bcbe44958deaa0b1b73bd0aa4707f185f34868f5444e99ca8e4ed02dce4100486543fb7dec82d609996de6f90f81acf12fbf1217466115ba985
+  metadata.gz: 2e07b28a11faa8ec051b78eb107fa9d4f63fbd483b1118fd7605632665773525cb07f147f3dc25136b3876976b8d9a470db5ff5277a8647cffc64efb8bf4b298
+  data.tar.gz: b1afad6a62b1dd0d1edfa7c8bffc5e324d407c489c2ff9fc4056c43d8af0f076ec30fdcf946582ee092b642210414912baa39be279770a88f78279c2ec707127

data/CHANGELOG.md

@@ -6,6 +6,34 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
 
+## [0.1.0.beta2] - 2026-06-16
+
+### Added
+
+- Reliable delivery (opt-in, client-driven). A client may tag a document update
+  with an `"id"`; the server replies `{ "ack": <id> }` once the update has been
+  accepted (recorded in audit mode, applied in fast mode). This lets an
+  ack-aware client retain and retransmit an update until delivery is confirmed,
+  so an edit can't be silently lost on a flaky connection. Stock clients send no
+  `"id"`, never get acks, and behave exactly as before.
+- A vendored, ack-aware `@y-rb/actioncable` provider in the demo
+  (`reliable_actioncable_provider.mjs`) that adds reliable delivery with
+  "sync-since-last-ack" framing (the unacknowledged tail is sent as one merged,
+  causally-complete delta), plus a minimal reference client and an intensive
+  message-loss stress test.
+
+### Fixed
+
+- Causal-gap protection. The authoritative, fast, and store paths now reject a
+  document update that isn't causally ready -- one whose dependencies are
+  missing because an earlier update was lost in transit or its durable record
+  failed -- and ask the client to resync, instead of recording or relaying an
+  un-integrable update that would leave the log permanently pending. Adds native
+  `Doc#update_ready?`/`#pending?` (cheap, read-only checks) used to gate the
+  record-before-distribute path.
+
+## [0.1.0.beta1]
+
 ### Added
 
 - Thread-safe `YrbLite::Doc` and `YrbLite::Awareness` over `yrs` (magnus/rb-sys
@@ -26,4 +54,6 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 - Precompiled native gems for common platforms (no Rust toolchain needed to
   install) via the cross-gem workflow.
 
-[Unreleased]: https://github.com/jpcamara/yrb-lite/commits/main
+[Unreleased]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta2...main
+[0.1.0.beta2]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta1...v0.1.0.beta2
+[0.1.0.beta1]: https://github.com/jpcamara/yrb-lite/releases/tag/v0.1.0.beta1

data/README.md

@@ -292,6 +292,56 @@ mode (in [`examples/actioncable-demo`](examples/actioncable-demo)) wires
 `on_change` to an fsync'd append-only log and checks, end to end, that the log
 alone rebuilds the document.
 
+#### Reliable delivery (acks)
+
+The y-websocket protocol is fire-and-forget. If a client's update is lost in
+transit (a flaky socket, a send that never lands) and the client makes no
+further edits, the server stays idle and never asks anyone to resync, so that
+edit is gone -- even though the client believes it was saved. CRDTs converge the
+state everyone *has*; they don't recover an update that never arrived.
+
+yrb-lite closes that gap with an opt-in, client-driven acknowledgement. If an
+incoming frame carries an `"id"`, the server replies `{ "ack": <id> }` once the
+update has been **accepted** -- recorded in audit mode, applied in fast mode. A
+causally-gapped update is not acked (it gets a resync instead), so the client
+knows it hasn't landed yet.
+
+```
+client -> server   { "m": "<base64 update>", "id": 42 }
+server -> client    { "ack": 42 }     # update accepted; safe to forget
+```
+
+That's the whole server side. A reliable client tags each outgoing update with
+an incrementing id, keeps it in a pending buffer, and retransmits on a timer (and
+on reconnect) until the matching ack returns. Because CRDT apply is idempotent, a
+resend that already landed is a harmless no-op that just re-acks. An update lost
+in transit is recovered by the client's own retransmit -- no reconnect required,
+and no dependence on a later edit happening to trigger a resync.
+
+This is entirely **self-gating**: stock clients send no `"id"`, so they never get
+acks and behave exactly as before. Only a client that opts in by tagging its
+frames participates.
+
+Two client examples ship in the demo:
+
+- [`frontend/reliable.mjs`](examples/actioncable-demo/frontend/reliable.mjs) — a
+  minimal reference client showing the raw mechanism (tag with an id, retain,
+  retransmit on a timer, drain on ack), with an end-to-end test that loses an
+  update mid-flight and recovers it purely by retransmit.
+- [`frontend/provider/reliable_actioncable_provider.mjs`](examples/actioncable-demo/frontend/provider/reliable_actioncable_provider.mjs)
+  — the standard `@y-rb/actioncable` `WebsocketProvider`, vendored and augmented
+  for production use. It's a drop-in replacement that speaks the same protocol
+  and envelope, and adds reliability with **sync-since-last-ack** framing: rather
+  than retransmitting updates one by one, it keeps the unacknowledged local
+  updates in a queue and sends their *merge* as a single causally-complete delta,
+  with the id being the highest sequence in the batch (so one `{ ack: id }`
+  cumulatively confirms everything up to it). Because the whole unacked tail goes
+  as one self-contained delta, the server never sees an internal gap and never
+  has to round-trip a resync for a lost middle update — the next edit, or the
+  next timer tick, carries it. Awareness stays fire-and-forget; against a server
+  that doesn't implement acks it warns once and falls back to plain delivery; and
+  `reliable: false` opts out entirely. The demo's editor uses this provider.
+
 ### User Awareness/Presence
 
 ```ruby

data/ext/yrb_lite/src/lib.rs

@@ -171,6 +171,24 @@ fn awareness_client_ids_in(bytes: &[u8]) -> Result<Vec<u64>, String> {
     Ok(ids)
 }
 
+/// True if applying `update_bytes` to `doc` would integrate cleanly: every
+/// dependency the update references is already present (the doc's state vector
+/// covers the update's lower bound). A pure read; does not mutate the doc.
+/// When false, applying it would park a pending struct -- the signal that an
+/// earlier, causally-prior update is missing.
+fn update_is_ready(doc: &Doc, update_bytes: &[u8]) -> Result<bool, String> {
+    let update = yrs::Update::decode_v1(update_bytes).map_err(|e| e.to_string())?;
+    Ok(doc.transact().state_vector() >= update.state_vector_lower())
+}
+
+/// True if the doc holds pending structs or a pending delete set -- blocks that
+/// couldn't integrate because a dependency is missing. Used as a backstop after
+/// loading from storage: leftover pending means the stored log has a causal gap.
+fn doc_has_pending(doc: &Doc) -> bool {
+    let txn = doc.transact();
+    txn.store().pending_update().is_some() || txn.store().pending_ds().is_some()
+}
+
 // ============================================================================
 // Doc Implementation
 // ============================================================================
@@ -240,6 +258,22 @@ impl RbDoc {
         .map_err(runtime_error)
     }
 
+    /// True if applying `update` would integrate cleanly (its dependencies are
+    /// all present). False means it would leave a pending struct -- an earlier
+    /// update is missing. Pure read; does not mutate.
+    fn update_ready(&self, update: RString) -> Result<bool, Error> {
+        let update_bytes = copy_bytes(update);
+        let doc = &self.0;
+        nogvl(move || update_is_ready(doc, &update_bytes)).map_err(runtime_error)
+    }
+
+    /// True if the document holds pending (un-integrable) structs waiting on a
+    /// missing dependency.
+    fn pending(&self) -> bool {
+        let doc = &self.0;
+        nogvl(move || doc_has_pending(doc))
+    }
+
     /// Sync step 1: Create a sync message with our state vector
     fn sync_step1(&self) -> RString {
         let doc = &self.0;
@@ -318,7 +352,6 @@ impl RbDoc {
         let msg = Message::Sync(SyncMessage::Update(update_bytes.to_vec()));
         binary_string(&msg.encode_v1())
     }
-
 }
 
 // ============================================================================
@@ -487,6 +520,22 @@ impl RbAwareness {
         .map_err(runtime_error)
     }
 
+    /// True if applying `update` would integrate cleanly (its dependencies are
+    /// all present). False means it depends on a missing, causally-prior update.
+    /// Pure read; does not mutate.
+    fn update_ready(&self, update: RString) -> Result<bool, Error> {
+        let update_bytes = copy_bytes(update);
+        let doc = self.0.doc();
+        nogvl(move || update_is_ready(doc, &update_bytes)).map_err(runtime_error)
+    }
+
+    /// True if the document holds pending (un-integrable) structs waiting on a
+    /// missing dependency.
+    fn pending(&self) -> bool {
+        let doc = self.0.doc();
+        nogvl(move || doc_has_pending(doc))
+    }
+
     /// Decode the awareness client IDs referenced by a protocol message
     /// (which may pack several sub-messages together). Sync sub-messages are
     /// ignored. The ActionCable layer uses this to learn which presence
@@ -577,6 +626,8 @@ fn init(ruby: &Ruby) -> Result<(), Error> {
         method!(RbDoc::encode_state_as_update, -1),
     )?;
     doc_class.define_method("apply_update", method!(RbDoc::apply_update, 1))?;
+    doc_class.define_method("update_ready?", method!(RbDoc::update_ready, 1))?;
+    doc_class.define_method("pending?", method!(RbDoc::pending, 0))?;
     doc_class.define_method("sync_step1", method!(RbDoc::sync_step1, 0))?;
     doc_class.define_method("sync_step2", method!(RbDoc::sync_step2, 1))?;
     doc_class.define_method(
@@ -606,6 +657,8 @@ fn init(ruby: &Ruby) -> Result<(), Error> {
         method!(RbAwareness::encode_state_as_update, -1),
     )?;
     awareness_class.define_method("apply_update", method!(RbAwareness::apply_update, 1))?;
+    awareness_class.define_method("update_ready?", method!(RbAwareness::update_ready, 1))?;
+    awareness_class.define_method("pending?", method!(RbAwareness::pending, 0))?;
     awareness_class.define_method("set_local_state", method!(RbAwareness::set_local_state, 1))?;
     awareness_class.define_method("local_state", method!(RbAwareness::local_state, 0))?;
     awareness_class.define_method(
@@ -749,4 +802,50 @@ mod tests {
             .unwrap()
             .is_empty());
     }
+
+    #[test]
+    fn update_readiness_and_pending_detect_a_causal_gap() {
+        // Three sequential single-char inserts from one client: A, then B, then
+        // C. Each delta depends on the previous, so C can't integrate without B.
+        let src = Doc::new();
+        let txt = src.get_or_insert_text("t");
+        let mut deltas: Vec<Vec<u8>> = Vec::new();
+        let mut prev = yrs::StateVector::default();
+        for (i, ch) in ["A", "B", "C"].into_iter().enumerate() {
+            txt.insert(&mut src.transact_mut(), i as u32, ch);
+            deltas.push(src.transact().encode_state_as_update_v1(&prev));
+            prev = src.transact().state_vector();
+        }
+        let (u1, u2, u3) = (&deltas[0], &deltas[1], &deltas[2]);
+
+        // A doc holding only u1 (u2 was lost in transit / its record failed):
+        let doc = Doc::new();
+        doc.transact_mut()
+            .apply_update(yrs::Update::decode_v1(u1).unwrap())
+            .unwrap();
+        assert!(update_is_ready(&doc, u1).unwrap(), "u1 has no missing deps");
+        assert!(
+            !update_is_ready(&doc, u3).unwrap(),
+            "u3 depends on the missing u2"
+        );
+        assert!(
+            !doc_has_pending(&doc),
+            "nothing pending until u3 is applied"
+        );
+
+        // Applying u3 anyway parks it as a pending struct.
+        doc.transact_mut()
+            .apply_update(yrs::Update::decode_v1(u3).unwrap())
+            .unwrap();
+        assert!(
+            doc_has_pending(&doc),
+            "u3 is pending: its parent u2 is missing"
+        );
+
+        // Once u2 arrives (via resync), u3 integrates and pending clears.
+        doc.transact_mut()
+            .apply_update(yrs::Update::decode_v1(u2).unwrap())
+            .unwrap();
+        assert!(!doc_has_pending(&doc), "u2 arrived; u3 integrated");
+    }
 }

data/lib/yrb_lite/sync.rb

@@ -128,6 +128,13 @@ module YrbLite
     # If an `on_change` recorder is registered, document changes take the
     # strict authoritative path (record -> apply -> broadcast, serialized per
     # document); otherwise the fast path is used.
+    #
+    # Reliable delivery (opt-in, client-driven): if the frame carries an "id",
+    # the server replies `{ "ack" => id }` once the update has been accepted
+    # (recorded in audit mode, applied in fast mode). A causally-gapped update
+    # is not acked -- it gets a resync instead -- so an ack-aware client knows
+    # to retransmit until the update lands. Stock clients send no "id", never
+    # get acks, and are completely unaffected.
     def sync_receive(data, key = nil)
       # Pass `key` (params[:id]) when your transport doesn't keep the channel
       # instance alive across actions. Under AnyCable each RPC command gets a
@@ -139,13 +146,23 @@ module YrbLite
       m = data.is_a?(Hash) ? (data["m"] || data["update"]) : nil
       return unless m.is_a?(String)
 
+      # Optional client-supplied id for reliable delivery (see sync_send_ack).
+      id = data.is_a?(Hash) ? data["id"] : nil
+
       begin
         bytes = Base64.strict_decode64(m)
       rescue ArgumentError
         return # not valid base64; ignore the frame and keep the connection
       end
 
-      return sync_receive_store_backed(m, bytes) if self.class.sync_backend == :store
+      sync_send_ack(id, sync_dispatch(m, bytes))
+    end
+
+    # Route a decoded frame to the backend/path that handles it and return the
+    # outcome symbol (:recorded/:applied/:gap/:noop) used by the reliable-
+    # delivery ack. A dropped frame returns nil (never acked).
+    def sync_dispatch(encoded, bytes)
+      return sync_receive_store_backed(encoded, bytes) if self.class.sync_backend == :store
 
       awareness = sync_awareness
       kind = awareness.message_kind(bytes)
@@ -156,9 +173,9 @@ module YrbLite
       sync_track_clients(awareness, bytes) if kind == MSG_KIND_AWARENESS
 
       if kind == MSG_KIND_UPDATE && self.class.on_change
-        sync_apply_authoritative(awareness, m, bytes)
+        sync_apply_authoritative(awareness, encoded, bytes)
       else
-        sync_apply_fast(awareness, m, bytes, kind)
+        sync_apply_fast(awareness, encoded, bytes, kind)
       end
     end
 
@@ -207,14 +224,37 @@ module YrbLite
     # Document changes (SyncStep2, Update) and awareness get relayed; requests
     # (SyncStep1, awareness-query) are answered above and not relayed. An
     # optional on_save snapshot is taken after a document change.
+    #
+    # Returns an outcome symbol for the reliable-delivery ack: :applied when a
+    # document update was integrated and relayed, :gap when it was rejected for
+    # a resync, :noop for everything else (requests, awareness, empty updates).
     def sync_apply_fast(awareness, encoded, bytes, kind)
+      # A document update that isn't causally ready (an earlier one was lost in
+      # transit) would relay an un-integrable change to peers and stall the
+      # replica. Drop it and ask the client to resync instead, which re-delivers
+      # the missing piece. See sync_apply_authoritative for the durable variant.
+      if kind == MSG_KIND_UPDATE
+        update = awareness.update_from_message(bytes)
+        # A no-op message (e.g. the empty SyncStep2 in an opening handshake)
+        # carries no change, so there's nothing to relay, persist, or ack.
+        return :noop unless update
+
+        unless awareness.update_ready?(update)
+          sync_request_resync(awareness)
+          return :gap
+        end
+      end
+
       response = awareness.handle(bytes)
       sync_transmit(response) unless response.empty?
 
-      return unless [MSG_KIND_UPDATE, MSG_KIND_AWARENESS].include?(kind)
+      return :noop unless [MSG_KIND_UPDATE, MSG_KIND_AWARENESS].include?(kind)
 
       sync_distribute(encoded)
-      sync_persist if kind == MSG_KIND_UPDATE
+      return :noop unless kind == MSG_KIND_UPDATE
+
+      sync_persist
+      :applied
     end
 
     # Authoritative path: record the change durably, then apply it to the
@@ -224,22 +264,64 @@ module YrbLite
     # before it has been recorded. If the recorder raises, the change is
     # rejected (not applied, not broadcast) and the exception propagates, so the
     # channel can surface it and the client can resync.
+    #
+    # Before recording, the update must be causally ready: every dependency it
+    # references must already be in the doc. If an earlier update was lost in
+    # transit, or its record failed, a later update arrives with a gap. Recording
+    # it would write a permanently-pending entry to the log -- one that can never
+    # be replayed until the missing update shows up. Such an update is rejected
+    # (not recorded, not applied, not relayed) and the client is asked to resync,
+    # which re-delivers the missing range as one causally-complete delta.
     def sync_apply_authoritative(awareness, encoded, bytes)
       recorder = self.class.on_change
 
-      modified = Sync.lock_for(@sync_key).synchronize do
+      outcome = Sync.lock_for(@sync_key).synchronize do
         update = awareness.update_from_message(bytes)
         # A no-op message (e.g. the empty SyncStep2 in a client's opening
         # handshake) carries no change, so there's nothing to record or relay.
-        next false unless update
+        next :noop unless update
+        next :gap unless awareness.update_ready?(update)
 
         recorder.call(@sync_key, update) # durable write; raise to reject
         awareness.apply_update(update)   # only recorded changes reach the doc
         sync_distribute(encoded)         # ...and only then the wire
-        true
+        :recorded
+      end
+
+      case outcome
+      when :recorded then sync_persist
+      when :gap      then sync_request_resync(awareness)
       end
 
-      sync_persist if modified
+      # Surface the outcome for the reliable-delivery ack: :recorded means the
+      # update is durably written (and will be acked); :gap triggered a resync
+      # (no ack); :noop carried no change.
+      outcome
+    end
+
+    # Ask this connection's client to resync: re-send SyncStep1 carrying the
+    # server's current (gap-free) state vector. The client replies SyncStep2
+    # with everything the server is missing, delivered as one causally-complete
+    # delta -- which heals the gap that triggered the resync.
+    def sync_request_resync(awareness)
+      sync_transmit(awareness.sync_step1)
+    end
+
+    # Reliable delivery: acknowledge an accepted update back to the sending
+    # connection. An ack-aware client tags each outgoing update with an "id"
+    # and retains it until the matching `{ "ack" => id }` returns, retransmitting
+    # on a timer or reconnect; idempotent CRDT apply makes resends free. We ack
+    # only when the client supplied an id (so stock clients are unaffected) and
+    # the update was actually accepted -- recorded in audit mode, applied in fast
+    # mode. A gapped update gets no ack (it got a resync), so the client keeps
+    # retransmitting until the missing range lands and the update can integrate.
+    def sync_send_ack(id, outcome)
+      return if id.nil?
+      return unless %i[recorded applied].include?(outcome)
+
+      # Braces are load-bearing: a bare hash would bind to transmit's `via:`
+      # keyword instead of its positional data argument.
+      transmit({ "ack" => id })
     end
 
     # Single broadcast point for both paths (and presence removal), so the
@@ -314,19 +396,40 @@ module YrbLite
     # changes are recorded durably before relay and then broadcast, and
     # awareness is relayed best-effort. Echoing back to the sender is harmless,
     # since the CRDT apply is idempotent.
+    #
+    # Returns an outcome symbol for the reliable-delivery ack: :recorded when a
+    # document update was durably recorded and relayed, :gap when it was
+    # rejected for a resync, :noop for everything else.
     def sync_receive_store_backed(encoded, bytes)
       case Sync.codec.message_kind(bytes)
       when MSG_KIND_SYNC_STEP1
         result = sync_load_doc.handle_sync_message(bytes)
         sync_transmit(result[2]) if result
+        :noop
       when MSG_KIND_UPDATE
         update = Sync.codec.update_from_message(bytes)
-        return unless update
+        return :noop unless update
+
+        # Store mode keeps no warm replica, so to tell whether this update is
+        # causally ready we rebuild the doc from the store and check against it.
+        # That's an O(history) load per update (mitigated by snapshotting the
+        # store on the load path). A gappy update -- an earlier one was lost or
+        # its record failed -- is rejected and the client asked to resync,
+        # rather than written to the log as a permanently-pending entry.
+        doc = sync_load_doc
+        unless doc.update_ready?(update)
+          sync_transmit(doc.sync_step1)
+          return :gap
+        end
 
         self.class.on_change&.call(@sync_key, update) # record before relay
         sync_distribute(encoded)
+        :recorded
       when MSG_KIND_AWARENESS
         sync_distribute(encoded)
+        :noop
+      else
+        :noop
       end
     end
 

data/lib/yrb_lite/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module YrbLite
-  VERSION = "0.1.0.beta1"
+  VERSION = "0.1.0.beta2"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: yrb-lite
 version: !ruby/object:Gem::Version
-  version: 0.1.0.beta1
+  version: 0.1.0.beta2
 platform: ruby
 authors:
 - JP Camara