yrby 0.2.3-x86_64-linux → 0.3.1-x86_64-linux

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5adb084218ebd515a711af9d4ad2b051397a11f233335665d43f7a92a9e43105
-  data.tar.gz: ab5215dd6b731b05fa9524f39c1f6d71920c737ac87dc0e6ff1948a42bdb71c0
+  metadata.gz: c620cb3e9292b752df6043cc54100f53ec8b43c09d2936e3128b9a3a7550e03c
+  data.tar.gz: 1842414d1e77000e2c73cee64d6b637ef24be943b7ab50552fd6290475e744e4
 SHA512:
-  metadata.gz: dcaf9f9d47d3850e417c2d9a8be15a429bf6c994c57b40941a4b43c1bb66b3254d83b31d8165fadd2c6068386c6cac927f2c205801059126e5dae06c05eef235
-  data.tar.gz: 10d81ed6a00d6fec909eaf31f26a3a8bc8fe6097a199b37897cddebddf590e05e0aa6dc357c959a30ada68d1aa15110872b4fbbb07d9c034c5373caf7be880e1
+  metadata.gz: 48d5510ba6bd0870cd2715988291f480627160bc8c3b18e2150d51cd2841f610d6f747ad8af4abcd6345d46e8666aaeef478204549040c33d33668bdc308d59c
+  data.tar.gz: cb3e10a0935635b10a755c7ffad0c5bccbb5f21c5e43bdf0dfb9d3d61fe9b8e9f07b9f81ba90cd1db5767700595118deadafac588596a1dbaa5474eab7af9e4e

data/CHANGELOG.md

@@ -6,6 +6,73 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
 
+## [0.3.1] - 2026-07-01
+
+Fixes from a full source review.
+
+### Fixed
+
+- **`Doc#update_ready?` is now exact.** It previously checked only the
+  per-client clock lower bound, but yrs's real integration gate also requires
+  every block referenced by an item's origin / right-origin / parent — which
+  routinely belong to *other* clients — and post-Skip blocks in a merged update
+  sit above the lower bound. An update could pass the clock check yet park as
+  pending; downstream, `update_advances?` then misread the parked update as an
+  already-applied retry (pending doesn't move a state vector) and the sync
+  channel **acked and dropped real content**. `update_ready?` now
+  trial-integrates on a throwaway probe seeded with the doc's integrated state
+  (the clock check remains as a cheap pre-filter), so a cross-client-origin gap
+  is correctly rejected for a resync. `update_advances?` also gained defense in
+  depth: an update that would park reports as advancing, never as a duplicate.
+- **`Doc#read_text` could deadlock the process.** It opened a second read
+  transaction while still holding the first (a chained temporary); yrs's lock is
+  write-preferring, so a concurrent writer between the two acquisitions
+  deadlocked reader-vs-writer inside the GVL-released (uninterruptible) region.
+  Now uses a single transaction.
+- **TOCTOU in gap-free encoding.** The pending check and the encode ran in
+  separate transactions, so a concurrent gappy `apply_update` between them could
+  make `handle_sync_message`/`compacted_state_update` serve pending structs
+  anyway. Both now happen under one transaction.
+- `read_xml`: Lexical soft line breaks and tabs now come through as `\n`/`\t`
+  instead of vanishing (`"foo⏎bar"` no longer extracts as `"foobar"`).
+
+### Changed
+
+- `update_advances?` skips its full-document probe when the update carries
+  blocks beyond the doc's state vector (a novel update trivially advances) —
+  the common case no longer pays O(doc) per frame.
+- The gem no longer packages the `yrby-decoder` gem's files (they ship in that
+  gem; the duplicate copy could shadow a newer standalone release), and now
+  ships `Cargo.lock` so source builds compile the exact crate graph CI tested.
+
+## [0.3.0] - 2026-07-01
+
+### Fixed
+
+- **Sync no longer serves un-integrable pending structs.** When a doc holds a
+  *pending* struct (a gappy update whose causally-prior update is missing — e.g.
+  legacy data recorded before the `update_ready?` gate existed), its integrated
+  state vector is empty but `encode_state_as_update` merges the pending bytes back
+  in. Answering a peer's `SyncStep1` with that state handed the peer content it
+  couldn't integrate, so it parked the same pending forever and the empty-SV /
+  non-empty-content mismatch drove endless resync traffic (observed as a browser
+  re-sending frames several times a second). `handle_sync_message` now answers
+  `SyncStep1` with **integrated-only** state, so a server never serves a struct it
+  can't integrate itself. Neutralizes existing poisoned server state on deploy —
+  no migration needed. The server's own pending is untouched and still heals if
+  the missing dependency later arrives (only then does the content become
+  visible in sync). Live delta relay (`Update` frames) is unchanged.
+
+### Added
+
+- `Doc#pending?` — true if the doc holds un-integrable pending structs or a
+  pending delete set (content waiting on a missing causally-prior update).
+- `Doc#compacted_state_update` — like `encode_state_as_update` (full state) but
+  **gap-free**: excludes pending structs/delete set. Use it when persisting or
+  serving state other peers will apply. Non-destructive — the doc keeps its
+  pending (so it can still heal), and `encode_state_as_update` stays lossless for
+  raw-update recovery.
+
 ## [0.2.3] - 2026-07-01
 
 ### Fixed

data/README.md

@@ -142,18 +142,39 @@ doc = Y::Doc.new(12345) # specific client ID (used for CRDT identity)
 
 # Encoding
 doc.encode_state_vector           # => current state vector
-doc.encode_state_as_update        # => full update
+doc.encode_state_as_update        # => full update (lossless: keeps pending)
 doc.encode_state_as_update(sv)    # => update diff against state vector
+doc.compacted_state_update        # => full update, gap-free (excludes pending)
 
 # Applying updates
 doc.apply_update(update_bytes)    # apply raw V1 update
+doc.pending?                      # => true if holding un-integrable pending structs
 
 # Sync protocol
 doc.sync_step1                    # => SyncStep1 message (this doc's state vector)
 doc.handle_sync_message(data)     # => [msg_type, sync_type, response]; answers a
-                                  #    peer's SyncStep1 with a SyncStep2
+                                  #    peer's SyncStep1 with an integrated-only
+                                  #    SyncStep2 (never serves pending structs)
 ```
 
+### Pending structs and gap-free state
+
+If a doc applies an update whose causally-prior update is missing (a "gappy"
+update), yrs parks it as a **pending** struct: the integrated state vector stays
+empty, but the pending block is held as a recovery buffer and heals if the
+missing dependency later arrives. `Doc#pending?` reports this.
+
+Pending structs are *not* document state, so they must not cross the sync
+boundary — a peer that receives one can't integrate it and gets stuck. Two
+guarantees keep serving safe:
+
+- `handle_sync_message` answers `SyncStep1` with **integrated-only** state, so a
+  server never serves a struct it can't integrate itself (this is automatic).
+- `Doc#compacted_state_update` gives you the same gap-free full-state update for
+  when you persist or hand off state yourself. It's non-destructive (the doc
+  keeps its pending), while `encode_state_as_update` stays lossless so you can
+  still preserve the raw pending bytes for recovery.
+
 ### Protocol codec (module functions)
 
 Classifying and unwrapping wire frames is stateless, so it's exposed as
@@ -221,6 +242,21 @@ servers:
   causal dependencies are already in the store (checked against `on_load`); a
   causally-incomplete update triggers a resync instead, so the log always
   rebuilds cleanly.
+- **An unhealable gap is dropped, not resynced forever.** A resync heals a gap
+  whose missing dependency is still in flight. But a *permanently*-orphaned update
+  (its dependency is gone for good) stays gappy through every resync, and a client
+  retransmitting it would loop endlessly (server resyncs → client resends →
+  repeat). After `gap_strike_limit` rejections of the same update on one
+  connection (default 3, minimum 2), the channel settles it with
+  `{ "ack" => id, "dropped" => true }` and drops it instead of resyncing again —
+  breaking the loop while never dropping a *healable* gap (those heal within a
+  resync or two, and healing frees the strike). The `dropped` flag lets the
+  client surface the loss (`yrby-client` reports it via `onError`) instead of
+  silently showing synced. Set `gap_strike_limit nil` to disable. Works on both
+  transports: plain ActionCable keeps strikes on the channel instance; under
+  AnyCable (fresh instance per RPC command) they persist through
+  anycable-rails' `state_attr_accessor` (istate), declared automatically when
+  anycable-rails is loaded.
 - **`on_change` is at-least-once, and the durable guarantee is that replaying the
   log reconstructs the document.** Every update triggers `on_change` before it's acked or
   broadcast (record-before-distribute). If exactly-once updates matter for you, **you

data/lib/y/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Y
-  VERSION = "0.2.3"
+  VERSION = "0.3.1"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: yrby
 version: !ruby/object:Gem::Version
-  version: 0.2.3
+  version: 0.3.1
 platform: x86_64-linux
 authors:
 - JP Camara
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2026-07-01 00:00:00.000000000 Z
+date: 2026-07-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: minitest
@@ -68,10 +68,7 @@ files:
 - lib/y.rb
 - lib/y/3.4/yrby.so
 - lib/y/4.0/yrby.so
-- lib/y/decoder.rb
-- lib/y/decoder/version.rb
 - lib/y/version.rb
-- lib/yrby-decoder.rb
 - lib/yrby.rb
 homepage: https://github.com/jpcamara/yrby
 licenses:

data/lib/y/decoder/version.rb

@@ -1,7 +0,0 @@
-# frozen_string_literal: true
-
-module Y
-  module Decoder
-    VERSION = "0.1.0.alpha1"
-  end
-end

data/lib/y/decoder.rb

@@ -1,66 +0,0 @@
-# frozen_string_literal: true
-
-require "y"
-require "y/decoder/version"
-
-module Y
-  # Plain-text reconstruction of a stored Yjs document, in pure Ruby — for search
-  # indexing and previews. The core `yrby` gem moves and stores opaque CRDT
-  # updates without reading them; this reads the text out of the shared type the
-  # editor uses (Lexical's `Y.XmlText`, plain `Y.Text`, or ProseMirror's
-  # `Y.XmlFragment`), in-process, on the native extension core already ships — no
-  # Node, no subprocess, no binary.
-  #
-  #   state = doc.encode_state_as_update        # opaque CRDT bytes from the store
-  #   Y::Decoder.text(state)              # => "hello world"
-  #   Y::Decoder.preview(state, 280)      # => "hello world…"
-  #
-  # Full-fidelity reconstruction (the exact Lexical EditorState / HTML, which
-  # needs @lexical/yjs) is a separate, opt-in concern — see the `yrby-decode`
-  # package's Bun binary. This gem stays pure Ruby on purpose.
-  module Decoder
-    class Error < Y::Error; end
-
-    module_function
-
-    # Plain text of the document. `field` pins the root key (Lexical: the editor
-    # id; ProseMirror: "default"); omit it to use the document's sole root.
-    def text(state, field: nil)
-      field ||= Y::Doc.new.tap { |d| d.apply_update(state) }.root_names.first
-      return "" unless field
-
-      # A plain `Y.Text` root (a simple shared-text editor) reads straight out.
-      # (A yrs root's type is fixed by its first typed access, so each reader
-      # gets a fresh doc to try a different shared type against the same state.)
-      direct = load(state).read_text(field)
-      return normalize(direct) if direct && !direct.strip.empty?
-
-      # Lexical (each block a sibling `Y.XmlText`) and ProseMirror (blocks are
-      # `Y.XmlElement`s) both come back from read_xml as block-per-line markup;
-      # strip any element tags to plain text.
-      markup = load(state).read_xml(field)
-      markup ? normalize(strip_tags(markup)) : ""
-    end
-
-    # A compact, single-line preview for list UIs.
-    def preview(state, limit: 280, field: nil)
-      body = text(state, field: field).gsub(/\s+/, " ").strip
-      body.length > limit ? "#{body[0, limit].rstrip}…" : body
-    end
-
-    def load(state)
-      Y::Doc.new.tap { |doc| doc.apply_update(state) }
-    end
-
-    def strip_tags(markup)
-      markup.gsub(/<[^>]*>/, " ")
-    end
-
-    def normalize(text)
-      text.gsub(/[ \t]+/, " ")     # collapse runs of spaces/tabs
-          .gsub(/ *\n */, "\n")    # trim spaces left around block separators
-          .gsub(/\n{3,}/, "\n\n")  # cap blank-line runs
-          .strip
-    end
-  end
-end

data/lib/yrby-decoder.rb

@@ -1,4 +0,0 @@
-# frozen_string_literal: true
-
-# Entry point matching the gem name, so `Bundler.require` loads it automatically.
-require "y/decoder"