dionysus-rb 1.5.0 → 1.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fa1bffce325e2384847246f0125606bb90dc46d3226d32764e3ad650a74dccc4
-  data.tar.gz: 07fec41263e2726ddabdcc2cd8b5fae1dcd7452185013e2e9eda2d68b9ec6007
+  metadata.gz: 1b3fb6e16f40d4396b115342f72098bbb2b3f56d8fb630898123e63acce866dc
+  data.tar.gz: ff8f56aa9b8a142131d92b53d83f7014b987439158c39a7518c096f9e05de980
 SHA512:
-  metadata.gz: 76e8759abcf8533a5f3ef0559b97f3fb7fdcc379552204ef33456820108acd0271e88cc9c66f757f539731dbd0d95e574dd4afaa9e5e537e18468a191ae4c26c
-  data.tar.gz: d7f572eb5e5440bf599a38d33fb31c78d194d226a414f17890f536edf9150e58193242a761f47a46e4073d532c7001e676fe48487812a45cee5d66d711ec910b
+  metadata.gz: 9250e85c200c5190a6f85fef44404fa0576d39621960af375b3cbd3cd7ffe172b19fff8021c4f32b1cd63ac453457c6c3bffd504abac9851091d3f4091530b45
+  data.tar.gz: 28194d92924dbac55258734f66ed3a4090ebe514d2edf8dcfd0368a244c4373111c70be23a32b1d3fb08f271f52ade9434529b4ea6dd91015bee8d0bfaa077b5

data/CHANGELOG.md

@@ -1,5 +1,13 @@
 ## [Unreleased]
 
+## [1.6.0]
+- Judge a payload's embedded children on their own timestamps instead of dropping them with the parent. When `persist_with_dionysus?` rejects a record, `persist` used to `next` past the whole loop body - the child `persist` calls included - so every `has_many` and `has_one` record embedded in that payload was discarded with it, with no error and no counter. The parent is still not written; only the traversal of its children is decoupled from its verdict. Each child re-enters `persist` and is guarded on its own `synced_updated_at || synced_created_at`, so a child the consumer has never seen is created and a child older than the row held locally is still skipped.
+- Reproduced at the consumer, not argued: `spec/embedded_children_persistence_spec.rb` runs a whole batch through the generated consumer with deduplication in place, from two failures observed in production. In the first, all seven money-bearing messages carried an `updated_at` 44 ms older than the zero-money payload already accepted, and the payment embedded in them - a row that did not exist locally - was never created. Three control examples pin the fixture down: the same shape at a timestamp the guard accepts persists the money, creates the embedded payment and calls `resolve_to_many_association`, so a red assertion is a statement about this gem rather than about the payload being malformed.
+- `resolve_to_many_association` and `resolve_to_one_association` are suppressed for the whole subtree beneath a rejected parent. Cancelling or deleting the children absent from the payload's list is how a genuine deletion propagates, and it requires a complete current list - which is exactly what a payload already judged stale cannot provide. Consuming applications implement that hook destructively, several with `destroy_all`, and the old `next` was shielding them by accident; suppressing it deliberately keeps that behaviour rather than changing it.
+- Children beneath a rejected parent are compared strictly, through the new `SynchronizableModel#advances_dionysus_state?`. The ordinary guard accepts a tie, and a purge stamps the cancel column with `update_all` without bumping `synced_updated_at` - so a stale payload that merely ties on a child's timestamp is precisely how a cancelled child would be revived. A tie carries nothing that is not already held, so requiring the child to be demonstrably newer closes that without costing a repair. `persist_with_dionysus?` itself is untouched and behaves as before on the ordinary path. A child the payload carries no timestamp for at all is refused outright for the same reason: a payload already judged stale proves nothing about it, and where `resolve_to_many_association` hard-deletes, applying one would bring back a child a fresher payload had removed.
+- The stale verdict is sticky for the whole subtree, which is a trade rather than a free win: a child that is itself fresh now advances while its own children keep the generation they had, because the purge needs a complete list that only a non-stale ancestor can vouch for. Before this change the child did not advance either, so the subtree stayed internally consistent and equally wrong; a later payload that passes the root guard repairs it. Recorded as a pending example.
+- Not fixed here, and left as a pending example rather than a silent gap: when deduplication keeps a message whose `updated_at` the guard rejects while a sibling in the same group would have been accepted, the parent's own columns are still lost. Recovering that needs the guard - not a payload timestamp - to choose the survivor. Keeping siblings whose `updated_at` looks newer is disproven by the existing torn-payload fixture in the deduplication specs, where the correct payload carries the *lower* `updated_at`; ranking on any single key has a production counterexample, so `RemoveDuplicatesStrategy` is unchanged. A consuming application that needs the parent half today can pass `params_batch_transformation: nil` for the topic, which is covered by an example here.
+
 ## [1.5.0]
 - Publish the survivor of a collapsed run of duplicates once more after a delay, behind `config.republish_deduplicated_records` (default `false`) with `config.republish_deduplicated_records_delay` (default 30 seconds). It does nothing unless `remove_consecutive_duplicates_before_publishing` is also on, since there is no run to collapse otherwise.
 - What a collapsed run means. `DuplicatesFilter` keeps the last record of each consecutive run for the same resource, event and topic. A run longer than one only happens when the record was written again while an earlier message for it was still queued - which is the same window in which a payload can be serialized across a write and end up carrying an `updated_at` older than its own contents. `publish_consistent_snapshots` bounds its retries and publishes the last attempt regardless, so a payload that may still be torn does get published; this schedules one more publish once the writes have settled.

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    dionysus-rb (1.5.0)
+    dionysus-rb (1.6.0)
       activerecord (>= 5)
       activesupport (>= 3.2)
       concurrent-ruby

data/lib/dionysus/consumer/persistor.rb

@@ -9,14 +9,14 @@ class Dionysus::Consumer::Persistor
     @topic = topic
   end
 
-  def persist(dionysus_event, batch_number)
+  def persist(dionysus_event, batch_number, under_stale_parent: false)
     if dionysus_event.generic_event?
       if dionysus_event.created? && topic.options[:import] == true
         persist_via_dionysus_create(dionysus_event, batch_number)
       elsif dionysus_event.destroyed? && topic.options[:import] == true
         persist_via_dionysus_destroy(dionysus_event, batch_number)
       else
-        persist_standard_event(dionysus_event, batch_number)
+        persist_standard_event(dionysus_event, batch_number, under_stale_parent: under_stale_parent)
       end
     else
       log_unknown_event_type(dionysus_event)
@@ -43,12 +43,9 @@ class Dionysus::Consumer::Persistor
     config.model_factory.for_model(dionysus_event.model_name)
   end
 
-  def persist_standard_event(dionysus_event, batch_number)
+  def persist_standard_event(dionysus_event, batch_number, under_stale_parent: false)
     Array.wrap(dionysus_event.transformed_data).each do |deseralized_record|
       model_klass = find_model_klass(dionysus_event) or return
-      attributes = deseralized_record.attributes
-      has_one_relationships = deseralized_record.has_one
-      has_many_relationships = deseralized_record.has_many
       synced_id = deseralized_record.synced_id
 
       if synced_id.nil?
@@ -59,38 +56,53 @@ class Dionysus::Consumer::Persistor
       record = Dionysus::Consumer::SynchronizableModel.new(config,
         model_klass.find_or_initialize_by(config.synced_id_attribute => synced_id))
       event_updated_at = deseralized_record.synced_updated_at || deseralized_record.synced_created_at
+      persistable = persistable?(record, event_updated_at, under_stale_parent)
 
-      next unless record.persist_with_dionysus?(event_updated_at)
+      apply_record(dionysus_event, record, deseralized_record, synced_id, batch_number) if persistable
 
-      record.assign_attributes_from_dionysus(attributes)
-      if dionysus_event.destroyed?
-        instrument("dionysus.consume.#{topic}.batch_number_#{batch_number}.persist.remove_with_dionysus") do
-          record.remove_with_dionysus(deseralized_record) if dionysus_event.aggregate_root?
-        end
-      else
-        instrument("dionysus.consume.#{topic}.batch_number_#{batch_number}.persist.restore_with_dionysus") do
-          record.restore_with_dionysus if record.restorable?(deseralized_record)
-        end
-      end
+      # children re-enter persist and are guarded on their own timestamp; only the child list is untrusted
+      persist_relationships(dionysus_event, record, deseralized_record, batch_number,
+        under_stale_parent: under_stale_parent || !persistable)
+    end
+  end
 
-      dionysus_event.local_changes[[dionysus_event.model_name, synced_id]] = record.changes if record.changes.present?
+  def persistable?(record, event_updated_at, under_stale_parent)
+    if under_stale_parent
+      record.advances_dionysus_state?(event_updated_at)
+    else
+      record.persist_with_dionysus?(event_updated_at)
+    end
+  end
 
-      instrument("dionysus.consume.#{topic}.batch_number_#{batch_number}.persist.save") do
-        record.save unless record.destroyed?
+  def apply_record(dionysus_event, record, deseralized_record, synced_id, batch_number)
+    record.assign_attributes_from_dionysus(deseralized_record.attributes)
+    if dionysus_event.destroyed?
+      instrument("dionysus.consume.#{topic}.batch_number_#{batch_number}.persist.remove_with_dionysus") do
+        record.remove_with_dionysus(deseralized_record) if dionysus_event.aggregate_root?
+      end
+    else
+      instrument("dionysus.consume.#{topic}.batch_number_#{batch_number}.persist.restore_with_dionysus") do
+        record.restore_with_dionysus if record.restorable?(deseralized_record)
       end
+    end
+
+    dionysus_event.local_changes[[dionysus_event.model_name, synced_id]] = record.changes if record.changes.present?
+
+    instrument("dionysus.consume.#{topic}.batch_number_#{batch_number}.persist.save") do
+      record.save unless record.destroyed?
+    end
+  end
 
-      instrument("dionysus.consume.#{topic}.batch_number_#{batch_number}.persist.persist_to_many_relationships") do
-        has_many_relationships.each do |relationship_name, relationship_records|
-          persist_to_many_relationship(dionysus_event, relationship_name, record, relationship_records,
-            batch_number)
-        end
+  def persist_relationships(dionysus_event, record, deseralized_record, batch_number, under_stale_parent:)
+    instrument("dionysus.consume.#{topic}.batch_number_#{batch_number}.persist.persist_to_many_relationships") do
+      deseralized_record.has_many.each do |relationship|
+        persist_to_many_relationship(dionysus_event, relationship, record, batch_number, under_stale_parent: under_stale_parent)
       end
+    end
 
-      instrument("dionysus.consume.#{topic}.batch_number_#{batch_number}.persist.persist_to_one_relationships") do
-        has_one_relationships.each do |relationship_name, relationship_record|
-          persist_to_one_relationship(dionysus_event, relationship_name, record, relationship_record,
-            batch_number)
-        end
+    instrument("dionysus.consume.#{topic}.batch_number_#{batch_number}.persist.persist_to_one_relationships") do
+      deseralized_record.has_one.each do |relationship|
+        persist_to_one_relationship(dionysus_event, relationship, record, batch_number, under_stale_parent: under_stale_parent)
       end
     end
   end
@@ -99,7 +111,9 @@ class Dionysus::Consumer::Persistor
     Dionysus.logger.debug("[Dionysus] unknown event type #{dionysus_event.event_name}")
   end
 
-  def persist_to_one_relationship(original_event, relationship_name, parent_model_record, record, batch_number)
+  def persist_to_one_relationship(original_event, relationship, parent_model_record, batch_number,
+    under_stale_parent: false)
+    relationship_name, record = relationship
     instrumentation_arguments = {
       event_name: original_event.event_name,
       parent_model_record: parent_model_record.model_name.to_s,
@@ -115,17 +129,21 @@ class Dionysus::Consumer::Persistor
         relationship_name, records, aggregate_root: false)
 
       instrument("dionysus.consume.#{topic}.batch_number_#{batch_number}.persist.persist_to_one_relationship.#{relationship_name}.persist") do
-        persist(dionysus_event, batch_number)
+        persist(dionysus_event, batch_number, under_stale_parent: under_stale_parent)
         original_event.local_changes.merge!(dionysus_event.local_changes)
       end
 
+      return if under_stale_parent
+
       instrument("dionysus.consume.#{topic}.batch_number_#{batch_number}.persist.persist_to_one_relationship.#{relationship_name}.resolve_to_one_association") do
         parent_model_record.resolve_to_one_association(relationship_name, record.synced_id)
       end
     end
   end
 
-  def persist_to_many_relationship(original_event, relationship_name, parent_model_record, records, batch_number)
+  def persist_to_many_relationship(original_event, relationship, parent_model_record, batch_number,
+    under_stale_parent: false)
+    relationship_name, records = relationship
     instrumentation_arguments = {
       event_name: original_event.event_name,
       parent_model_record: parent_model_record.model_name.to_s,
@@ -140,10 +158,13 @@ class Dionysus::Consumer::Persistor
         relationship_name, records, aggregate_root: false)
 
       instrument("dionysus.consume.#{topic}.batch_number_#{batch_number}.persist.persist_to_many_relationship.#{relationship_name}.persist") do
-        persist(dionysus_event, batch_number)
+        persist(dionysus_event, batch_number, under_stale_parent: under_stale_parent)
         original_event.local_changes.merge!(dionysus_event.local_changes)
       end
 
+      # the purge needs a complete current list, which a payload already judged stale cannot provide
+      return if under_stale_parent
+
       synced_ids_of_related_records = records.map(&:synced_id)
       instrument("dionysus.consume.#{topic}.batch_number_#{batch_number}.persist.persist_to_many_relationship.#{relationship_name}.resolve_to_many_association") do
         parent_model_record.resolve_to_many_association(relationship_name, synced_ids_of_related_records)

data/lib/dionysus/consumer/synchronizable_model.rb

@@ -25,6 +25,18 @@ class Dionysus::Consumer::SynchronizableModel < SimpleDelegator
     (synced_at && event_updated_at && event_updated_at >= synced_at) || synced_at.nil? || event_updated_at.nil?
   end
 
+  # A purge stamps the cancel column without bumping synced_updated_at, so under a stale payload a tie
+  # on the child's timestamp is how a cancelled child gets revived. Demand demonstrably newer.
+  # An unstamped child is refused outright: a payload the guard has already rejected proves nothing
+  # about a child it carries no timestamp for, and applying one would resurrect a child that a fresher
+  # payload had already deleted. Refusing it is exactly what the old blanket `next` did.
+  def advances_dionysus_state?(event_updated_at)
+    return false if event_updated_at.nil?
+
+    persist_with_dionysus?(event_updated_at) &&
+      !(synced_at && event_updated_at == synced_at)
+  end
+
   def assign_attributes_from_dionysus(attributes)
     public_send("#{synced_data_attribute}=", attributes)
     reverse_mapping = config.attributes_mapping_for_model(model.model_name).to_a.to_h(&:reverse)

data/lib/dionysus/version.rb

@@ -3,5 +3,5 @@
 module Dionysus
   module Version
   end
-  VERSION = "1.5.0"
+  VERSION = "1.6.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: dionysus-rb
 version: !ruby/object:Gem::Version
-  version: 1.5.0
+  version: 1.6.0
 platform: ruby
 authors:
 - Karol Galanciak
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2026-08-31 00:00:00.000000000 Z
+date: 2026-09-01 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord