mongo_trails 17.0.1 → 17.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 943d508b66d94b893dc1a553eb9f974b17484294750410f3b0b986073c4de415
-  data.tar.gz: 784ec37eed06ee8ffc0674d2f82a9a59d07892bd652a4006ac51b702b7b6e02e
+  metadata.gz: 56dbb9f210528411b5b75400afafe8cedb67a22c9fe815534cf5702ffde9dfc0
+  data.tar.gz: 1b7e64ade5beddbaedb94c28ce4643373314994776e94a67e4af4b2340773b53
 SHA512:
-  metadata.gz: d56c0b823e24c0b18da5dedbbffb79fb1ecd9dc86fdc47c5455353f6a9e50ad242e346aa40d1d1c628985c93ed00a4f13e0947720487bfe4dd13a32b56c3d814
-  data.tar.gz: b6846cf53080d5f5d11c22bc6bfdbf25904a676a019b64f78311c8d44f803d853d7d5d959b7bce9143b9925706d7ed200fb2e9b3112adbfcfa58aed67ad91ea0
+  metadata.gz: 87875ed778feadc9e4d0b36716e12beb84fddf893aae69b222140cbaccf2dfb9ad600c2ef12da86d76c5c1340b25308ba1c138e13d07cfa11ca23f2a10175159
+  data.tar.gz: f034ef6b1babac2f7952d12865730e9bf7cbc1c5bcd00f8f1e570d5ad7ae2773c83aaec8167e40d3341b2670d709854ae8973191de018e63ee13343f3b5883d8

data/README.md

@@ -33,22 +33,6 @@ require 'mongo_trails/mongo_support/config'
 
 Done!
 
-## Restores
-
-Paper Trail records `create`, `update` and `destroy`, but a soft-deleted record brought back by [Paranoia](https://github.com/rubysherpas/paranoia) is restored with `update_columns`, which Paper Trail never sees. Include `MongoTrails::Restorable` in a Paranoia model to record a version with event `restore` (`MongoTrails::RESTORE_EVENT`) each time it is restored:
-
-```ruby
-class Post < ActiveRecord::Base
-  acts_as_paranoid
-  has_paper_trail
-  include MongoTrails::Restorable # after acts_as_paranoid, which defines the restore callbacks
-end
-```
-
-The version's changes are the columns Paranoia writes on restore (`paranoia_restore_attributes`), from their deleted values to the restored ones. Associated records Paranoia leaves deleted (outside the recovery window) get no version. Paranoia is not a dependency of this gem.
-
-To record a restore done some other way, call `record.paper_trail.record_restore(changes)` with each restored column mapped to `[before, after]`, for example `{ 'deleted_at' => [deleted_at, nil] }`. Like `record_update_columns`, it does nothing while Paper Trail is disabled and respects the record size limit.
-
 ## Config Options
 
 `PaperTrail.config.mongo_config = { hosts: ['localhost:27017'], database: 'my_test_db' }` - Options for MongoDB connection
@@ -59,6 +43,21 @@ To record a restore done some other way, call `record.paper_trail.record_restore
 `PaperTrail.config.sidekiq_worker = PaperTrail::WriteVersionWorker` - Worker class
 `PaperTrail.config.sidekiq_options = { queue: :default }` - Options for Sidekiq
 
+`PaperTrail.config.enable_integer_id = true` - Give every version a sequential `integer_id` from a per-collection counter, and order previous/next versions by it. With `false` no counter is used, `integer_id` stays empty and versions are ordered by `created_at`. The first write to a collection with no counter scans the whole collection for its highest `integer_id`, which is expensive on large collections (and on DocumentDB, where no index serves it).
+
+`PaperTrail.config.on_write_error = nil` - A callable receiving `(error, version)` when a version write fails because MongoDB is unreachable or drops the connection (`MongoTrails::Version::WRITE_ERRORS`). Inside a transaction versions are written from the after-commit callbacks, so a raising write skips every after-commit callback behind it; a handler lets them run. Without a handler the error raises. Other errors always raise.
+
+```ruby
+PaperTrail.config.on_write_error = lambda do |error, version|
+  ErrorReporter.report(error, item_type: version.item_type, item_id: version.item_id)
+  version.enqueue_write unless PaperTrail.config.enable_sidekiq # retry through Sidekiq
+end
+```
+
+`PaperTrail::WriteVersionWorker` rebuilds the queued version with `MongoTrails::Version.rebuild` (its `_id`, `created_at` and metadata are kept, `#initialize` is not run again), and treats a version whose `_id` is already stored as written, so a retried write is safe.
+
+`MongoTrails::Version.bulk_insert(versions)` - Inserts many versions in one `insert_many`, allocating their `integer_id`s in one counter update when `enable_integer_id` is on.
+
 ## Releasing
 
 Update `MongoTrails::VERSION` and all appraisal lockfiles, then validate and release:

data/gemfiles/rails_7.gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: ..
   specs:
-    mongo_trails (17.0.1)
+    mongo_trails (17.1.0)
       mongoid (>= 7, < 10)
       ostruct
       paper_trail (>= 16, < 18)
@@ -172,7 +172,7 @@ CHECKSUMS
   mcp (0.8.0) sha256=ae8bd146bb8e168852866fd26f805f52744f6326afb3211e073f78a95e0c34fb
   minitest (5.27.0) sha256=2d3b17f8a36fe7801c1adcffdbc38233b938eb0b4966e97a6739055a45fa77d5
   mongo (2.25.0) sha256=b4663884d23523d24f50c579846956444410f939093e8c2a95fa4b76bb929053
-  mongo_trails (17.0.1)
+  mongo_trails (17.1.0)
   mongoid (9.1.0) sha256=ef7b50f4c84e636850c3431bc3c92f98481f4e1d147ff06a1c21242d0b242a9e
   ostruct (0.6.3) sha256=95a2ed4a4bd1d190784e666b47b2d3f078e4a9efda2fccf18f84ddc6538ed912
   paper_trail (17.0.0) sha256=1c2842061d3874ca7015908e821e2aa14f9b982af2acb2a7974713bf79021c85

data/gemfiles/rails_8.gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: ..
   specs:
-    mongo_trails (17.0.1)
+    mongo_trails (17.1.0)
       mongoid (>= 7, < 10)
       ostruct
       paper_trail (>= 16, < 18)
@@ -172,7 +172,7 @@ CHECKSUMS
   mcp (0.8.0) sha256=ae8bd146bb8e168852866fd26f805f52744f6326afb3211e073f78a95e0c34fb
   minitest (5.27.0) sha256=2d3b17f8a36fe7801c1adcffdbc38233b938eb0b4966e97a6739055a45fa77d5
   mongo (2.25.0) sha256=b4663884d23523d24f50c579846956444410f939093e8c2a95fa4b76bb929053
-  mongo_trails (17.0.1)
+  mongo_trails (17.1.0)
   mongoid (9.1.0) sha256=ef7b50f4c84e636850c3431bc3c92f98481f4e1d147ff06a1c21242d0b242a9e
   ostruct (0.6.3) sha256=95a2ed4a4bd1d190784e666b47b2d3f078e4a9efda2fccf18f84ddc6538ed912
   paper_trail (17.0.0) sha256=1c2842061d3874ca7015908e821e2aa14f9b982af2acb2a7974713bf79021c85

data/gemfiles/rails_8_1.gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: ..
   specs:
-    mongo_trails (17.0.1)
+    mongo_trails (17.1.0)
       mongoid (>= 7, < 10)
       ostruct
       paper_trail (>= 16, < 18)

data/gemfiles/rails_8_paper_trail_16.gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: ..
   specs:
-    mongo_trails (17.0.1)
+    mongo_trails (17.1.0)
       mongoid (>= 7, < 10)
       ostruct
       paper_trail (>= 16, < 18)
@@ -175,7 +175,7 @@ CHECKSUMS
   mcp (0.8.0) sha256=ae8bd146bb8e168852866fd26f805f52744f6326afb3211e073f78a95e0c34fb
   minitest (5.27.0) sha256=2d3b17f8a36fe7801c1adcffdbc38233b938eb0b4966e97a6739055a45fa77d5
   mongo (2.25.0) sha256=b4663884d23523d24f50c579846956444410f939093e8c2a95fa4b76bb929053
-  mongo_trails (17.0.1)
+  mongo_trails (17.1.0)
   mongoid (9.1.0) sha256=ef7b50f4c84e636850c3431bc3c92f98481f4e1d147ff06a1c21242d0b242a9e
   ostruct (0.6.3) sha256=95a2ed4a4bd1d190784e666b47b2d3f078e4a9efda2fccf18f84ddc6538ed912
   paper_trail (16.0.0) sha256=e9b9f0fb1b8b590c8231cfa931b282ba92f90e066e393930a5e1c61ae4c5019d

data/lib/mongo_trails/config.rb

@@ -8,7 +8,9 @@ module PaperTrail
       :enable_sidekiq,
       :sidekiq_worker,
       :sidekiq_options,
-      :mongo_trails_config
+      :mongo_trails_config,
+      :enable_integer_id,
+      :on_write_error
     )
 
     def initialize
@@ -19,6 +21,8 @@ module PaperTrail
       @has_paper_trail_defaults = {}
       @enable_sidekiq = false
       @sidekiq_options = { queue: :default, retry: true, backtrace: false }
+      @enable_integer_id = true
+      @on_write_error = nil
     end
   end
 end

data/lib/mongo_trails/mongo_support/version.rb

@@ -24,9 +24,35 @@ PaperTrail.config.has_paper_trail_defaults = { versions: { class_name: 'MongoTra
 
 module MongoTrails
   class Version
+    # Errors a version write can raise when MongoDB is unreachable or drops the connection. Only these reach
+    # PaperTrail.config.on_write_error; anything else (a validation error, a bug) still raises.
+    WRITE_ERRORS = [Mongo::Error, IOError, SystemCallError, Timeout::Error].freeze
+    DUPLICATE_KEY_ERROR_CODE = 11_000
+
     class << self
       def find(id)
-        find_by(integer_id: id)
+        integer_id_enabled? ? find_by(integer_id: id) : super
+      end
+
+      def integer_id_enabled?
+        PaperTrail.config.enable_integer_id != false
+      end
+
+      # PaperTrail orders previous/next versions by primary key when it is an integer, which here means integer_id.
+      # Without integer_id they are ordered by created_at.
+      def primary_key_is_int?
+        integer_id_enabled?
+      end
+
+      # Rebuilds a version from the JSON of one that was already built (see #enqueue_write), keeping its _id,
+      # created_at and metadata. It does not run #initialize, so build-time defaults are not applied a second time.
+      def rebuild(attributes)
+        document = attributes.to_h.stringify_keys.to_h { |name, value| [name, from_json_value(name, value)] }
+        instantiate(document).tap { |version| version.new_record = true }
+      end
+
+      def duplicate_key_error?(error)
+        error.respond_to?(:code) && error.code == DUPLICATE_KEY_ERROR_CODE
       end
 
       def incrementing_fields
@@ -62,6 +88,17 @@ module MongoTrails
         ((max_integer_id - count + 1)..max_integer_id).to_a
       end
 
+      def bulk_insert(versions)
+        return [] if versions.empty?
+
+        documents = versions.map { |version| bulk_insert_attributes(version) }
+        if integer_id_enabled?
+          integer_ids = next_integer_ids(documents.count { |document| document['integer_id'].nil? })
+          documents.each { |document| document['integer_id'] ||= integer_ids.shift }
+        end
+        collection.insert_many(documents)
+      end
+
       def bulk_insert_attributes(version)
         version = version.is_a?(self) ? version : new(version)
         attrs = version.attributes.dup
@@ -91,6 +128,17 @@ module MongoTrails
 
       private
 
+      # as_json turns ObjectIds and times into strings; every other field type survives JSON as is.
+      def from_json_value(name, value)
+        return value unless value.is_a?(String)
+
+        type = fields[name]&.type
+        return BSON::ObjectId.legal?(value) ? BSON::ObjectId.from_string(value) : value if type == BSON::ObjectId
+        return (Time.zone || Time).parse(value) if [Time, DateTime].include?(type)
+
+        value
+      end
+
       def counter_key(scope)
         "#{name}:#{scope || prefix_map}"
       end
@@ -152,6 +200,17 @@ module MongoTrails
       schedule_version_persistence(bang: true)
     end
 
+    # Queues the write of this version for PaperTrail::WriteVersionWorker (or the configured sidekiq_worker).
+    def enqueue_write
+      worker = defined?(PaperTrail.config.sidekiq_worker.queue) ? PaperTrail.config.sidekiq_worker : PaperTrail::WriteVersionWorker
+      args = attributes.as_json
+      if worker == PaperTrail::WriteVersionWorker
+        worker.set(PaperTrail.config.sidekiq_options).perform_async(args)
+      else
+        worker.perform_async(args)
+      end
+    end
+
     def initialize(data)
       data = data.to_h.deep_symbolize_keys
       item = data.delete(:item)
@@ -236,6 +295,8 @@ module MongoTrails
     end
 
     def assign_integer_id
+      return unless self.class.integer_id_enabled?
+
       self.integer_id ||= self.class.next_integer_id
     end
 
@@ -248,10 +309,19 @@ module MongoTrails
       current_time
     end
 
+    # Inside a transaction this runs from VersionCommitWrap#committed!, among the source record's after-commit
+    # callbacks, so an error raised here skips every callback queued behind it. PaperTrail.config.on_write_error
+    # receives the connection errors instead of letting them raise; without a handler they raise as before.
     def persist_version(version, bang: false)
-      return version.send(:async_save!) if sidekiq_enabled?
+      return version.enqueue_write if sidekiq_enabled?
 
       bang ? version.save! : version.save
+    rescue *WRITE_ERRORS => e
+      handler = PaperTrail.config.on_write_error
+      raise unless handler
+
+      handler.call(e, version)
+      nil
     end
 
     def parse_time(value)
@@ -273,14 +343,7 @@ module MongoTrails
       @parser ||= URI::Parser.new
     end
 
-    def async_save!
-      worker = defined?(PaperTrail.config.sidekiq_worker.queue) ? PaperTrail.config.sidekiq_worker : PaperTrail::WriteVersionWorker
-      args = attributes.as_json
-      if worker == PaperTrail::WriteVersionWorker
-        worker.set(PaperTrail.config.sidekiq_options).perform_async(args)
-      else
-        worker.perform_async(args)
-      end
-    end
+    # Kept for callers of the old private name; use #enqueue_write.
+    alias async_save! enqueue_write
   end
 end

data/lib/mongo_trails/mongo_support/write_version_worker.rb

@@ -4,8 +4,12 @@ module PaperTrail
   class WriteVersionWorker
     include Sidekiq::Job
 
+    # The job carries a version that was already built, so it is rebuilt as is rather than built again. A retry after a
+    # write whose acknowledgement was lost finds its _id already stored: the version is written, so the job is done.
     def perform(obj)
-      MongoTrails::Version.new(obj).save!
+      MongoTrails::Version.rebuild(obj).save!
+    rescue Mongo::Error::OperationFailure => e
+      raise unless MongoTrails::Version.duplicate_key_error?(e)
     end
   end
 end

data/lib/mongo_trails/record_trail.rb

@@ -43,23 +43,8 @@ module PaperTrail
     def record_update_columns(changes)
       return unless enabled?
 
-      save_update_columns_version(changes)
-    end
-
-    # Records a 'restore' version for a soft-deleted record brought back outside Paper Trail's callbacks,
-    # e.g. by Paranoia, which writes with update_columns. `changes` maps each restored column to
-    # [before, after], such as { 'deleted_at' => [deleted_at, nil] }.
-    def record_restore(changes)
-      return unless enabled?
-
-      save_update_columns_version(changes, event: MongoTrails::RESTORE_EVENT)
-    end
-
-    private
-
-    def save_update_columns_version(changes, event: nil)
-      data = Events::Update.new(@record, false, false, changes).data.merge(data_for_update_columns)
-      data[:event] = event if event
+      event = Events::Update.new(@record, false, false, changes)
+      data = event.data.merge(data_for_update_columns)
       versions_assoc = @record.send(@record.class.versions_association_name)
       version = versions_assoc.new(data)
       return if exceeds_record_size_limit?(version)
@@ -68,6 +53,8 @@ module PaperTrail
       version.save_version
     end
 
+    private
+
     def exceeds_record_size_limit?(version)
       size_limit = PaperTrail.config.mongo_trails_config&.dig(:record_size_limit)
       size_limit && size_limit.to_i < version.to_json.to_s.bytesize

data/lib/mongo_trails/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module MongoTrails
-  VERSION = '17.0.1'
+  VERSION = '17.1.0'
 end

data/lib/mongo_trails.rb

@@ -12,7 +12,6 @@ require 'mongo_trails/events/base'
 require 'mongo_trails/model_config'
 require 'mongo_trails/version_concern'
 require 'mongo_trails/record_trail'
-require 'mongo_trails/restorable'
 
 ActiveSupport.on_load(:active_record) do
   require 'mongo_trails/mongo_support/version'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: mongo_trails
 version: !ruby/object:Gem::Version
-  version: 17.0.1
+  version: 17.1.0
 platform: ruby
 authors:
 - Alex Tsirel
@@ -123,7 +123,6 @@ files:
 - lib/mongo_trails/mongo_support/version_commit_wrap.rb
 - lib/mongo_trails/mongo_support/write_version_worker.rb
 - lib/mongo_trails/record_trail.rb
-- lib/mongo_trails/restorable.rb
 - lib/mongo_trails/version.rb
 - lib/mongo_trails/version_concern.rb
 - mongo_trails.gemspec

data/lib/mongo_trails/restorable.rb

@@ -1,46 +0,0 @@
-# frozen_string_literal: true
-
-module MongoTrails
-  # Paper Trail event of the version recorded when a soft-deleted record is restored (RecordTrail#record_restore)
-  RESTORE_EVENT = 'restore'
-
-  # Records a 'restore' version when a Paranoia model is restored. Paranoia restores with update_columns,
-  # which Paper Trail never sees, so the version history would otherwise jump from 'destroy' to the next update.
-  #
-  #   class Post < ActiveRecord::Base
-  #     acts_as_paranoid
-  #     has_paper_trail
-  #     include MongoTrails::Restorable
-  #   end
-  #
-  # Include it after acts_as_paranoid, which defines the restore callbacks. The version's changes are the columns
-  # Paranoia writes on restore (paranoia_restore_attributes), from their deleted values to the restored ones.
-  # Paranoia runs the callbacks for associated records outside the recovery window too, so only a record that
-  # was actually restored gets a version.
-  module Restorable
-    extend ActiveSupport::Concern
-
-    included do
-      before_restore :remember_mongo_trails_deleted_attributes
-      after_restore :record_mongo_trails_restore
-    end
-
-    private
-
-    def remember_mongo_trails_deleted_attributes
-      @mongo_trails_deleted_attributes = paranoia_destroyed? ? attributes.slice(*mongo_trails_restore_columns) : nil
-    end
-
-    def record_mongo_trails_restore
-      deleted_attributes = @mongo_trails_deleted_attributes
-      @mongo_trails_deleted_attributes = nil
-      return if deleted_attributes.nil? || paranoia_destroyed?
-
-      paper_trail.record_restore(deleted_attributes.to_h { |column, value| [column, [value, self[column]]] })
-    end
-
-    def mongo_trails_restore_columns
-      send(:paranoia_restore_attributes).keys.map(&:to_s)
-    end
-  end
-end