serega 0.36.0 → 0.37.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 461e4c9b6b9dcaf1c93cc490c8a4270bcc8ae576eed523ef14ee3bc1e6be0944
-  data.tar.gz: 7ac9f86f0952070deb5c6a5a812ec9384d9cadab9f783fa5ae5e705ff6cd3116
+  metadata.gz: cf4c5f1a34256084ebedcba18d08dcdc30f9a3090ca3b555a7ee91fc8e0dfc44
+  data.tar.gz: 05022337ad49d098c50f25c267f33a2db6c210b169f3243b88c010a5f9bc519e
 SHA512:
-  metadata.gz: addb468352c4adc023c4f04e30dc6533fefbf7faba9755e02cd243e9f4631c30a0b3b4dd65b160a0f5c012e76717e2d67ddf3c0ac712e5cd0b3008ea3cd8df7d
-  data.tar.gz: da936fb8dd2fac9bafe4e670da795f3b3075d591b3c88411b306898d9a59f944354dc8b69941e19b7b2012bda6a6111a25f9dd0347d5ad8c76b56f310c0353ec
+  metadata.gz: a9b5a2797086500903904617124810a4401f0a223ea75ba88d78c6cf3cba90157e9ff666f54fd76183fbbed4ab54d7be33704fd7c45665537e825f5f0f67222c
+  data.tar.gz: 150fcf62d92d01076c61d38a4189a31d789904fc643c2ce06c569280099d598582e3e07b7ff32a3482e0754d47045123dcf7f01baf7f7f65afebe2c1465e3c93

data/README.md

@@ -1,7 +1,7 @@
 # Serega Ruby Serializer
 
 [![Gem Version](https://badge.fury.io/rb/serega.svg)](https://badge.fury.io/rb/serega)
-[![GitHub Actions](https://github.com/aglushkov/serega/actions/workflows/main.yml/badge.svg?event=push)](https://github.com/aglushkov/serega/actions/workflows/main.yml)
+[![GitHub Actions][build-badge]][build]
 
 The Serega Ruby Serializer provides easy and powerful DSL to describe your
 objects and serialize them to Hash or JSON.
@@ -428,28 +428,38 @@ end
 
 ## Preloads
 
-Serega includes built-in preloads functionality that allows you to define
-`:preloads` to attributes and then merge preloads from serialized attributes
-into a single associations hash.
+Serega includes built-in preloads functionality that lets you declare
+`:preload` on attributes. It is used by the `:activerecord_preloads` plugin:
+enable it and every association you declare is loaded once during
+serialization, automatically, with no N+1.
 
 Configuration options:
 
 - `config.auto_preload` - default `false` (use `true` or `{ has_delegate_option: true, has_serializer_option: true }`)
-- `config.hide_by_default` - default `false` (use `[:preload]` to hide attributes with preloads)
+- `config.hide_by_default` - default `false` (use `:auto` to hide attributes with preloads or batch loaders)
 
 These options are extremely useful if you want to forget about finding
 preloads manually.
 
-Preloads can be disabled with the `preload: false` attribute option.
-Automatically added preloads can be overwritten with the manually specified
-`preload: :xxx` option.
+The `:preload` value is passed to the preload handler exactly as written — a
+Symbol, Array, Hash, or any custom value your ORM understands. Preloads can be
+disabled with `preload: false` (or `preload: nil`); `preload: true` is not a
+valid value. Automatically added preloads can be overwritten with the manually
+specified `preload: :xxx` option.
+
+The actual loading is performed by a handler registered with `preload_with`. The
+`:activerecord_preloads` plugin registers one for you; to preload with another
+ORM — or to attach data to plain non-ORM objects by your own rules — register
+your own (see [Custom preloading](#custom-preloading)). Declaring `:preload`
+without a registered handler raises an error, so a preload never silently does
+nothing.
 
 For some examples, **please read the comments in the code below**
 
 ```ruby
 class AppSerializer < Serega
   config.auto_preload = true
-  config.hide_by_default = [:preload]
+  config.hide_by_default = :auto
 end
 
 class UserSerializer < AppSerializer
@@ -473,36 +483,30 @@ class AlbumSerializer < AppSerializer
   attribute :images_count, delegate: { to: :album_stats }
 end
 
-# By default, preloads are empty, as we specify `hide_by_default = [:preload]`
-# so attributes with preloads will be skipped and nothing will be preloaded
-UserSerializer.new.preloads
-# => {}
-
-UserSerializer.new(with: :followers_count).preloads
-# => {:user_stats=>{}}
-
-UserSerializer.new(with: %i[followers_count comments_count]).preloads
-# => {:user_stats=>{}}
-
-UserSerializer.new(
-  with: [:followers_count, :comments_count, { albums: :images_count }]
-).preloads
-# => {:user_stats=>{}, :albums=>{:album_stats=>{}}}
+# With `hide_by_default = :auto`, attributes that declare a preload are hidden
+# unless explicitly requested with `:with`. When requested, their associations
+# are preloaded automatically. For example:
+#
+#   UserSerializer.to_h(users, with: [:followers_count, { albums: :images_count }])
+#
+# preloads `:user_stats` and `:albums` on the users, and `:album_stats` on the
+# albums.
 ```
 
 ---
 
-### SPECIFIC CASE #1: Serializing the same object in association
+### SPECIFIC CASE: Serializing the same object in association
 
 For example, you show your current user as "user" and use the same user object
 to serialize "user_stats". `UserStatSerializer` relies on user fields and any
-other user associations. You should specify `preload: nil` to preload
-`UserStatSerializer` nested associations to the "user" object.
+other user associations. Specify `preload: nil` so that nothing is preloaded for
+this attribute on the "user" object — the associations declared in
+`UserStatSerializer` are still preloaded onto the same user object.
 
 ```ruby
 class AppSerializer < Serega
   config.auto_preload = true
-  config.hide_by_default = [:preload]
+  config.hide_by_default = :auto
 end
 
 class UserSerializer < AppSerializer
@@ -514,69 +518,47 @@ class UserSerializer < AppSerializer
 end
 ```
 
-### SPECIFIC CASE #2: Serializing multiple associations as a single relation
+---
 
-For example, "user" has two relations - "new_profile" and "old_profile". Also
-profiles have the "avatar" association. And you decided to serialize profiles in
-one array. You can specify `preload_path: [[:new_profile], [:old_profile]]` to
-achieve this:
+### Custom preloading
 
-```ruby
-class AppSerializer < Serega
-  config.auto_preload = true
-end
-
-class UserSerializer < AppSerializer
-  attribute :username
-  attribute :profiles,
-    serializer: 'ProfileSerializer',
-    value: proc { |user| [user.new_profile, user.old_profile] },
-    preload: [:new_profile, :old_profile],
-    preload_path: [[:new_profile], [:old_profile]] # <--- like here
-end
+The `:activerecord_preloads` plugin works out of the box, but the preload
+mechanism itself is not tied to ActiveRecord — or to ORMs at all. Register a
+handler with `preload_with` and it is called once per preloaded attribute with
+the gathered objects and that attribute's `:preload` value, before the
+attribute is serialized:
 
-class ProfileSerializer < AppSerializer
-  attribute :avatar, serializer: 'AvatarSerializer'
-end
+```ruby
+class UserSerializer < Serega
+  # `objects` is the gathered users; `preloads` is this attribute's
+  # `:preload` value (`:posts`). `MyORM.preload` does the eager loading.
+  preload_with { |objects, preloads| MyORM.preload(objects, preloads) }
 
-class AvatarSerializer < AppSerializer
+  attribute :name
+  attribute :posts, serializer: PostSerializer, preload: :posts
 end
-
-UserSerializer.new.preloads
-# => {:new_profile=>{:avatar=>{}}, :old_profile=>{:avatar=>{}}}
-```
-
-### SPECIFIC CASE #3: Preload association through another association
-
-```ruby
-attribute :image,
-  preload: { attachment: :blob }, # <--------- like this one
-  value: proc { |record| record.attachment },
-  serializer: ImageSerializer,
-  preload_path: [:attachment] # or preload_path: [:attachment, :blob]
 ```
 
-In this case, we don't know if preloads defined in ImageSerializer, should be
-preloaded to `attachment` or `blob`, so `preload_path` must be specified manually.
-You can specify `preload_path: nil` if you are sure that there are no preloads
-inside ImageSerializer.
+The handler can be a block or any callable taking `(objects, preloads)`. It is
+inherited by child serializers and can be overridden per subclass. The
+`:preload` value reaches the handler untouched, so you can pass whatever your
+handler expects — not necessarily something an ORM understands. The gathered
+objects can be plain Ruby objects too: the handler decides where the data comes
+from (another ORM, an HTTP API, a cache) and how to attach it to the objects,
+by whatever rules you define.
 
 ---
 
-📌 The built-in preloads functionality only allows to group preloads together
-in single Hash, but they should be preloaded manually.
-
-There is the [activerecord_preloads][activerecord_preloads] plugin that can be used to preload these associations automatically.
-
 ## Plugins
 
 ### Plugin :activerecord_preloads
 
 Automatically preloads associations to serialized objects.
 
-It takes all defined preloads from serialized attributes (including attributes
-from serialized associations), merges them into a single associations hash, and then
-uses ActiveRecord::Associations::Preloader to preload associations to objects.
+Every association you declare with `:preload` is loaded once during
+serialization using `ActiveRecord::Associations::Preloader`, so there are no
+N+1 queries. The plugin does this by registering a [`preload_with`](#custom-preloading)
+handler; to use a different ORM, register your own instead.
 
 ```ruby
 class AppSerializer < Serega
@@ -599,11 +581,10 @@ class AlbumSerializer < AppSerializer
 end
 
 UserSerializer.to_h(user)
-# => preloads {users_stats: {}, albums: { downloads: {} }}
+# preloads :user_stats and :albums on the user; the AlbumSerializer level
+# preloads :downloads on the albums
 ```
 
-For testing purposes preloading can be done manually with `#preload_associations_to(obj)` instance method
-
 ### Plugin :root
 
 Allows to add root key to your serialized data
@@ -1028,3 +1009,5 @@ The gem is available as open source under the terms of the [MIT License](https:/
 [root]: #plugin-root
 [string_modifiers]: #plugin-string_modifiers
 [if]: #plugin-if
+[build-badge]: https://github.com/aglushkov/serega/actions/workflows/main.yml/badge.svg?event=push
+[build]: https://github.com/aglushkov/serega/actions/workflows/main.yml

data/VERSION

@@ -1 +1 @@
-0.36.0
+0.37.0

data/lib/serega/attribute.rb

@@ -33,10 +33,6 @@ class Serega
       # @return [Hash, nil] Attribute :preloads option
       attr_reader :preloads
 
-      # Attribute :preloads_path option
-      # @return [Array, nil] Attribute :preloads_path option
-      attr_reader :preloads_path
-
       # Batch loader names required to detect attribute value
       # @return [Array<Symbol>] Batch loader names
       attr_reader :batch_loaders
@@ -143,7 +139,6 @@ class Serega
         @hide = normalizer.hide
         @serializer = normalizer.serializer
         @preloads = normalizer.preloads
-        @preloads_path = normalizer.preloads_path
         @batch_loaders = normalizer.batch_loaders
       end
     end

data/lib/serega/attribute_normalizer.rb

@@ -9,6 +9,10 @@ class Serega
     # AttributeNormalizer instance methods
     #
     module AttributeNormalizerInstanceMethods
+      # Identity loader that routes relation/preload attributes through the batch
+      # mechanism. Its result is never read — the attribute's value block still
+      # computes the value.
+      AUTO_BATCH_LOADER = proc { |records| records.to_h { |record| [record, record] } }
       # Attribute initial params
       # @return [Hash] Attribute initial params
       attr_reader :init_name, :init_opts, :init_block
@@ -119,17 +123,6 @@ class Serega
         @preloads = prepare_preloads
       end
 
-      #
-      # Shows normalized preloads_path for current attribute
-      #
-      # @return [Array] normalized preloads_path of current attribute
-      #
-      def preloads_path
-        return @preloads_path if instance_variable_defined?(:@preloads_path)
-
-        @preloads_path = prepare_preloads_path
-      end
-
       # Shows specified batch loaders names
       # @return [Array<Symbol>] specified serializer
       #
@@ -158,14 +151,8 @@ class Serega
         return hide if (hide == true) || (hide == false)
 
         hide_setting = config.hide_by_default
-
-        case hide_setting
-        when true
-          return true
-        when Array
-          return true if hide_setting.include?(:preload) && preloads # hide when attribute has `:preload` option
-          return true if hide_setting.include?(:batch) && batch_loaders.any? # hide when attribute has `:batch` option
-        end
+        return true if hide_setting == true
+        return true if hide_setting == :auto && (preloads || init_opts.key?(:batch))
 
         # Return nil for undefined value which means "not hide" but allows
         # to change this value by plugins
@@ -207,10 +194,16 @@ class Serega
 
       def prepare_batch_loaders
         batch_opt = init_opts[:batch]
+        return explicit_batch_loaders(batch_opt) if batch_opt
+        return FROZEN_EMPTY_ARRAY unless serializer || preloads
 
-        if batch_opt.nil?
-          []
-        elsif batch_opt == true || batch_opt.respond_to?(:call)
+        loader_name = :"__auto_batch_#{name}__"
+        self.class.serializer_class.batch(loader_name, AUTO_BATCH_LOADER)
+        [loader_name]
+      end
+
+      def explicit_batch_loaders(batch_opt)
+        if batch_opt == true || batch_opt.respond_to?(:call)
           [name]
         elsif batch_opt.is_a?(Symbol)
           [batch_opt]
@@ -237,64 +230,29 @@ class Serega
         AttributeValueResolvers::DelegateResolver.get(delegate_to, key_method_name, allow_nil)
       end
 
-      # Prepares preloads
-      # @return [Hash,nil]
-      #  - Nil means no need to add preloads and nested preloads
-      #  - Hash with any keys will add preloads
-      #  - Empty hash will skip only top-level preloads, but will allow to load nested preloads
+      # Prepares preloads for this attribute.
+      #
+      # The value is passed through as provided (Symbol, Array, Hash, or any
+      # custom value an ORM understands) — it is the data handed to the
+      # serializer's `preload_with` handler.
+      #
+      # @return [Object, nil] preloads as provided, or nil when the attribute has none
       def prepare_preloads
-        preload = init_opts[:preload]
+        # Explicit preload option (false or nil disables preloading)
+        return init_opts[:preload] || nil if init_opts.key?(:preload)
 
-        # Handle explicit preload option
-        if init_opts.key?(:preload)
-          return nil unless preload # return nil when false or nil
-          return SeregaUtils::FormatUserPreloads.call(preload)
-        end
-
-        # Auto-preload for delegate
+        # Auto-preload the delegated association
         if config.auto_preload.fetch(:has_delegate_option) && init_opts[:delegate]
-          delegate_to = init_opts[:delegate][:to]
-          return SeregaUtils::FormatUserPreloads.call(delegate_to)
+          return init_opts[:delegate][:to]
         end
 
-        # Auto-preload for serializer
+        # Auto-preload the nested serializer's association
         if config.auto_preload.fetch(:has_serializer_option) && init_opts[:serializer] && !init_opts.key?(:batch)
-          return SeregaUtils::FormatUserPreloads.call(method_name)
+          return method_name
         end
 
         nil
       end
-
-      def prepare_preloads_path
-        path = init_opts.fetch(:preload_path) { default_preload_path(preloads) }
-
-        if path && path[0].is_a?(Array)
-          prepare_many_preload_paths(path)
-        else
-          prepare_one_preload_path(path)
-        end
-      end
-
-      def prepare_one_preload_path(path)
-        return unless path
-
-        case path
-        when Array
-          path.map(&:to_sym).freeze
-        else
-          [path.to_sym].freeze
-        end
-      end
-
-      def prepare_many_preload_paths(paths)
-        paths.map { |path| prepare_one_preload_path(path) }.freeze
-      end
-
-      def default_preload_path(preloads)
-        return FROZEN_EMPTY_ARRAY if !preloads || preloads.empty?
-
-        [preloads.keys.first]
-      end
     end
 
     extend Serega::SeregaHelpers::SerializerClassHelper

data/lib/serega/batch/attribute_loader.rb

@@ -10,9 +10,10 @@ class Serega
       # Instance methods for AttributeLoader
       module InstanceMethods
         # @param point [SeregaPlanPoint]
-        def initialize(point)
+        # @param level [SeregaBatch::Level] Shared objects and results for this point's plan level
+        def initialize(point, level)
           @point = point
-          @objects = []
+          @level = level
           @serialized_object_attachers = []
         end
 
@@ -21,35 +22,51 @@ class Serega
         # @param attacher [Proc] Proc that attaches found values to originated attributes
         # @return [void]
         def store(object, attacher)
-          objects << object
           serialized_object_attachers << [object, attacher]
         end
 
-        # Loads serialized values for all stored objects for current attribute
-        # @param context [Hash] Current serialization context
+        # Preloads this attribute's associations onto the level's objects, once for
+        # the attribute (not per named batch loader), via the serializer's registered
+        # `preload_with` handler. Raises when `:preload` is declared but no handler
+        # exists, so a declared preload never silently does nothing.
         # @return [void]
-        def load_all(context)
-          batches = {}
-          serializer_class = point.class.serializer_class
+        def preload_associations
+          preloads = point.attribute.preloads
+          return unless preloads
 
-          point.batch_loaders.each do |batch_loader_name|
-            batches[batch_loader_name] ||= load_one(serializer_class, batch_loader_name, context)
+          handler = point.class.serializer_class.preload_with
+          unless handler
+            raise SeregaError, "The :preload option requires a preload handler. Register one with `preload_with` (the :activerecord_preloads plugin does this for you)."
           end
 
+          handler.call(level.objects, preloads)
+        rescue => error
+          reraise_with_serialized_attribute_details(error)
+        end
+
+        # Loads a single named batch for this level's objects. Caching across
+        # attributes that reuse the loader is handled by the Level.
+        # @param loader [SeregaBatch::Loader] Named batch loader
+        # @return [Object] Loaded batch values
+        def load_batch(loader)
+          level.load(loader)
+        rescue => error
+          reraise_with_serialized_attribute_details(error)
+        end
+
+        # Attaches already loaded batch values to every stored object.
+        # @param batches [Hash] Loaded values grouped by batch loader name
+        # @return [void]
+        def attach(batches)
           serialized_object_attachers.each do |object, attacher|
             attacher.call(object, batches)
           end
         end
 
-        private
+        attr_reader :point
+        attr_reader :level
 
-        # Patched in:
-        # - plugin :activerecord_preloads (preloads associations to found AR objects)
-        def load_one(serializer_class, batch_loader_name, context)
-          serializer_class.batch_loaders[batch_loader_name].load(objects, context)
-        rescue => error
-          reraise_with_serialized_attribute_details(error)
-        end
+        private
 
         def reraise_with_serialized_attribute_details(error)
           raise error.exception(<<~MESSAGE.strip)
@@ -58,8 +75,6 @@ class Serega
           MESSAGE
         end
 
-        attr_reader :point
-        attr_reader :objects
         attr_reader :serialized_object_attachers
       end
 

data/lib/serega/batch/attribute_loaders.rb

@@ -13,9 +13,24 @@ class Serega
       #
       # Initializes new batch loaders
       #
-      def initialize
-        @point_batch_loaders = []
+      # @param context [Hash] Serialization context, constant for the whole run
+      #
+      def initialize(context)
+        @context = context
+        @attribute_loaders = []
         @point_index = {}.compare_by_identity
+        @levels = {}.compare_by_identity
+      end
+
+      # Gathers a chunk of objects serialized at a plan level, so every attribute at
+      # that level shares one set of objects to batch load against.
+      #
+      # @param plan [SeregaPlan] Plan serializing these objects
+      # @param objects [Array] Objects being serialized at this level
+      #
+      # @return [void]
+      def add_objects(plan, objects)
+        level_for(plan).add_objects(objects)
       end
 
       # Remembers data for batch serialization:
@@ -26,34 +41,68 @@ class Serega
       #
       # @return [void]
       def remember(point, object, attacher)
-        point_batch_loader = point_index[point]
+        attribute_loader = point_index[point]
 
-        unless point_batch_loader
+        unless attribute_loader
           serializer_class = point.class.serializer_class
-          point_batch_loader = serializer_class::SeregaBatchAttributeLoader.new(point)
-          point_batch_loaders << point_batch_loader
-          point_index[point] = point_batch_loader
+          attribute_loader = serializer_class::SeregaBatchAttributeLoader.new(point, level_for(point.plan))
+          attribute_loaders << attribute_loader
+          point_index[point] = attribute_loader
         end
 
-        point_batch_loader.store(object, attacher)
+        attribute_loader.store(object, attacher)
       end
 
       #
-      # Loads all registered batches and removes them from registered list
+      # Loads all registered batches and attaches loaded values.
       #
-      def load_all(context)
-        point_batch_loaders.each do |point_batch_loader|
-          point_batch_loader.load_all(context)
+      # Iterates over each loader including loaders added during iteration
+      # (child serializers discovered inside a batch flush add new loaders).
+      def load_all
+        i = 0
+        while i < attribute_loaders.size
+          attribute_loader = attribute_loaders[i]
+          attribute_loader.attach(load_batches(attribute_loader))
+          i += 1
         end
       end
 
       private
 
-      # keeps all point_batch_loaders list
-      attr_reader :point_batch_loaders
+      # Serialization context shared by every level
+      attr_reader :context
 
-      # keeps tracking of already added serializers
+      # keeps all AttributeLoader instances
+      attr_reader :attribute_loaders
+
+      # keeps tracking of already added points
       attr_reader :point_index
+
+      # keeps one Level per plan (compare_by_identity: plan instance identifies a level)
+      attr_reader :levels
+
+      def level_for(plan)
+        levels[plan] ||= Level.new(context)
+      end
+
+      # Builds the { batch_loader_name => loaded_values } hash for one attribute.
+      #
+      # Loading is delegated to the attribute's Level, which loads each named batch
+      # once for the whole level: attributes sharing a loader over the same objects
+      # reuse a single result, deeper levels load separately.
+      def load_batches(attribute_loader)
+        attribute_loader.preload_associations
+
+        point = attribute_loader.point
+        serializer_class = point.class.serializer_class
+
+        batches = {}
+        point.batch_loaders.each do |batch_loader_name|
+          loader = serializer_class.batch_loaders[batch_loader_name]
+          batches[batch_loader_name] = attribute_loader.load_batch(loader)
+        end
+        batches
+      end
     end
   end
 end

data/lib/serega/batch/level.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+class Serega
+  #
+  # Batch feature main module
+  #
+  module SeregaBatch
+    #
+    # Objects serialized at one plan level, together with their loaded batch results.
+    #
+    # Every attribute serialized at the same level shares one Level, so a named batch
+    # loader reused by several of them runs once for the whole set of objects. The same
+    # loader over a different level (deeper nesting) uses a different Level and loads
+    # again.
+    #
+    class Level
+      # @return [Array] Objects gathered for this level
+      attr_reader :objects
+
+      # @param context [Hash] Serialization context
+      def initialize(context)
+        @context = context
+        @objects = []
+        @results = {}.compare_by_identity
+      end
+
+      # Adds a chunk of serialized objects to this level.
+      # @param objects [Array] Objects being serialized at this level
+      # @return [void]
+      def add_objects(objects)
+        @objects.concat(objects)
+      end
+
+      # Loads and returns values for the given batch loader, once per loader.
+      #
+      # Freezes the gathered objects on the first load: all of a level's objects
+      # are added before any of its batches load, so sealing the set here guards
+      # against a later stray `add_objects` and against a loader mutating it.
+      #
+      # @param loader [SeregaBatch::Loader] Named batch loader
+      # @return [Object] Loaded values
+      def load(loader)
+        @objects.freeze
+        @results[loader] ||= loader.load(@objects, @context)
+      end
+    end
+  end
+end