occams-record 0.29.0 → 0.31.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f01a553f2a0d8dcb7e2efaac70088239a05de6633375b8bf3f5c1f77a43808c8
-  data.tar.gz: 38095dbb827e293088f515e04bfdb907f088e1bddce0cfdc43f38455e8bafab4
+  metadata.gz: c2c1004e553179f934744fe17227319b53e6fe663d4b6e5c246017c0a7728808
+  data.tar.gz: 502a1fe403a0ccf1980447d7f2b6f960cc3988a72b574df7d30a1d3db3ab69a2
 SHA512:
-  metadata.gz: c221f51d83965ef207975806a2d5d7b4c6867ba7e1f2f9f08e2c2b78320de262d7863cfc1c215b37a12d5716b99377b2ee43d5697284e7e483f81f110572581b
-  data.tar.gz: 37c429c2af89af793ad80bbf87fe4b8083626d540e78cfb4887ffb3eb83c6d4019a4abfc26daf438a58256a36a136d2918d04499eb390cabf62016bf8da97208
+  metadata.gz: 49f84fc3ef76a06740b9c9702d49a2370d3ffaac28f5af2ec7747a911fe14031c477a53eae7b8ca50fb084c8b994186068fad1c272ddab32ed34280ab8ee1a04
+  data.tar.gz: 4615260b822144d68c8a7c858de3f02820fbaffd5c490ae8cdf33eed96bb89529721cc527c435f90f00618c85e88992b02b46bfce2094d96187ee985f61042d8

data/README.md

@@ -2,22 +2,27 @@
 
 > Do not multiply entities beyond necessity. -- Occam's Razor
 
-Occam's Record is a high-efficiency, advanced query library for ActiveRecord apps. It is **not** an ORM or an ActiveRecord replacement. Use it to solve pain points in your existing ActiveRecord app.
+Occam's Record is a high-efficiency, advanced query library for ActiveRecord apps. It is **not** an ORM or an ActiveRecord replacement. Use it to solve pain points in your existing ActiveRecord app. Occams Record gives you two things:
+
+**Performance**
 
 * 3x-5x faster than ActiveRecord queries.
 * Uses 1/3 the memory of ActiveRecord query results.
 * Eliminates the N+1 query problem.
-* Customize the SQL when eager loading associations.
-* `find_each`/`find_in_batches` respects `order` and `limit`.
-* Allows eager loading of associations when querying with raw SQL.
-* Allows `find_each`/`find_in_batches` when querying with raw SQL.
-* Eager load an ad hoc assocation using arbitrary SQL.
+
+**More powerful queries & eager loading**
+
+* Customize the SQL used to eager load associations.
+* Use `ORDER BY` with `find_each`/`find_in_batches`.
+* Use `find_each`/`find_in_batches` with raw SQL.
+* Eager load associations when you're writing raw SQL.
+* Eager load "ad hoc associations" using raw SQL.
 
 [Look over the speed and memory measurements yourself!](https://github.com/jhollinger/occams-record/wiki/Measurements) OccamsRecord achieves all of this by making some very specific trade-offs:
 
-* OccamsRecord results are **read-only**.
-* OccamsRecord results are **purely database rows** - they don't have any instance methods from your Rails models.
-* You **must eager load** each assocation you intend to use. If you forget one, an exception will be raised.
+* OccamsRecord results are *read-only*.
+* OccamsRecord results are *purely database rows* - they don't have any instance methods from your Rails models.
+* You *must eager load* each assocation you intend to use. If you forget one, an exception will be raised.
 
 ---
 
@@ -48,11 +53,11 @@ orders = OccamsRecord.
   run
 ````
 
-`each`, `map`, `reduce`, and other Enumerable methods may be used instead of *run*. `find_each` and `find_in_batches` are also supported. Unlike their ActiveRecord counterparts they respect *ORDER BY*. Occams Record has great support for raw SQL queries too, but we'll get to those later.
+`each`, `map`, `reduce`, and other Enumerable methods may be used instead of *run*. `find_each` and `find_in_batches` are also supported, and unlike their ActiveRecord counterparts they respect *ORDER BY*. Occams Record has great support for raw SQL queries too, but we'll get to those later.
 
 ## Basic eager loading
 
-Eager loading is similiar to ActiveRecord's `preload` (each association is loaded in a separate query). Nested associations use blocks instead of Hashes. And if you try to use an association you didn't eager load an exception will be raised.
+Eager loading is similiar to ActiveRecord's `preload` (each association is loaded in a separate query). Nested associations use blocks instead of Hashes. If you try to use an association you didn't eager load an exception will be raised.
 
 ```ruby
 orders = OccamsRecord.
@@ -60,6 +65,7 @@ orders = OccamsRecord.
   eager_load(:customer).
   eager_load(:line_items) {
     eager_load(:product)
+    eager_load(:something_else)
   }.
   run
   
@@ -80,33 +86,34 @@ Occams Record allows you to customize each eager load query using the full power
 ```ruby
 orders = OccamsRecord.
   query(q).
-  # Only SELECT these two columns. Your DBA will thank you, esp. on "wide" tables.
+  # Only SELECT the columns you need. Your DBA will thank you.
   eager_load(:customer, select: "id, name").
   
-  # A Proc can customize the query using any of ActiveRecord's query builders and
-  # any scopes you've defined on the LineItem model.
+  # A Proc can customize the query using any of ActiveRecord's query
+  # builders and any scopes you've defined on the LineItem model.
   eager_load(:line_items, ->(q) { q.active.order("created_at") }) {
     eager_load(:product)
+    eager_load(:something_else)
   }.
   run
 ```
 
-Occams Record also supports creating ad hoc associations using raw SQL. We'll get to that in the next section.
+Occams Record also supports loading ad hoc associations using raw SQL. We'll get to that in the next section.
 
 ## Raw SQL queries
 
-ActiveRecord has raw SQL "escape hatches" like `find_by_sql` or `exec_query`, but they both give up critical features like eager loading and `find_each`/`find_in_batches`. Not so with Occams Record!
+ActiveRecord has raw SQL escape hatches like `find_by_sql` and `exec_query`, but they give up critical features like eager loading and `find_each`/`find_in_batches`. Occams Record's escape hatches don't make you give up anything.
 
 **Batched loading**
 
-To use `find_each`/`find_in_batches` you must provide the limit and offset statements yourself. OccamsRecord will fill in the values for you. Also, notice that the binding syntax is a bit different (it uses Ruby's built-in named string substitution).
+To use `find_each`/`find_in_batches` you must provide the limit and offset statements yourself; Occams will provide the values. Also, notice that the binding syntax is a bit different (it uses Ruby's built-in named string substitution).
 
 ```ruby
 OccamsRecord.
   sql(%(
     SELECT * FROM orders
     WHERE order_date > %{date}
-    ORDER BY order_date DESC
+    ORDER BY order_date DESC, id
     LIMIT %{batch_limit}
     OFFSET %{batch_offset}
   ), {
@@ -119,14 +126,14 @@ OccamsRecord.
 
 **Eager loading**
 
-To use `eager_load` with a raw SQL query you must tell Occams what the base model is. (That doesn't apply if you're loading an ad hoc, raw SQL association. We'll get to those later).
+To use `eager_load` with a raw SQL query you must tell Occams what the base model is. (That doesn't apply if you're loading an ad hoc, raw SQL association. We'll get to those later.)
 
 ```ruby
 orders = OccamsRecord.
   sql(%(
     SELECT * FROM orders
     WHERE order_date > %{date}
-    ORDER BY order_date DESC
+    ORDER BY order_date DESC, id
   ), {
     date: 30.days.ago
   }).
@@ -153,7 +160,7 @@ products_with_orders = OccamsRecord.
   }
 ```
 
-But that's very wasteful. Occams gives us a better options: `eager_load_many` and `eager_load_one`.
+But that's very wasteful. Occams gives us better options: `eager_load_many` and `eager_load_one`.
 
 ```ruby
 products = OccamsRecord.
@@ -174,7 +181,7 @@ products = OccamsRecord.
 
 `eager_load_one` defines an ad hoc `has_one`/`belongs_to` association.
 
-These ad hoc eager loaders are available on both `OccamsRecord.query` and `OccamsRecord.sql`. While eager loading with `OccamsRecord.sql` normallly requires you to declare the model, that is not necessary with `eager_load_one`/`eager_load_many`.
+These ad hoc eager loaders are available on both `OccamsRecord.query` and `OccamsRecord.sql`. While eager loading with `OccamsRecord.sql` normallly requires you to declare the model, that isn't necessary when using these methods.
 
 ## Injecting instance methods
 
@@ -207,10 +214,11 @@ orders = OccamsRecord.
 
 The following ActiveRecord features are under consideration, but not high priority. Pull requests welcome!
 
-* `:through` associations.
+* Eager loading `through` associations that involve a `has_and_belongs_to_many`.
 
 The following ActiveRecord features are not supported, and likely never will be. Pull requests are still welcome, though.
 
+* Eager loading `through` associations that involve a polymorphic association.
 * ActiveRecord enum types
 * ActiveRecord serialized types
 

data/lib/occams-record/eager_loaders.rb

@@ -3,126 +3,34 @@ module OccamsRecord
   # Contains eager loaders for various kinds of associations.
   #
   module EagerLoaders
+    autoload :Builder, 'occams-record/eager_loaders/builder'
+    autoload :Context, 'occams-record/eager_loaders/context'
+
     autoload :Base, 'occams-record/eager_loaders/base'
     autoload :BelongsTo, 'occams-record/eager_loaders/belongs_to'
     autoload :PolymorphicBelongsTo, 'occams-record/eager_loaders/polymorphic_belongs_to'
     autoload :HasOne, 'occams-record/eager_loaders/has_one'
     autoload :HasMany, 'occams-record/eager_loaders/has_many'
     autoload :Habtm, 'occams-record/eager_loaders/habtm'
+    autoload :Through, 'occams-record/eager_loaders/through'
 
     autoload :AdHocBase, 'occams-record/eager_loaders/ad_hoc_base'
     autoload :AdHocOne, 'occams-record/eager_loaders/ad_hoc_one'
     autoload :AdHocMany, 'occams-record/eager_loaders/ad_hoc_many'
 
-    # Methods for adding eager loading to a query.
-    module Builder
-      #
-      # Specify an association to be eager-loaded. For maximum memory savings, only SELECT the
-      # colums you actually need.
-      #
-      # @param assoc [Symbol] name of association
-      # @param scope [Proc] a scope to apply to the query (optional). It will be passed an
-      # ActiveRecord::Relation on which you may call all the normal query hethods (select, where, etc) as well as any scopes you've defined on the model.
-      # @param select [String] a custom SELECT statement, minus the SELECT (optional)
-      # @param use [Array<Module>] optional Module to include in the result class (single or array)
-      # @param as [Symbol] Load the association usign a different attribute name
-      # @yield a block where you may perform eager loading on *this* association (optional)
-      # @return [OccamsRecord::Query] returns self
-      #
-      def eager_load(assoc, scope = nil, select: nil, use: nil, as: nil, &eval_block)
-        ref = @model ? @model.reflections[assoc.to_s] : nil
-        ref ||= @model.subclasses.map(&:reflections).detect { |x| x.has_key? assoc.to_s }&.[](assoc.to_s) if @model
-        raise "OccamsRecord: No assocation `:#{assoc}` on `#{@model&.name || '<model missing>'}` or subclasses" if ref.nil?
-        scope ||= ->(q) { q.select select } if select
-        @eager_loaders << eager_loader_for_association(ref).new(ref, scope, use: use, as: as, &eval_block)
-        self
-      end
-
-      #
-      # Specify some arbitrary SQL to be loaded into some arbitrary attribute ("name"). The attribute will
-      # hold either one record or none.
-      #
-      # In the example below, :category is NOT an association on Widget. Though if it where it would be a belongs_to. The
-      # mapping argument says "The id column in this table (categories) maps to the category_id column in the other table (widgets)".
-      # The %{ids} bind param will be provided for you, and in this case will be all the category_id values from the main
-      # query.
-      #
-      #   res = OccamsRecord.
-      #     query(Widget.order("name")).
-      #     eager_load_one(:category, {:id => :category_id}, %(
-      #       SELECT * FROM categories WHERE id IN (%{ids}) AND name != %{bad_name}
-      #     ), binds: {
-      #       bad_name: "Bad Category"
-      #     }).
-      #     run
-      #
-      # @param name [Symbol] name of attribute to load records into
-      # @param mapping [Hash] a one element Hash with the key being the local/child id and the value being the foreign/parent id
-      # @param sql [String] the SQL to query the associated records. Include a bind params called '%{ids}' for the foreign/parent ids.
-      # @param binds [Hash] any additional binds for your query.
-      # @param model [ActiveRecord::Base] optional - ActiveRecord model that represents what you're loading. required when using Sqlite.
-      # @param use [Array<Module>] optional - Ruby modules to include in the result objects (single or array)
-      # @yield eager load associations nested under this one
-      #
-      def eager_load_one(name, mapping, sql, binds: {}, model: nil, use: nil, &eval_block)
-        @eager_loaders << EagerLoaders::AdHocOne.new(name, mapping, sql, binds: binds, model: model, use: use, &eval_block)
-        self
-      end
-
-      #
-      # Specify some arbitrary SQL to be loaded into some arbitrary attribute ("name"). The attribute will
-      # hold an array of 0 or more associated records.
-      #
-      # In the example below, :parts is NOT an association on Widget. Though if it where it would be a has_many. The
-      # mapping argument says "The widget_id column in this table (parts) maps to the id column in the other table (widgets)".
-      # The %{ids} bind param will be provided for you, and in this case will be all the id values from the main
-      # query.
-      #
-      #   res = OccamsRecord.
-      #     query(Widget.order("name")).
-      #     eager_load_many(:parts, {:widget_id => :id}, %(
-      #       SELECT * FROM parts WHERE widget_id IN (%{ids}) AND sku NOT IN (%{bad_skus})
-      #     ), binds: {
-      #       bad_skus: ["G90023ASDf0"]
-      #     }).
-      #     run
-      #
-      # @param name [Symbol] name of attribute to load records into
-      # @param mapping [Hash] a one element Hash with the key being the local/child id and the value being the foreign/parent id
-      # @param sql [String] the SQL to query the associated records. Include a bind params called '%{ids}' for the foreign/parent ids.
-      # @param use [Array<Module>] optional - Ruby modules to include in the result objects (single or array)
-      # @param binds [Hash] any additional binds for your query.
-      # @param model [ActiveRecord::Base] optional - ActiveRecord model that represents what you're loading. required when using Sqlite.
-      # @yield eager load associations nested under this one
-      #
-      def eager_load_many(name, mapping, sql, binds: {}, model: nil, use: nil, &eval_block)
-        @eager_loaders << EagerLoaders::AdHocMany.new(name, mapping, sql, binds: binds, model: model, use: use, &eval_block)
-        self
-      end
-
-      private
-
-      # Run all defined eager loaders into the given result rows
-      def eager_load!(rows)
-        @eager_loaders.each { |loader|
-          loader.run(rows, query_logger: @query_logger)
-        }
-      end
-
-      # Fetch the appropriate eager loader for the given association type.
-      def eager_loader_for_association(ref)
-        case ref.macro
-        when :belongs_to
-          ref.options[:polymorphic] ? PolymorphicBelongsTo : BelongsTo
-        when :has_one
-          HasOne
-        when :has_many
-          HasMany
-        when :has_and_belongs_to_many
-          Habtm
-        else
-          raise "Unsupported association type `#{macro}`"
-        end
+    # Fetch the appropriate eager loader for the given association type.
+    def self.fetch!(ref)
+      case ref.macro
+      when :belongs_to
+        ref.polymorphic? ? PolymorphicBelongsTo : BelongsTo
+      when :has_one
+        HasOne
+      when :has_many
+        HasMany
+      when :has_and_belongs_to_many
+        Habtm
+      else
+        raise "Unsupported association type `#{macro}`"
       end
     end
   end

data/lib/occams-record/eager_loaders/ad_hoc_base.rb

@@ -4,6 +4,8 @@ module OccamsRecord
     # Base class for eager loading ad hoc associations.
     #
     class AdHocBase
+      include EagerLoaders::Builder
+
       # @return [String] association name
       attr_reader :name
 
@@ -18,12 +20,14 @@ module OccamsRecord
       # @param use [Array<Module>] optional - Ruby modules to include in the result objects (single or array)
       # @yield eager load associations nested under this one
       #
-      def initialize(name, mapping, sql, binds: {}, model: nil, use: nil, &eval_block)
+      def initialize(name, mapping, sql, binds: {}, model: nil, use: nil, &builder)
         @name = name.to_s
-        @sql, @binds, @use, @model, @eval_block = sql, binds, use, model, eval_block
+        @sql, @binds, @use, @model = sql, binds, use, model
         raise ArgumentError, "Add-hoc eager loading mapping must contain exactly one key-value pair" unless mapping.size == 1
         @local_key = mapping.keys.first
         @foreign_key = mapping.fetch(@local_key)
+        @eager_loaders = EagerLoaders::Context.new(@model)
+        instance_eval(&builder) if builder
       end
 
       #
@@ -36,7 +40,7 @@ module OccamsRecord
         calc_ids(rows) { |ids|
           assoc = if ids.any?
                     binds = @binds.merge({:ids => ids})
-                    RawQuery.new(@sql, binds, use: @use, query_logger: query_logger, &@eval_block).model(@model).run
+                    RawQuery.new(@sql, binds, use: @use, eager_loaders: @eager_loaders, query_logger: query_logger).run
                   else
                     []
                   end

data/lib/occams-record/eager_loaders/base.rb

@@ -4,6 +4,8 @@ module OccamsRecord
     # Base class for eagoer loading an association. IMPORTANT eager loaders MUST remain stateless after initialization!
     #
     class Base
+      include EagerLoaders::Builder
+
       # @return [String] association name
       attr_reader :name
 
@@ -13,12 +15,16 @@ module OccamsRecord
       # ActiveRecord::Relation on which you may call all the normal query hethods (select, where, etc) as well as any scopes you've defined on the model.
       # @param use [Array(Module)] optional Module to include in the result class (single or array)
       # @param as [Symbol] Load the association usign a different attribute name
+      # @param optimizer [Symbol] Only used for `through` associations. Options are :none (load all intermediate records) | :select (load all intermediate records but only SELECT the necessary columns)
       # @yield perform eager loading on *this* association (optional)
       #
-      def initialize(ref, scope = nil, use: nil, as: nil, &eval_block)
-        @ref, @scope, @use, @as, @eval_block = ref, scope, use, as, eval_block
+      def initialize(ref, scope = nil, use: nil, as: nil, optimizer: :select, &builder)
+        @ref, @scope, @use, @as = ref, scope, use, as
         @model = ref.klass
         @name = (as || ref.name).to_s
+        @eager_loaders = EagerLoaders::Context.new(@model)
+        @optimizer = optimizer
+        instance_eval(&builder) if builder
       end
 
       #
@@ -29,7 +35,7 @@ module OccamsRecord
       #
       def run(rows, query_logger: nil)
         query(rows) { |*args|
-          assoc_rows = args[0] ? Query.new(args[0], use: @use, query_logger: query_logger, &@eval_block).run : []
+          assoc_rows = args[0] ? Query.new(args[0], use: @use, eager_loaders: @eager_loaders, query_logger: query_logger).run : []
           merge! assoc_rows, rows, *args[1..-1]
         }
       end

data/lib/occams-record/eager_loaders/builder.rb

@@ -0,0 +1,112 @@
+module OccamsRecord
+  module EagerLoaders
+    #
+    # Methods for adding eager loading to a query.
+    #
+    # Users MUST have an OccamsRecord::EagerLoaders::Context at @eager_loaders.
+    #
+    module Builder
+      #
+      # Specify an association to be eager-loaded. For maximum memory savings, only SELECT the
+      # colums you actually need.
+      #
+      # @param assoc [Symbol] name of association
+      # @param scope [Proc] a scope to apply to the query (optional). It will be passed an
+      # ActiveRecord::Relation on which you may call all the normal query hethods (select, where, etc) as well as any scopes you've defined on the model.
+      # @param select [String] a custom SELECT statement, minus the SELECT (optional)
+      # @param use [Array<Module>] optional Module to include in the result class (single or array)
+      # @param as [Symbol] Load the association usign a different attribute name
+      # @param optimizer [Symbol] Only used for `through` associations. Options are :none (load all intermediate records) | :select (load all intermediate records but only SELECT the necessary columns)
+      # @yield a block where you may perform eager loading on *this* association (optional)
+      # @return [OccamsRecord::Query] returns self
+      #
+      def eager_load(assoc, scope = nil, select: nil, use: nil, as: nil, optimizer: :select, &builder)
+        @eager_loaders.add(assoc, scope, select: select, use: use, as: as, optimizer: optimizer, &builder)
+        self
+      end
+
+      #
+      # Same as eager_load, except it returns the new eager loader object instead of self. You can use the
+      # new object to call "nest" again, programtically building up nested eager loads instead of passing
+      # nested blocks.
+      #
+      # @param assoc [Symbol] name of association
+      # @param scope [Proc] a scope to apply to the query (optional). It will be passed an
+      # ActiveRecord::Relation on which you may call all the normal query hethods (select, where, etc) as well as any scopes you've defined on the model.
+      # @param select [String] a custom SELECT statement, minus the SELECT (optional)
+      # @param use [Array<Module>] optional Module to include in the result class (single or array)
+      # @param as [Symbol] Load the association usign a different attribute name
+      # @param optimizer [Symbol] Only used for `through` associations. Options are :none (load all intermediate records) | :select (load all intermediate records but only SELECT the necessary columns)
+      # @return [OccamsRecord::EagerLoaders::Base] returns self
+      #
+      #
+      def nest(assoc, scope = nil, select: nil, use: nil, as: nil, optimizer: :select)
+        raise ArgumentError, "OccamsRecord::EagerLoaders::Builder#nest does not accept a block!" if block_given?
+        @eager_loaders.add(assoc, scope, select: select, use: use, as: as, optimizer: optimizer) ||
+          raise("OccamsRecord::EagerLoaders::Builder#nest may not be called under a polymorphic association")
+      end
+
+      #
+      # Specify some arbitrary SQL to be loaded into some arbitrary attribute ("name"). The attribute will
+      # hold either one record or none.
+      #
+      # In the example below, :category is NOT an association on Widget. Though if it where it would be a belongs_to. The
+      # mapping argument says "The id column in this table (categories) maps to the category_id column in the other table (widgets)".
+      # The %{ids} bind param will be provided for you, and in this case will be all the category_id values from the main
+      # query.
+      #
+      #   res = OccamsRecord.
+      #     query(Widget.order("name")).
+      #     eager_load_one(:category, {:id => :category_id}, %(
+      #       SELECT * FROM categories WHERE id IN (%{ids}) AND name != %{bad_name}
+      #     ), binds: {
+      #       bad_name: "Bad Category"
+      #     }).
+      #     run
+      #
+      # @param name [Symbol] name of attribute to load records into
+      # @param mapping [Hash] a one element Hash with the key being the local/child id and the value being the foreign/parent id
+      # @param sql [String] the SQL to query the associated records. Include a bind params called '%{ids}' for the foreign/parent ids.
+      # @param binds [Hash] any additional binds for your query.
+      # @param model [ActiveRecord::Base] optional - ActiveRecord model that represents what you're loading. required when using Sqlite.
+      # @param use [Array<Module>] optional - Ruby modules to include in the result objects (single or array)
+      # @yield eager load associations nested under this one
+      #
+      def eager_load_one(name, mapping, sql, binds: {}, model: nil, use: nil, &builder)
+        @eager_loaders << EagerLoaders::AdHocOne.new(name, mapping, sql, binds: binds, model: model, use: use, &builder)
+        self
+      end
+
+      #
+      # Specify some arbitrary SQL to be loaded into some arbitrary attribute ("name"). The attribute will
+      # hold an array of 0 or more associated records.
+      #
+      # In the example below, :parts is NOT an association on Widget. Though if it where it would be a has_many. The
+      # mapping argument says "The widget_id column in this table (parts) maps to the id column in the other table (widgets)".
+      # The %{ids} bind param will be provided for you, and in this case will be all the id values from the main
+      # query.
+      #
+      #   res = OccamsRecord.
+      #     query(Widget.order("name")).
+      #     eager_load_many(:parts, {:widget_id => :id}, %(
+      #       SELECT * FROM parts WHERE widget_id IN (%{ids}) AND sku NOT IN (%{bad_skus})
+      #     ), binds: {
+      #       bad_skus: ["G90023ASDf0"]
+      #     }).
+      #     run
+      #
+      # @param name [Symbol] name of attribute to load records into
+      # @param mapping [Hash] a one element Hash with the key being the local/child id and the value being the foreign/parent id
+      # @param sql [String] the SQL to query the associated records. Include a bind params called '%{ids}' for the foreign/parent ids.
+      # @param use [Array<Module>] optional - Ruby modules to include in the result objects (single or array)
+      # @param binds [Hash] any additional binds for your query.
+      # @param model [ActiveRecord::Base] optional - ActiveRecord model that represents what you're loading. required when using Sqlite.
+      # @yield eager load associations nested under this one
+      #
+      def eager_load_many(name, mapping, sql, binds: {}, model: nil, use: nil, &builder)
+        @eager_loaders << EagerLoaders::AdHocMany.new(name, mapping, sql, binds: binds, model: model, use: use, &builder)
+        self
+      end
+    end
+  end
+end

data/lib/occams-record/eager_loaders/context.rb

@@ -0,0 +1,113 @@
+module OccamsRecord
+  module EagerLoaders
+    #
+    # A container for all eager loading on a particular Active Record model. Usually the context is initialized
+    # with the model, and all eager loaders are immediately initialized. Any errors (like a wrong association name
+    # ) will be thrown immediately and before any queries are run.
+    #
+    # However, in certain situations the model cannot be known until runtime (e.g. eager loading off of a
+    # polymorphic association). In these cases the model won't be set, or the eager loaders fully initialized,
+    # until the parent queries have run. This means that certain errors (like a wrong association name) won't be
+    # noticed until very late, after queries have started running.
+    #
+    class Context
+      # @return [ActiveRecord::Base]
+      attr_reader :model
+
+      #
+      # Initialize a new eager loading context.
+      #
+      # @param mode [ActiveRecord::Base] the model that contains the associations that will be referenced.
+      #
+      def initialize(model = nil)
+        @model = model
+        @loaders = []
+        @dynamic_loaders = []
+      end
+
+      #
+      # Set the model.
+      #
+      # @param model [ActiveRecord::Base]
+      #
+      def model=(model)
+        @model = model
+        @loaders = @loaders + @dynamic_loaders.map { |args|
+          build_loader(*args)
+        }
+        @dynamic_loaders = []
+      end
+
+      #
+      # Return the names of the associations being loaded.
+      #
+      # @return [Array<String>]
+      #
+      def names
+        @loaders.map(&:name) +
+          @loaders.select { |l| l.respond_to? :through_name }.map(&:through_name) # TODO make not hacky
+      end
+
+      #
+      # Append an already-initialized eager loader.
+      #
+      # @param loader [OccamsRecord::EagerLoaders::Base]
+      # @return [OccamsRecord::EagerLoaders::Base] the added loader
+      #
+      def <<(loader)
+        @loaders << loader
+        loader
+      end
+
+      #
+      # Specify an association to be eager-loaded. For maximum memory savings, only SELECT the
+      # colums you actually need.
+      #
+      # @param assoc [Symbol] name of association
+      # @param scope [Proc] a scope to apply to the query (optional). It will be passed an
+      # ActiveRecord::Relation on which you may call all the normal query hethods (select, where, etc) as well as any scopes you've defined on the model.
+      # @param select [String] a custom SELECT statement, minus the SELECT (optional)
+      # @param use [Array<Module>] optional Module to include in the result class (single or array)
+      # @param as [Symbol] Load the association usign a different attribute name
+      # @param optimizer [Symbol] Only used for `through` associations. Options are :none (load all intermediate records) | :select (load all intermediate records but only SELECT the necessary columns)
+      # @yield a block where you may perform eager loading on *this* association (optional)
+      # @return [OccamsRecord::EagerLoaders::Base] the new loader. if @model is nil, nil will be returned.
+      #
+      def add(assoc, scope = nil, select: nil, use: nil, as: nil, optimizer: :select, &builder)
+        if @model
+          loader = build_loader(assoc, scope, select, use, as, optimizer, builder)
+          @loaders << loader
+          loader
+        else
+          @dynamic_loaders << [assoc, scope, select, use, as, optimizer, builder]
+          nil
+        end
+      end
+
+      #
+      # Performs all eager loading in this context (and in any nested ones).
+      #
+      # @param rows [Array<ActiveRecord::Base>] the parent rows to load child rows into
+      # @param query_logger [Array] optional query logger
+      #
+      def run!(rows, query_logger: nil)
+        raise "Cannot run eager loaders when @model has not been set!" if @dynamic_loaders.any? and @model.nil?
+        @loaders.each { |loader|
+          loader.run(rows, query_logger: query_logger)
+        }
+        nil
+      end
+
+      private
+
+      def build_loader(assoc, scope, select, use, as, optimizer, builder)
+        ref = @model.reflections[assoc.to_s]
+        ref ||= @model.subclasses.map(&:reflections).detect { |x| x.has_key? assoc.to_s }&.[](assoc.to_s)
+        raise "OccamsRecord: No assocation `:#{assoc}` on `#{@model.name}` or subclasses" if ref.nil?
+        scope ||= ->(q) { q.select select } if select
+        loader_class = !!ref.through_reflection ? EagerLoaders::Through : EagerLoaders.fetch!(ref)
+        loader_class.new(ref, scope, use: use, as: as, optimizer: optimizer, &builder)
+      end
+    end
+  end
+end

data/lib/occams-record/eager_loaders/polymorphic_belongs_to.rb

@@ -2,6 +2,8 @@ module OccamsRecord
   module EagerLoaders
     # Eager loader for polymorphic belongs tos
     class PolymorphicBelongsTo
+      include EagerLoaders::Builder
+
       # @return [String] association name
       attr_reader :name
 
@@ -11,13 +13,16 @@ module OccamsRecord
       # ActiveRecord::Relation on which you may call all the normal query hethods (select, where, etc) as well as any scopes you've defined on the model.
       # @param use [Array<Module>] optional Module to include in the result class (single or array)
       # @param as [Symbol] Load the association usign a different attribute name
+      # @param optimizer [Symbol] Only used for `through` associations. A no op here.
       # @yield perform eager loading on *this* association (optional)
       #
-      def initialize(ref, scope = nil, use: nil, as: nil, &eval_block)
-        @ref, @scope, @use, @eval_block = ref, scope, use, eval_block
+      def initialize(ref, scope = nil, use: nil, as: nil, optimizer: nil, &builder)
+        @ref, @scope, @use = ref, scope, use
         @name = (as || ref.name).to_s
         @foreign_type = @ref.foreign_type.to_sym
         @foreign_key = @ref.foreign_key.to_sym
+        @eager_loaders = EagerLoaders::Context.new
+        instance_eval(&builder) if builder
       end
 
       #
@@ -28,7 +33,9 @@ module OccamsRecord
       #
       def run(rows, query_logger: nil)
         query(rows) { |scope|
-          assoc_rows = Query.new(scope, use: @use, query_logger: query_logger, &@eval_block).run
+          eager_loaders = @eager_loaders.dup
+          eager_loaders.model = scope.klass
+          assoc_rows = Query.new(scope, use: @use, eager_loaders: eager_loaders, query_logger: query_logger).run
           merge! assoc_rows, rows
         }
       end

data/lib/occams-record/eager_loaders/through.rb

@@ -0,0 +1,107 @@
+module OccamsRecord
+  module EagerLoaders
+    #
+    # Handles :through associations for has_many and has_one. Polymorphic associations are not supported.
+    #
+    class Through < Base
+      Link = Struct.new(:name, :macro, :ref, :next_ref)
+
+      #
+      # See documentation for OccamsRecord::EagerLoaders::Base.
+      #
+      def initialize(*args)
+        super
+
+        unless @ref.macro == :has_one or @ref.macro == :has_many
+          raise ArgumentError, "#{@ref.active_record.name}##{@ref.name} cannot be eager loaded because only `has_one` and `has_many` are supported for `through` associations"
+        end
+        if (polys = @ref.chain.select(&:polymorphic?)).any?
+          names = polys.map { |r| "#{r.active_record.name}##{r.name}" }
+          raise ArgumentError, "#{@ref.active_record.name}##{@ref.name} cannot be eager loaded because these `through` associations are polymorphic: #{names.join ', '}"
+        end
+        unless @optimizer == :none or @optimizer == :select
+          raise ArgumentError, "Unrecognized optimizer '#{@optimizer}'"
+        end
+
+        chain = @ref.chain.reverse
+        @chain = chain.each_with_index.map { |x, i|
+          Link.new(x.source_reflection.name, x.source_reflection.macro, x, chain[i + 1])
+        }
+        @loader = build_loader
+      end
+
+      # TODO make not hacky
+      def through_name
+        @loader.name
+      end
+
+      def run(rows, query_logger: nil)
+        results = @loader.run(rows, query_logger: query_logger)
+        attr_set = "#{name}="
+        results.each do |row|
+          row.send(attr_set, reduce(row))
+        end
+        results
+      end
+
+      private
+
+      def reduce(node, depth = 0)
+        link = @chain[depth]
+        case link&.macro
+        when nil
+          node
+        when :has_many
+          node.send(link.name).reduce(Set.new) { |a, child|
+            result = reduce(child, depth + 1)
+            case result
+            when Array then a + result
+            else a << result
+            end
+          }.to_a
+        when :has_one, :belongs_to
+          child = node.send(link.name)
+          reduce(child, depth + 1)
+        else
+          raise "Unsupported through chain link type '#{link.macro}'"
+        end
+      end
+
+      def build_loader
+        head = @chain[0]
+        links = @chain[1..-2]
+        tail = @chain[-1]
+
+        outer_loader = EagerLoaders.fetch!(head.ref).new(head.ref, optimized_select(head))
+
+        links.
+          reduce(outer_loader) { |loader, link|
+            loader.nest(link.ref.source_reflection.name, optimized_select(link))
+          }.
+          nest(tail.ref.source_reflection.name, @scope, use: @use, as: @as)
+
+        outer_loader
+      end
+
+      def optimized_select(link)
+        return nil unless @optimizer == :select
+
+        cols = case link.macro
+               when :belongs_to
+                 [link.ref.association_primary_key]
+               when :has_one, :has_many
+                 [link.ref.association_primary_key, link.ref.foreign_key]
+               else
+                 raise "Unsupported through chain link type '#{link.macro}'"
+               end
+
+        case link.next_ref.source_reflection.macro
+        when :belongs_to
+          cols << link.next_ref.foreign_key
+        end
+
+        ->(q) { q.select(cols.join(", ")) }
+      end
+    end
+  end
+end

data/lib/occams-record/query.rb

@@ -46,16 +46,14 @@ module OccamsRecord
     # @param scope [ActiveRecord::Relation]
     # @param use [Array<Module>] optional Module to include in the result class (single or array)
     # @param query_logger [Array] (optional) an array into which all queries will be inserted for logging/debug purposes
-    # @param eager_loaders [OccamsRecord::EagerLoaders::Base]
-    # @yield will be eval'd on this instance. Can be used for eager loading. (optional)
+    # @param eager_loaders [OccamsRecord::EagerLoaders::Context]
     #
-    def initialize(scope, use: nil, query_logger: nil, eager_loaders: [], &eval_block)
+    def initialize(scope, use: nil, eager_loaders: nil, query_logger: nil)
       @model = scope.klass
       @scope = scope
-      @eager_loaders = eager_loaders
+      @eager_loaders = eager_loaders || EagerLoaders::Context.new(@model)
       @use = use
       @query_logger = query_logger
-      instance_eval(&eval_block) if eval_block
     end
 
     #
@@ -82,9 +80,9 @@ module OccamsRecord
       sql = block_given? ? yield(scope).to_sql : scope.to_sql
       @query_logger << sql if @query_logger
       result = model.connection.exec_query sql
-      row_class = OccamsRecord::Results.klass(result.columns, result.column_types, @eager_loaders.map(&:name), model: model, modules: @use)
+      row_class = OccamsRecord::Results.klass(result.columns, result.column_types, @eager_loaders.names, model: model, modules: @use)
       rows = result.rows.map { |row| row_class.new row }
-      eager_load! rows
+      @eager_loaders.run!(rows, query_logger: @query_logger)
       rows
     end
 

data/lib/occams-record/raw_query.rb

@@ -60,18 +60,16 @@ module OccamsRecord
     # @param sql [String] The SELECT statement to run. Binds should use Ruby's named string substitution.
     # @param binds [Hash] Bind values (Symbol keys)
     # @param use [Array<Module>] optional Module to include in the result class (single or array)
-    # @param eager_loaders [OccamsRecord::EagerLoaders::Base]
+    # @param eager_loaders [OccamsRecord::EagerLoaders::Context]
     # @param query_logger [Array] (optional) an array into which all queries will be inserted for logging/debug purposes
     #
-    def initialize(sql, binds, use: nil, eager_loaders: [], query_logger: nil, &eval_block)
+    def initialize(sql, binds, use: nil, eager_loaders: nil, query_logger: nil)
       @sql = sql
       @binds = binds
       @use = use
-      @eager_loaders = eager_loaders
+      @eager_loaders = eager_loaders || EagerLoaders::Context.new
       @query_logger = query_logger
-      @model = nil
-      @conn = @model&.connection || ActiveRecord::Base.connection
-      instance_eval(&eval_block) if eval_block
+      @conn = @eager_loaders.model&.connection || ActiveRecord::Base.connection
     end
 
     #
@@ -85,7 +83,7 @@ module OccamsRecord
     # @return [OccamsRecord::RawQuery] self
     #
     def model(klass)
-      @model = klass
+      @eager_loaders.model = klass
       self
     end
 
@@ -98,9 +96,9 @@ module OccamsRecord
       _escaped_sql = escaped_sql
       @query_logger << _escaped_sql if @query_logger
       result = @conn.exec_query _escaped_sql
-      row_class = OccamsRecord::Results.klass(result.columns, result.column_types, @eager_loaders.map(&:name), model: @model, modules: @use)
+      row_class = OccamsRecord::Results.klass(result.columns, result.column_types, @eager_loaders.names, model: @eager_loaders.model, modules: @use)
       rows = result.rows.map { |row| row_class.new row }
-      eager_load! rows
+      @eager_loaders.run!(rows, query_logger: @query_logger)
       rows
     end
 
@@ -154,7 +152,7 @@ module OccamsRecord
           results = RawQuery.new(@sql, @binds.merge({
             batch_limit: of,
             batch_offset: offset,
-          }), use: @use, query_logger: @query_logger, eager_loaders: @eager_loaders).model(@model).run
+          }), use: @use, query_logger: @query_logger, eager_loaders: @eager_loaders).run
 
           y.yield results if results.any?
           break if results.size < of

data/lib/occams-record/version.rb

@@ -3,5 +3,5 @@
 #
 module OccamsRecord
   # Library version
-  VERSION = '0.29.0'.freeze
+  VERSION = '0.31.0'.freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: occams-record
 version: !ruby/object:Gem::Version
-  version: 0.29.0
+  version: 0.31.0
 platform: ruby
 authors:
 - Jordan Hollinger
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-05-02 00:00:00.000000000 Z
+date: 2018-08-07 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -46,10 +46,13 @@ files:
 - lib/occams-record/eager_loaders/ad_hoc_one.rb
 - lib/occams-record/eager_loaders/base.rb
 - lib/occams-record/eager_loaders/belongs_to.rb
+- lib/occams-record/eager_loaders/builder.rb
+- lib/occams-record/eager_loaders/context.rb
 - lib/occams-record/eager_loaders/habtm.rb
 - lib/occams-record/eager_loaders/has_many.rb
 - lib/occams-record/eager_loaders/has_one.rb
 - lib/occams-record/eager_loaders/polymorphic_belongs_to.rb
+- lib/occams-record/eager_loaders/through.rb
 - lib/occams-record/errors.rb
 - lib/occams-record/merge.rb
 - lib/occams-record/query.rb
@@ -76,7 +79,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.7.3
+rubygems_version: 2.7.6
 signing_key: 
 specification_version: 4
 summary: The missing high-efficiency query API for ActiveRecord