batch-loader-active-record 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 7fc87d482f6405949526e737f312943dd267ed43
-  data.tar.gz: 4364f28a2e7899eff06c363ca02e0eef1c3d34db
+  metadata.gz: c7c24c7d661f0d2449d676892c1d328f8c5147e3
+  data.tar.gz: 83d9fa13f7dcab069ed2b1d1f11fc4e1f36834f9
 SHA512:
-  metadata.gz: b5303b39f9f88e0b95128b0d6a506944eb5f9bba3683801df644bfde40bc197e47a6f709b470a228ac4f9819cb7601dd3563cde2183afdbfd7402024fec870f2
-  data.tar.gz: db6035acfdb9d1fd2beff9a9aa9e616bd5107eeeac514b12588ebbaa6480b93215e9ed51260995c1fe685d2fffe8cdf9082fd89fe3eff5109c70096e856845e9
+  metadata.gz: 0677e7a74dfe10a238bd431d171d44bbff07b3c8a6f775bfbe9bf595f345ea0fce7c35a494eceb7702e5abb56ffcada2ea426c23097286a77bfb200a0022ebb3
+  data.tar.gz: 0e0e3edd2866a7c578567f5171e23fae3356b8469ada0123781d5d7b0276c655af33149baa1da985e8c4d198b8ddf3ad48f961b38609b03ed959ea7dfaeff58b

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    batch-loader-active-record (0.1.0)
+    batch-loader-active-record (0.2.0)
       activerecord (>= 4.2.0, < 5.2.0)
       activesupport (>= 4.2.0, < 5.2.0)
       batch-loader (~> 1.2.0)

data/README.md

@@ -1,9 +1,48 @@
 # Batch Loader - Active Record #
 
 [![Build Status](https://travis-ci.org/mathieul/batch-loader-active-record.svg?branch=master)](https://travis-ci.org/mathieul/batch-loader-active-record)
+[![Gem Version](https://badge.fury.io/rb/batch-loader-active-record.svg)](https://badge.fury.io/rb/batch-loader-active-record)
 
 This gem allows to leverage the awesome [batch-loader gem](https://github.com/exAspArk/batch-loader) to generate lazy Active Record relationships without any boilerplate.
 
+It is not intended to be used for all associations though, but only where necessary. It should be used as a complement to vanilla batch loaders written directly using [batch-loader gem](https://github.com/exAspArk/batch-loader).
+
+**This gem is in active deployment and is likely not yet ready to be used on production.**
+
+
+## Description
+
+This is a very simple gem which is basically a mixin containg replacement macros for the 3 active record association macros (note that **polymorphic associations and association scopes are not yet supported**):
+
+* `belongs_to_lazy`
+* `has_one_lazy`
+* `has_many_lazy`
+
+Those are intended to use in replacement of the original Active Record class methods when you also want to generate a method to avoid N+1 calls to the database when accessed for the elements of a collection. For more details on why N+1 queries are common when fetching associations read [the batch-loader gem README](https://github.com/exAspArk/batch-loader/#why).
+
+For example let's imagine a post which can have many comments:
+
+```ruby
+class Post < ActiveRecord::Base
+  has_many_lazy :comments
+end
+
+class Comment < ActiveRecord::Base
+  belongs_to :post
+end
+```
+
+Now we get a list of post objects and we want to fetch all the comments for each post. When we know in advance that we'll need the post comments, then Active Record query [#includes](http://api.rubyonrails.org/classes/ActiveRecord/QueryMethods.html#method-i-includes) will trigger a single query to fetch posts and comments.
+
+But often we don't know in advance in the code responsible to fetch the posts if we'll need access to the comments as well. When implenting a GraphQL API for instance, the post resolver doesn't know if the comments are also part of the GraphQL query.Using `#includes` in this case would be wasteful and slower for the cases when we don't need the comments.
+
+When using the lazy association accessor (i.e.: `post.comments_lazy`), a Batch Loader object is returned instead of a model relation and the query with the post id is buffered temporarily in the thread hash. No query to the database is executed yet. Calling the same association accessor on another post instance will add this post id to the list in the tread context. And so on until we access one of those Batch Loader objects returned. Only then is the database query executed and all Batch Loader objects are replaced by the records just fetched (not really replaced, they use delegation under the cover).
+
+It is important to note that Active Record association accessors return relations which can be chained using the Active Record query API. But the lazy association accessors generated by `batch-loader-active-record` return (for all intents and purposes) an active record instance or an array of active record instances which can't be chained.
+
+To benefit from the query batching we must first collect the lazy associations for each model instance in our collection, and only then we can start using them to access their content. Accessing a lazy object too early triggers the database query too early. For instance using `#flat_map` to collect and use the lazy objects would fail as `#flat_map` does access each element of the collection immediately in order to flatten the result.
+
+
 ## Installation
 
 Add this line to your application's Gemfile:
@@ -20,15 +59,135 @@ Or install it yourself as:
 
     $ gem install batch-loader-active-record
 
+
 ## Usage
 
-This is a very simple gem which just contains a mixin to include and give access to class methods for each association kind:
+Include the `BatchLoaderActiveRecord` module at the beginning of the model classes where lazy associations are needed, and use one of the lazy class macros to declare all lazy associations.
 
-* `belongs_to_lazy`
-* `has_one_lazy`
-* `has_many_lazy`
+### Belongs To ###
+
+Consider the following data model:
+
+```ruby
+class Post < ActiveRecord::Base
+  has_many :comments
+end
+
+class Comment < ActiveRecord::Base
+  include BatchLoaderActiveRecord
+  belongs_to_lazy :post
+end
+```
+
+We need to know the `post` owning each instance of `comments`:
+
+```ruby
+posts = comments.map(&:post_lazy)
+# no DB query executed yet
+posts.map(&:author_first_name)
+# DB query was executed
+# => ["Jane", "Anne", ...]
+```
+
+### Has One ###
+
+Consider the following data model:
+
+```ruby
+class Account < ActiveRecord::Base
+  include BatchLoaderActiveRecord
+  has_one_lazy :affiliate
+end
+
+class Affiliate < ActiveRecord::Base
+  belongs_to :account
+end
+```
+
+Fetching all affiliates for the accounts who do have one affiliate:
+
+```ruby
+affiliates = accounts.map(&:affiliate_lazy)
+# no DB query executed yet
+affiliates.first.name
+# DB query was executed
+affiliates.compact
+# => [#<Affiliate id: 123>, #<Affiliate id: 456>]
+```
+
+### Has Many ###
+
+Consider the following data model:
+
+```ruby
+class Contact < ActiveRecord::Base
+  include BatchLoaderActiveRecord
+  has_many_lazy :phone_numbers
+end
+
+class PhoneNumber < ActiveRecord::Base
+  belongs_to :contact
+  scope :active, -> { where(enabled: true) }
+end
+```
+
+This time we want the list of phone numbers for a collection of contacts.
+
+```ruby
+contacts.map(&:phone_numbers_lazy).flatten
+```
+
+It is also possible to apply scopes and conditions to a lazy has_many association. For instance if we want to only fetch active phone numbers in the example above, you would specify the scope like so:
+
+```ruby
+contacts.map { |contact| contact.phone_numbers_lazy(PhoneNumber.active) }.flatten
+```
+
+
+### Has Many :through ###
+
+Consider the following data model with a has-many association going through another has-many-through association. Agents can have many phones they use to call providers:
+
+```ruby
+class Agent < ActiveRecord::Base
+  include BatchLoaderActiveRecord
+  has_many :phones
+  has_many_lazy :providers, through: :phones
+end
+
+class Phone < ActiveRecord::Base
+  belongs_to :agent
+  has_many :calls
+  has_many :providers, through: :calls
+end
+
+class Call < ActiveRecord::Base
+  belongs_to :provider
+  belongs_to :phone
+end
+
+class Provider < ActiveRecord::Base
+  has_many :calls
+end
+```
+
+We want to fetch the list of providers who were called by a list of agents:
+
+```ruby
+agents.map(&:providers_lazy).uniq
+```
+
+This would trigger this query for the collection of agents with ids 4212, 265 and 2309:
+
+```sql
+SELECT providers.*, agents.ID AS _instance_id
+FROM providers
+INNER JOIN calls ON calls.provider_id = providers.ID
+INNER JOIN phones ON phones.ID = calls.phone_id
+INNER JOIN agents ON agents.ID = phones.agent_id
+WHERE (agents. ID IN(4212, 265, 2309))
+```
 
-You use those generators in replacement of the original Active Record association class methods.
 
 ## Development
 

data/lib/batch_loader_active_record.rb

@@ -11,12 +11,12 @@ module BatchLoaderActiveRecord
   module ClassMethods
     def belongs_to_lazy(*args)
       belongs_to(*args).tap do |reflections|
-        assoc = reflections.values.last
-        batch_key = [table_name, assoc.name]
-        define_method(:"#{assoc.name}_lazy") do
-          foreign_key_value = send(assoc.foreign_key) or return nil
+        reflect = reflections.values.last
+        batch_key = [table_name, reflect.name]
+        define_method(:"#{reflect.name}_lazy") do
+          foreign_key_value = send(reflect.foreign_key) or return nil
           BatchLoader.for(foreign_key_value).batch(key: batch_key) do |foreign_key_values, loader|
-            assoc.klass.where(id: foreign_key_values).each { |instance| loader.call(instance.id, instance) }
+            reflect.klass.where(id: foreign_key_values).each { |instance| loader.call(instance.id, instance) }
           end
         end
       end
@@ -24,12 +24,12 @@ module BatchLoaderActiveRecord
 
     def has_one_lazy(*args)
       has_one(*args).tap do |reflections|
-        assoc = reflections.values.last
-        batch_key = [table_name, assoc.name]
-        define_method(:"#{assoc.name}_lazy") do
+        reflect = reflections.values.last
+        batch_key = [table_name, reflect.name]
+        define_method(:"#{reflect.name}_lazy") do
           BatchLoader.for(id).batch(key: batch_key) do |model_ids, loader|
-            assoc.klass.where(assoc.foreign_key => model_ids).each do |instance|
-              loader.call(instance.public_send(assoc.foreign_key), instance)
+            reflect.klass.where(reflect.foreign_key => model_ids).each do |instance|
+              loader.call(instance.public_send(reflect.foreign_key), instance)
             end
           end
         end
@@ -38,16 +38,73 @@ module BatchLoaderActiveRecord
 
     def has_many_lazy(*args)
       has_many(*args).tap do |reflections|
-        assoc = reflections.values.last
-        batch_key = [table_name, assoc.name]
-        define_method(:"#{assoc.name}_lazy") do
+        reflect = reflections.values.last
+        base_key = [table_name, reflect.name]
+        define_method(:"#{reflect.name}_lazy") do |instance_scope = nil|
+          batch_key = base_key
+          batch_key += [instance_scope.to_sql.hash] unless instance_scope.nil?
           BatchLoader.for(id).batch(default_value: [], key: batch_key) do |model_ids, loader|
-            assoc.klass.where(assoc.foreign_key => model_ids).each do |instance|
-              loader.call(instance.public_send(assoc.foreign_key)) { |value| value << instance }
+            relation = instance_scope || reflect.klass
+            if reflect.through_reflection?
+              instances = self.class.fetch_for_model_ids(model_ids, relation: relation, reflection: reflect)
+              instances.each do |instance|
+                loader.call(instance.public_send(:_instance_id)) { |value| value << instance }
+              end
+            else
+              relation.where(reflect.foreign_key => model_ids).each do |instance|
+                loader.call(instance.public_send(reflect.foreign_key)) { |value| value << instance }
+              end
             end
           end
         end
       end
     end
+
+    def fetch_for_model_ids(ids, relation:, reflection:)
+      instance_id_path = "#{reflection.active_record.table_name}.#{reflection.active_record.primary_key}"
+      model_class = reflection.active_record
+      reflections = reflection_chain(reflection)
+      join_strings = [reflection_join(reflections.first, relation)]
+      reflections.each_cons(2) do |previous, current|
+        join_strings << reflection_join(current, previous.active_record)
+      end
+      select_relation = join_strings.reduce(relation) do |select_relation, join_string|
+        select_relation.joins(join_string)
+      end
+      select_relation
+        .where("#{model_class.table_name}.#{model_class.primary_key} IN (?)", ids)
+        .select("#{relation.table_name}.*, #{instance_id_path} AS _instance_id")
+    end
+
+    def reflection_chain(reflection)
+      reflections = [reflection]
+      begin
+        previous   = reflection
+        reflection = previous.source_reflection
+        if reflection && reflection != previous
+          reflections << reflection
+        else
+          reflection = nil
+        end
+      end while reflection
+      reflections.reverse
+    end
+
+    def reflection_join(orig_reflection, model_class)
+      reflection = orig_reflection.through_reflection? ? orig_reflection.through_reflection : orig_reflection
+      id_path = id_path_for(reflection, model_class)
+      table_name = reflection.active_record.table_name
+      id_column = reflection.belongs_to? ? reflection.foreign_key : reflection.active_record.primary_key
+      "INNER JOIN #{table_name} ON #{table_name}.#{id_column} = #{id_path}"
+    end
+
+    def id_path_for(reflection, model_class)
+      id_column = if reflection.belongs_to?
+        model_class.primary_key
+      else
+        reflection.foreign_key
+      end
+      "#{model_class.table_name}.#{id_column}"
+    end
   end
 end

data/lib/batch_loader_active_record/version.rb

@@ -1,3 +1,3 @@
 module BatchLoaderActiveRecord
-  VERSION = "0.1.0"
+  VERSION = "0.2.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: batch-loader-active-record
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - mathieul
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-11-19 00:00:00.000000000 Z
+date: 2017-11-23 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: batch-loader