autobots 0.0.1 → 0.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 2f3045074e374f37719b3922bfea5023f73fa5c0
-  data.tar.gz: 19c06ac4b4e923aee25392adcc4bfb5577f4b183
+  metadata.gz: cced0fc73b998d0e2c7ec8d62db5006e4e3191fd
+  data.tar.gz: df8529087829518b6982070e58c443c779588fc9
 SHA512:
-  metadata.gz: 290662e0a21676241bca53b76cc4c966f9733d095291a5c167c78ff4f2b8e7b483a8d1908ad93d352571fba8ac6fbe2569733e1938330be00bd3f26b477ffb8d
-  data.tar.gz: 13c683363bd0f4230e447440e38168c9dd0497e0e15a68d725c6e4e907c22bdc16352159b09113a38c0b0626306e9c134723b9a5e5e6555dfa399a05b4a5ae5f
+  metadata.gz: 253704d48566fe2a4be8c11d2c3274b84045e739592baef54d80982539b73096fa270f1706918efa77f4c755ff95a6e855ba9ef884a5950c240d0a511041c92a
+  data.tar.gz: 91cf503c5dc1136547588320aba92696320476b252a48796e3ff85bb7f603f67695839650a60576ab442eff784695e772eb688c069043dfcd3c1f74f9c5b1757

data/README.md

@@ -4,134 +4,135 @@ Loading and serializing models in bulk with caching.
 
 Separate the loading/serialization of resources from their protocol (http/json, avro, xml).
 
-## Motivation
-
 We want to improve api response time by loading the minimum amount of data needed to check cache keys. If we miss, we want to optimally load our data, serialize it, cache it and then return it.
 
-## Installation
-
-Add this line to your application's Gemfile:
+## Problem
 
-```ruby
-gem 'autobots'
-```
+Say we have a simple api action:
 
-And then execute:
+	def index
+	  # find our records
+	  projects = Project.includes(issues: :comments).first(10)
+	  render json: projects, each_serializer: ProjectSerializer
+	end
 
-    $ bundle
+How do we cache this?
 
-Or install it yourself as:
+    class ProjectSerializer < ActiveModel::Serializer
+      cached
+    end
 
-    $ gem install autobots
+    def index
+      projects = Project.includes(issues: :comments).first(10)
+      render json: projects, each_serializer: ProjectSerializer
+    end
 
-## Usage
-
-When trying to render a resource for an API, we have 3 parts:
-
-1. Fetching data
-2. Figuring out the data to return
-3. Format it for the protocol
+There are 2 problems with this approach:
 
-### Fetching data
+1. We're making 3 sql calls every time regardless of what keys are required for checking the cache
+2. We're making 10 cache fetch requests
 
-`autobots` uses `Autobots::Assembler` classes to fetch data from given identifiers.  The assembler's job is to fetch all data needed for serialization as optimally as possible. The identifiers should be the minimum amount of data needed to fetch data and determine caching.
+We can fix this by fetching the bare minimum amount of data needed to check the cache, checking the cache in bulk, then loading the data needed in an optimized fashion for cache misses only.
 
-```ruby
-
-class ProjectAssembler < Autobots::Assembler
-
-  # assembles the base objects needed for cache key generation
-  def assemble(identifiers)
-    Project.where(id: identifiers).to_a
-  end
-
-end
+This gem's goal is to abstract all that logic into a testable, declarative model that can improve your api's response time. We also want to return serializable data so that we decouple the definition of a resource from its protocols (not locked to json or xml)
 
+## Usage
 
-project_ids = [1,2,3]
-assembler = ProjectAssembler.new(project_ids)
+An `Autobots::Assembler` is the core of the `autobots` gem. The input for an autobot is that you provide an array of objects needed to build your cache keys. The output is an array of serializable data that corresponds to the input set (retaining order).
 
-# returns preloaded objects for serialization
-resources = assembler.resources
+![flow](docs/flow.png)
 
-```
+### Lifecycle of an Autobot
 
-### Figuring out the data to return
+When trying to render a resource for an API, we have 3 parts:
 
-We use the `active_model_serializers` gem to accomplish serialization. An assembler declares the type of serializer used to specify the data returned
+1. Fetching data
+2. Figuring out the data to return
+3. Format it for the protocol
 
-```ruby
+Each of these methods corresponds to a single lifecycle method of an autobot:
 
-class ProjectAssembler < Autobots::Assembler
-  self.serializer = ProjectSerializer # an ActiveModel::Serializer
-end
+1. `assemble`
+2. `transform`
+3. `roll_out`
 
-```
 
-### Formatting the response
+#### Fetching data (assemble)
 
-`autobots` gem only handles the loading and representation of the data.
+An `assembler` can optionally load whatever data is required to build it's cache keys. The assembler's job is to fetch all data needed for serialization as optimally as possible. The identifiers should be the minimum amount of data needed to fetch data and determine caching.
 
-### Caching
+	class ProjectAssembler < Autobots::Assembler
+	
+	  # assembles the base objects needed for cache key generation
+	  def assemble(identifiers)
+	    Project.where(id: identifiers).to_a
+	  end
+	
+	end
 
-We can get large performance boosts by caching our serializable data. Caching is straight forward:
+	project_ids = [1,2,3]
+	assembler = ProjectAssembler.new(project_ids)
+	
+	# returns preloaded objects for serialization
+	objects = assembler.objects
 
-```ruby
+#### Figuring out the data to return (transform)
 
-# some sort of ActiveSupport::CacheStore
-cache = Rails.cache
-assembler = ProjectAssembler.new(project_ids, cache: cache)
+The transform lifecycle event is called with every object that needs refreshing. It allows us to to optimize our loading of nested resources by using bulk loading. If caching is enabled, we only need to run this on resources that missed the cache. We may not even need to run it at all!
 
-```
+An example of this would be using `ActiveRecord::Associations::Preloader` to fetch included records if necessary:
 
-The cache store must implement `read_multi` as we use [`bulk_cache_fetcher` gem](https://github.com/justinweiss/bulk_cache_fetcher/)
+	class ProjectAssembler < Autobots::Assembler
+	  def transform(resources)
+	  	ActiveRecord::Associations::Preloader.new().preload(resources, {issues: :comments})
+	  end
+	end
 
-By default, we use each resource's `cache_key` implementation. You can provide your own cache key generator by passing in a cache_key proc:
+Since this is a fairly common pattern, I created the `Autobots::ActiveRecordAssembler` that has this default behavior:
 
-```ruby
+	class ProjectAssembler < Autobots::ActiveRecordAssembler
+	  self.preloads = {issues: :comments}
+	end
 
-assembler = ProjectAssembler.new(project_ids, {
-  cache: cache,
-  cache_key: -> (obj, assembler) {
-    "#{obj.cache_key}-foo"
-  }
-})
+The strength of this model lies in minimizing data fetch requests when building our data to serialize. We do this by delaying the loading of nested models and only load them if we have to.
 
-```
+If a resources's cache is up to date, we shouldn't have to fetch it's dependencies from the database. However, if we have a cache miss, we want to optimally load the data needed.
 
-### Minimizing fetch requests
+#### Formatting the response (roll_out)
 
-The strength of this model lies in minimizing data fetch requests when building our data to serialize. We do this by delaying the loading of nested models and only load them if we have to.
+We use the `active_model_serializers` gem to accomplish serialization. An assembler declares the type of serializer used to specify the data returned
 
-If a resources's cache is up to date, we shouldn't have to fetch it's dependencies from the database. However, if we have a cache miss, we want to optimally load the data needed.
 
-```ruby
+	class ProjectAssembler < Autobots::Assembler
+	  self.serializer = ProjectSerializer # an ActiveModel::Serializer
+	end
+	
+	# returns an array of data for serialization
+	data = assembler.data
 
-class ProjectAssembler < Autobots::Assembler
+The default behavior of an assembler is to wrap each object with the declared serializer and returning its serializable_hash.
 
-  # for any missing resources, preload the nested attributes
-  def transform(resources)
-    ActiveRecord::Associations::Preloader.new.preload(resources, {issues: :comments})
-  end
+`autobots` gem only handles the loading and representation of the data.
 
-end
+### Caching
 
-```
+We can get large performance boosts by caching our serializable data. Caching is straight forward:
 
-The preloading step can be customized (load from an external service or whatever you want). We provide a helpful mixin if you're using ActiveRecord and you want preloading:
+	# some sort of ActiveSupport::CacheStore
+	cache = Rails.cache
+	assembler = ProjectAssembler.new(project_ids, cache: cache)
 
-```ruby
+The cache store must implement `read_multi` as we use [`bulk_cache_fetcher` gem](https://github.com/justinweiss/bulk_cache_fetcher/)
 
-class ProjectAssembler < Autobots::Assembler
-  include Autobots::Helpers::ActiveRecordPreloading
+By default, we use each resource's `cache_key` implementation. You can provide your own cache key generator by passing in a cache_key proc:
 
-  # in this case, project has_many issues which has_many comments
-  def preloads
-    {issues: :comments}
-  end
-end
+	assembler = ProjectAssembler.new(project_ids, {
+	  cache: cache,
+	  cache_key: -> (obj, assembler) {
+	    "#{obj.cache_key}-foo"
+	  }
+	})
 
-```
 
 ## Contributing
 

data/lib/autobots.rb

@@ -4,6 +4,7 @@ require 'active_model_serializers'
 require 'bulk_cache_fetcher'
 
 module Autobots
+  autoload :ActiveRecordAssembler, 'autobots/active_record_assembler'
   autoload :Assembler, 'autobots/assembler'
   autoload :Helpers, 'autobots/helpers'
 end

data/lib/autobots/active_record_assembler.rb

@@ -0,0 +1,5 @@
+module Autobots
+  class ActiveRecordAssembler < Assembler
+    include Helpers::ActiveRecordPreloading
+  end
+end

data/lib/autobots/helpers/active_record_preloading.rb

@@ -1,6 +1,12 @@
 module Autobots
   module Helpers
     module ActiveRecordPreloading
+      extend ActiveSupport::Concern
+
+      included do
+        class_attribute :preloads
+        self.preloads = []
+      end
 
       def transform(objects)
         ActiveRecord::Associations::Preloader.new(objects, preloads).run
@@ -10,9 +16,6 @@ module Autobots
         objects
       end
 
-      def preloads
-        []
-      end
     end
   end
 end

data/lib/autobots/version.rb

@@ -1,3 +1,3 @@
 module Autobots
-  VERSION = "0.0.1"
+  VERSION = "0.1.0"
 end

data/test/test_models.rb

@@ -31,10 +31,8 @@ class ProjectPreloadAssembler < Autobots::Assembler
   end
 end
 
-class ProjectPreloadIncludedAssembler < Autobots::Assembler
+class ProjectPreloadIncludedAssembler < Autobots::ActiveRecordAssembler
   self.serializer = ProjectSerializer
-  include Autobots::Helpers::ActiveRecordPreloading
-
   def preloads
     {issues: :comments}
   end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: autobots
 version: !ruby/object:Gem::Version
-  version: 0.0.1
+  version: 0.1.0
 platform: ruby
 authors:
 - Jeff Ching
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-11-21 00:00:00.000000000 Z
+date: 2014-11-24 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bulk_cache_fetcher
@@ -108,7 +108,9 @@ files:
 - README.md
 - Rakefile
 - autobots.gemspec
+- docs/flow.png
 - lib/autobots.rb
+- lib/autobots/active_record_assembler.rb
 - lib/autobots/assembler.rb
 - lib/autobots/helpers.rb
 - lib/autobots/helpers/active_record_preloading.rb