autobots 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 2f3045074e374f37719b3922bfea5023f73fa5c0
+  data.tar.gz: 19c06ac4b4e923aee25392adcc4bfb5577f4b183
+SHA512:
+  metadata.gz: 290662e0a21676241bca53b76cc4c966f9733d095291a5c167c78ff4f2b8e7b483a8d1908ad93d352571fba8ac6fbe2569733e1938330be00bd3f26b477ffb8d
+  data.tar.gz: 13c683363bd0f4230e447440e38168c9dd0497e0e15a68d725c6e4e907c22bdc16352159b09113a38c0b0626306e9c134723b9a5e5e6555dfa399a05b4a5ae5f

data/.gitignore

@@ -0,0 +1,15 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/
+*.bundle
+*.so
+*.o
+*.a
+*.log
+mkmf.log

data/.travis.yml

@@ -0,0 +1,8 @@
+language: ruby
+rvm:
+  - 2.0.0
+  - 2.1.0
+env:
+  matrix:
+    - "RAILS_VERSION=4.0.0"
+    - "RAILS_VERSION=4.1.0"

data/Gemfile

@@ -0,0 +1,15 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in autobots.gemspec
+gemspec
+
+rails_version = ENV["RAILS_VERSION"] || "default"
+
+rails = case rails_version
+when "default"
+  ">= 3.2.0"
+else
+  "~> #{rails_version}"
+end
+
+gem "rails", rails

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2014 Jeff Ching
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,142 @@
+# Autobots [![Build Status](https://travis-ci.org/chingor13/autobots.png)](https://travis-ci.org/chingor13/autobots)
+
+Loading and serializing models in bulk with caching.
+
+Separate the loading/serialization of resources from their protocol (http/json, avro, xml).
+
+## Motivation
+
+We want to improve api response time by loading the minimum amount of data needed to check cache keys. If we miss, we want to optimally load our data, serialize it, cache it and then return it.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'autobots'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install autobots
+
+## Usage
+
+When trying to render a resource for an API, we have 3 parts:
+
+1. Fetching data
+2. Figuring out the data to return
+3. Format it for the protocol
+
+### Fetching data
+
+`autobots` uses `Autobots::Assembler` classes to fetch data from given identifiers.  The assembler's job is to fetch all data needed for serialization as optimally as possible. The identifiers should be the minimum amount of data needed to fetch data and determine caching.
+
+```ruby
+
+class ProjectAssembler < Autobots::Assembler
+
+  # assembles the base objects needed for cache key generation
+  def assemble(identifiers)
+    Project.where(id: identifiers).to_a
+  end
+
+end
+
+
+project_ids = [1,2,3]
+assembler = ProjectAssembler.new(project_ids)
+
+# returns preloaded objects for serialization
+resources = assembler.resources
+
+```
+
+### Figuring out the data to return
+
+We use the `active_model_serializers` gem to accomplish serialization. An assembler declares the type of serializer used to specify the data returned
+
+```ruby
+
+class ProjectAssembler < Autobots::Assembler
+  self.serializer = ProjectSerializer # an ActiveModel::Serializer
+end
+
+```
+
+### Formatting the response
+
+`autobots` gem only handles the loading and representation of the data.
+
+### Caching
+
+We can get large performance boosts by caching our serializable data. Caching is straight forward:
+
+```ruby
+
+# some sort of ActiveSupport::CacheStore
+cache = Rails.cache
+assembler = ProjectAssembler.new(project_ids, cache: cache)
+
+```
+
+The cache store must implement `read_multi` as we use [`bulk_cache_fetcher` gem](https://github.com/justinweiss/bulk_cache_fetcher/)
+
+By default, we use each resource's `cache_key` implementation. You can provide your own cache key generator by passing in a cache_key proc:
+
+```ruby
+
+assembler = ProjectAssembler.new(project_ids, {
+  cache: cache,
+  cache_key: -> (obj, assembler) {
+    "#{obj.cache_key}-foo"
+  }
+})
+
+```
+
+### Minimizing fetch requests
+
+The strength of this model lies in minimizing data fetch requests when building our data to serialize. We do this by delaying the loading of nested models and only load them if we have to.
+
+If a resources's cache is up to date, we shouldn't have to fetch it's dependencies from the database. However, if we have a cache miss, we want to optimally load the data needed.
+
+```ruby
+
+class ProjectAssembler < Autobots::Assembler
+
+  # for any missing resources, preload the nested attributes
+  def transform(resources)
+    ActiveRecord::Associations::Preloader.new.preload(resources, {issues: :comments})
+  end
+
+end
+
+```
+
+The preloading step can be customized (load from an external service or whatever you want). We provide a helpful mixin if you're using ActiveRecord and you want preloading:
+
+```ruby
+
+class ProjectAssembler < Autobots::Assembler
+  include Autobots::Helpers::ActiveRecordPreloading
+
+  # in this case, project has_many issues which has_many comments
+  def preloads
+    {issues: :comments}
+  end
+end
+
+```
+
+## Contributing
+
+1. Fork it ( https://github.com/chingor13/autobots/fork )
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create a new Pull Request

data/Rakefile

@@ -0,0 +1,13 @@
+require "bundler/gem_tasks"
+
+require 'rake/testtask'
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'lib'
+  t.libs << 'test'
+  t.pattern = 'test/**/*_test.rb'
+  t.verbose = false
+end
+
+
+task default: :test

data/autobots.gemspec

@@ -0,0 +1,28 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'autobots/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "autobots"
+  spec.version       = Autobots::VERSION
+  spec.authors       = ["Jeff Ching"]
+  spec.email         = ["jching@avvo.com"]
+  spec.summary       = "Loading and serializing models"
+  spec.description   = "Loading and serializing models in bulk with caching"
+  spec.homepage      = ""
+  spec.license       = "MIT"
+
+  spec.files         = `git ls-files -z`.split("\x0")
+  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
+  spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency "bulk_cache_fetcher", '~> 0.0.3'
+  spec.add_dependency "activesupport", '~> 4.0'
+  spec.add_dependency "active_model_serializers", '~> 0.8.0'
+
+  spec.add_development_dependency "bundler", "~> 1.6"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "sqlite3"
+end

data/lib/autobots.rb

@@ -0,0 +1,9 @@
+require 'autobots/version'
+require 'active_support/all'
+require 'active_model_serializers'
+require 'bulk_cache_fetcher'
+
+module Autobots
+  autoload :Assembler, 'autobots/assembler'
+  autoload :Helpers, 'autobots/helpers'
+end

data/lib/autobots/assembler.rb

@@ -0,0 +1,35 @@
+module Autobots
+  class Assembler
+    class_attribute :serializer
+    attr_reader :identifiers, :options, :objects
+
+    def initialize(identifiers, options = {})
+      @identifiers, @options = identifiers, options
+      @objects = assemble(identifiers)
+    end
+
+    def data
+      roll_out(objects)
+    end
+
+    private
+
+    # builds our skeleton resources
+    def assemble(identifiers)
+      identifiers
+    end
+
+    # fetch any additional objects and modify our resources
+    def transform(resources)
+      resources
+    end
+
+    # serialize everything
+    def roll_out(objects)
+      transform(objects).map{|obj| serializer.new(obj).serializable_hash }
+    end
+
+    prepend Helpers::Caching
+
+  end
+end

data/lib/autobots/helpers.rb

@@ -0,0 +1,6 @@
+module Autobots
+  module Helpers
+    autoload :ActiveRecordPreloading, 'autobots/helpers/active_record_preloading'
+    autoload :Caching, 'autobots/helpers/caching'
+  end
+end

data/lib/autobots/helpers/active_record_preloading.rb

@@ -0,0 +1,18 @@
+module Autobots
+  module Helpers
+    module ActiveRecordPreloading
+
+      def transform(objects)
+        ActiveRecord::Associations::Preloader.new(objects, preloads).run
+        objects
+      rescue ArgumentError
+        ActiveRecord::Associations::Preloader.new.preload(objects, preloads)
+        objects
+      end
+
+      def preloads
+        []
+      end
+    end
+  end
+end

data/lib/autobots/helpers/caching.rb

@@ -0,0 +1,38 @@
+module Autobots
+  module Helpers
+    module Caching
+      def initialize(_, options = {})
+        super
+        @cache = options[:cache]
+      end
+
+      def data
+        return @data if defined?(@data)
+
+        if @cache
+          key_proc = options.fetch(:cache_key) do
+            method(:cache_key)
+          end
+          identifiers = objects.inject({}) do |acc, obj|
+            acc[key_proc.call(obj, self)] = obj
+            acc
+          end
+
+          # misses: { key => obj }
+          @data = BulkCacheFetcher.new(@cache).fetch(identifiers) do |misses|
+            roll_out(misses.values)
+          end
+        else
+          @data = super
+        end
+        @data
+      end
+
+      protected
+
+      def cache_key(object, _)
+        [object.cache_key, serializer.name, 'serializable-hash']
+      end
+    end
+  end
+end

data/lib/autobots/version.rb

@@ -0,0 +1,3 @@
+module Autobots
+  VERSION = "0.0.1"
+end

data/test/active_record_test.rb

@@ -0,0 +1,66 @@
+require 'test_helper'
+
+class ActiveRecordTest < ActiveSupport::TestCase
+  fixtures :all
+
+  def test_basic
+    projects = Project.all
+    assembler = ProjectAssembler.new(projects)
+
+    data = nil
+    assert_queries 5 do
+      data = assembler.data
+    end
+
+    assert data.is_a?(Array)
+    data.each do |item|
+      assert item.is_a?(Hash)
+    end
+  end
+
+  def test_can_preload
+    projects = Project.all
+    assembler = ProjectPreloadAssembler.new(projects)
+
+    data = nil
+    assert_queries 3 do
+      data = assembler.data
+    end
+
+    assert data.is_a?(Array)
+    data.each do |item|
+      assert item.is_a?(Hash)
+    end
+  end
+
+  def test_can_preload_with_mixin
+    projects = Project.all
+    assembler = ProjectPreloadIncludedAssembler.new(projects)
+
+    data = nil
+    assert_queries 3 do
+      data = assembler.data
+    end
+
+    assert data.is_a?(Array)
+    data.each do |item|
+      assert item.is_a?(Hash)
+    end
+  end
+
+  def test_can_use_ids
+    project_ids = Project.all.pluck(:id)
+    assembler = ProjectIdAssembler.new(project_ids)
+
+    data = nil
+    assert_queries 4 do
+      data = assembler.data
+    end
+
+    assert data.is_a?(Array)
+    data.each do |item|
+      assert item.is_a?(Hash)
+    end
+  end
+
+end