knuckles 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 19a29c342c27e5b5dd82cf59118e2145707f2d06
-  data.tar.gz: 9660418feed584f2ae6b0b75a54ece8a31fab387
+  metadata.gz: ac7f83df75c4e52ee59dc85ff6d6d98c863f5963
+  data.tar.gz: e1ff17c2903f39eac251ee3bbaaa56ab1adfc1e5
 SHA512:
-  metadata.gz: a29a4f519dcae90684811f5905e3641d20823e681c487dbefaea4a0157abc8e242ec368fa6a6fc5fbbcca6faddef9a9d7dadea8f41f241c6be3c7af158bcfd2f
-  data.tar.gz: e6662ef32ed48439b934cb2997f8cbb8f21b539aba41ac28e30aa57ebdcc7847732e9b1028fd58485f1795a220cc655165ad3d759851e781b8faf75f122c3894
+  metadata.gz: d49c80cd7f0e4b2cefc1861188b84bf9ccd752975d6714d068c8a97798a8e95e615a6192762afe721a6461f9ec23105d66d46651b1cd809fa7c3d6abb577eb8d
+  data.tar.gz: 9e7bd47dd8b17edba6e3e88fa367bca797654b876fc8a4fb2f87f25189bbb503a3c72e8cca288c301ec6c0876e94a5a693ded4ca8a51031bb90e1649b5e80ec3

data/CHANGELOG.md

@@ -1,3 +1,11 @@
+## v0.3.0 - 2016-04-07
+
+* Added: Tons of documentation.
+* Removed: Remove stage manipulation (`insert_*`, `delete`). All configuration
+  must be done on pipeline init.
+* Change: Move stage modules into the `Stages` namespace.
+* Change: Use module names rather than provide a custom name method.
+
 ## v0.2.0 - 2016-03-23
 
 * Add: Introduce the enhancer stage, used to augment data after it has been

data/README.md

@@ -4,7 +4,10 @@
 
 # Knuckles (Because Sonic was Taken)
 
-What's it all about?
+Knuckles is a performance focused data serialization pipeline. More simply, it
+tries to serialize models into large JSON payloads as quickly as possible.
+
+### What's it all about?
 
 * Emphasis on caching as a composable operation
 * Reduced object instantiation
@@ -13,6 +16,26 @@ What's it all about?
 * Minimal runtime dependencies
 * Explicit serializer view API with as little overhead and no DSL
 
+### Is It Better?
+
+Knuckles is absolutely faster and has a lower memory overhead than uncached
+or cached usage of `ActiveModelSerializers`, and significantly faster than
+cached use of `ActiveModelSerializers` with the [perforated][perforated] gem.
+
+Here are performance and memory comparisons for an endpoint that has been cached
+with Perforated and with Knuckles. All measurments were done with production
+settings over the local network.
+
+|                | average | longest | shortest | allocated | retained |
+| -------------- | ------- | ------- | -------- | --------- | -------- |
+| perforated/ams | 230ms   | 560ms   | 190ms    | 148,735   | 18,203   |
+| knuckles/ams	 | 30ms    | 60ms    | 20ms     | 19,603    | 136      |
+
+These are measurements for a sizable payload with hundreds of associated
+records.
+
+[perforated]: https://github.com/sorentwo/perforated
+
 ## Installation
 
 Add this line to your application's Gemfile:
@@ -46,28 +69,32 @@ end
 With the top level module configured it is simple to jump right into rendering,
 but we'll look at configuring the pipeline first.
 
-## Defining Views
+## Defining Views for Rendering
 
-The interface for defining serializers is largely based on Active Model
-Serializers, but with a few key differences.
+While you can use Knuckles with other serializers, you can also use the provided
+view layer. Knuckles views are simple templates that let you build up data and
+relations. They look like this:
 
 ```ruby
-class ScoutSerializer
-  include Knuckles::Serializer
-
-  root 'scout'
+module ScoutView
+  extend Knuckles::View
 
-  attributes :id, :email, :display_name
+  def self.root
+    :scouts
+  end
 
-  has_one  :thing, serializer: ThingSerializer
-  has_many :other, serializer: OtherThingSerializer
+  def self.data(object, options)
+    {id: object.id, email: object.email, name: object.name}
+  end
 
-  def display_name(object, options)
-    "#{object.first_name} #{object.last_name}"
+  def self.relations(object, options)
+    {things: has_many(object.things, ThingView)}
   end
 end
 ```
 
+See `Knuckles::View` for more usage details.
+
 ## Contributing
 
 1. Fork it ( https://github.com/sorentwo/knuckles/fork )

data/lib/knuckles/keygen.rb

@@ -1,9 +1,31 @@
 # frozen_string_literal: true
 
 module Knuckles
+  # The Keygen module provides simple cache key generation. The global keygen
+  # strategy can be changed at the top level by configuring `Knuckles.keygen`.
+  #
+  # Any object that responds to `expand_key` with a single argument can be
+  # used instead.
+  #
+  # @example Change global keygen stragey
+  #
+  #   Knuckles.keygen = MyCustomKeygen.new
+  #
   module Keygen
     extend self
 
+    # Calculates a cache key for the given object. It first attempts to use
+    # the object's `cache_key` method (present on `ActiveRecord` models). It
+    # falls back to combining the object's `id` and `updated_at` values.
+    #
+    # @param [Object] object An object to caclculate the key from
+    #
+    # @return [String] Computed cache key
+    #
+    # @example
+    #
+    #   Knuckles::Keygen.expand_key(model) #=> "MyModel/1/1234567890"
+    #
     def expand_key(object)
       if object.respond_to?(:cache_key)
         object.cache_key

data/lib/knuckles/pipeline.rb

@@ -1,39 +1,81 @@
 # frozen_string_literal: true
 
 module Knuckles
+  # The `Pipeline` is used to transform a collection of objects into a
+  # serialized result. A pipeline reduces a collection of objects through a set
+  # of stages, passing the result of each stage on to the next. This means that
+  # each stage can act in isolation and can compose with new custom stages.
+  # Additionally, stages are idividually instrumented for performance
+  # monitoring.
+  #
+  # The `Pipeline` class provides a set of default stages but can be overridden
+  # on initialization. Note that after initialization, the stages are frozen to
+  # prevent unpredictable mutation.
   class Pipeline
-    def self.default_stages
-      [Knuckles::Fetcher,
-       Knuckles::Hydrator,
-       Knuckles::Renderer,
-       Knuckles::Writer,
-       Knuckles::Enhancer,
-       Knuckles::Combiner,
-       Knuckles::Dumper]
-    end
-
     attr_reader :stages
 
-    def initialize(stages: self.class.default_stages)
-      @stages = stages
-    end
-
-    def delete(stage)
-      stages.delete(stage)
+    # Creates a new instance of `Knuckles::Pipeline`, optionally with custom
+    # stages.
+    #
+    # @option [Array] :stages (default_stages) An array of stages to pipe
+    #   results through
+    #
+    # @example Create a default pipeline
+    #
+    #   Knuckles::Pipeline.new
+    #
+    # @example Create a customized pipeline without any caching
+    #
+    #   Knuckles::Pipeline.new(stages: [
+    #     Knuckles::Stages::Renderer,
+    #     Knuckles::Stages::Combiner,
+    #     Knuckles::Stages::Dumper
+    #   ]
+    def initialize(stages: default_stages)
+      @stages = stages.freeze
     end
 
-    def insert_after(stage, new_stage)
-      index = stages.index(stage)
-
-      stages.insert(index + 1, new_stage)
-    end
-
-    def insert_before(stage, new_stage)
-      index = stages.index(stage)
-
-      stages.insert(index, new_stage)
+    # Provides default stage modules in the intended order. These are the
+    # stages that are used if nothing is passed during initialization. The
+    # defaults are defined as a method to make overriding with a subclass easy.
+    #
+    # @example Override `default_stages` within a subclass
+    #
+    #  class CustomPipeline < Knuckles::Pipeline
+    #    def default_stages
+    #      [Knuckles::Stages::Renderer,
+    #       Knuckles::Stages::Combiner,
+    #       Knuckles::Stages::Dumper]
+    #    end
+    #  end
+    #
+    def default_stages
+      [Knuckles::Stages::Fetcher,
+       Knuckles::Stages::Hydrator,
+       Knuckles::Stages::Renderer,
+       Knuckles::Stages::Writer,
+       Knuckles::Stages::Enhancer,
+       Knuckles::Stages::Combiner,
+       Knuckles::Stages::Dumper]
     end
 
+    # Push a collection of objects through the stages of the pipeline. In
+    # normal usage this will render the objects out to a JSON structure.
+    #
+    # @param [Enumerable] objects A collection of objects (models) to be
+    #   serialized and processed.
+    # @param [Hash] options The `call` method doesn't use any options itself,
+    #   they are forwarded on to each stage. See the documentation for specific
+    #   stages for the options they accept.
+    #
+    # @return [String] The final result as transformed by the pipeline,
+    #   typically a JSON string.
+    #
+    # @example Basic pipeline rendering
+    #
+    #   pipeline = Knuckles::Pipeline.new
+    #   pipeline.call([tag_a, tag_b]) #=> '{"tags":[{"id":1},{"id":2}]}'
+    #
     def call(objects, options)
       prepared = prepare(objects)
 
@@ -44,6 +86,20 @@ module Knuckles
       end
     end
 
+    # Convert a collection of objects into a collection of hashes that enclose
+    # the object. The resulting hashes are populated with the keys and default
+    # values necessary for use with the standard pipeline stages.
+    #
+    # @param [Enumerable] objects A collection of objects to prepare
+    #
+    # @return [Array[Hash]] An array of hashes, each with the keys `:object`,
+    #   `:key`, `:cached?`, and `:result`.
+    #
+    # @example Prepare a single object
+    #
+    #   pipeline.prepare([model]) #=> [{object: model, key: nil,
+    #                                   cached?: false, result: nil}]
+    #
     def prepare(objects)
       objects.map do |object|
         {object: object, key: nil, cached?: false, result: nil}

data/lib/knuckles/stages/combiner.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+module Knuckles
+  module Stages
+    module Combiner
+      extend self
+
+      def call(prepared, _)
+        prepared.each_with_object(array_backed_hash) do |hash, memo|
+          hash[:result].each do |root, values|
+            case values
+            when Hash  then memo[root.to_s] << values
+            when Array then memo[root.to_s] += values
+            end
+          end
+        end
+      end
+
+      def array_backed_hash
+        Hash.new { |hash, key| hash[key] = [] }
+      end
+    end
+  end
+end

data/lib/knuckles/stages/dumper.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+module Knuckles
+  module Stages
+    module Dumper
+      extend self
+
+      def call(objects, _options)
+        Knuckles.serializer.dump(keys_to_arrays(objects))
+      end
+
+      private
+
+      def keys_to_arrays(objects)
+        objects.each do |_, value|
+          if value.is_a?(Array)
+            value.uniq! { |hash| hash["id"] || hash[:id] }
+          end
+        end
+      end
+    end
+  end
+end

data/lib/knuckles/stages/enhancer.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+module Knuckles
+  module Stages
+    module Enhancer
+      extend self
+
+      def call(prepared, options)
+        enhancer = options[:enhancer]
+
+        if enhancer
+          prepared.each do |hash|
+            hash[:result] = enhancer.call(hash[:result], options)
+          end
+        end
+
+        prepared
+      end
+    end
+  end
+end

data/lib/knuckles/stages/fetcher.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+module Knuckles
+  module Stages
+    module Fetcher
+      extend self
+
+      def call(prepared, options)
+        results = get_cached(prepared, options)
+
+        prepared.each do |hash|
+          result = results[hash[:key]]
+          hash[:cached?] = !result.nil?
+          hash[:result] = result
+        end
+      end
+
+      private
+
+      def get_cached(prepared, options)
+        kgen = options.fetch(:keygen, Knuckles.keygen)
+        keys = prepared.map do |hash|
+          hash[:key] = kgen.expand_key(hash[:object])
+        end
+
+        Knuckles.cache.read_multi(*keys)
+      end
+    end
+  end
+end

data/lib/knuckles/stages/hydrator.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+module Knuckles
+  module Stages
+    module Hydrator
+      extend self
+
+      def call(prepared, options)
+        hydrate = options[:hydrate]
+
+        if hydrate && any_missing?(prepared)
+          hydrate.call(hydratable(prepared))
+        end
+
+        prepared
+      end
+
+      private
+
+      def any_missing?(prepared)
+        prepared.any? { |hash| !hash[:cached?] }
+      end
+
+      def hydratable(prepared)
+        prepared.reject { |hash| hash[:cached?] }
+      end
+    end
+  end
+end

data/lib/knuckles/stages/renderer.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+module Knuckles
+  module Stages
+    module Renderer
+      extend self
+
+      def call(objects, options)
+        view = options.fetch(:view)
+
+        objects.each do |hash|
+          unless hash[:cached?]
+            hash[:result] = do_render(hash[:object], view, options)
+          end
+        end
+      end
+
+      private
+
+      def do_render(object, view, options)
+        view.relations(object, options).merge!(
+          view.root => [view.data(object, options)]
+        )
+      end
+    end
+  end
+end

data/lib/knuckles/stages/writer.rb

@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+module Knuckles
+  module Stages
+    module Writer
+      extend self
+
+      def call(objects, _)
+        if cache.respond_to?(:write_multi)
+          write_multi(objects)
+        else
+          write_each(objects)
+        end
+
+        objects
+      end
+
+      private
+
+      def cache
+        Knuckles.cache
+      end
+
+      def write_each(objects)
+        objects.each do |hash|
+          cache.write(hash[:key], hash[:result]) unless hash[:cached?]
+        end
+      end
+
+      def write_multi(objects)
+        writable = objects.each_with_object({}) do |hash, memo|
+          next if hash[:cached?]
+
+          memo[hash[:key]] = hash[:result]
+        end
+
+        cache.write_multi(writable) if writable.any?
+      end
+    end
+  end
+end

data/lib/knuckles/stages.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+module Knuckles
+  module Stages
+    autoload :Combiner, "knuckles/stages/combiner"
+    autoload :Dumper, "knuckles/stages/dumper"
+    autoload :Enhancer, "knuckles/stages/enhancer"
+    autoload :Fetcher, "knuckles/stages/fetcher"
+    autoload :Hydrator, "knuckles/stages/hydrator"
+    autoload :Renderer, "knuckles/stages/renderer"
+    autoload :Writer, "knuckles/stages/writer"
+  end
+end

data/lib/knuckles/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Knuckles
-  VERSION = "0.2.0".freeze
+  VERSION = "0.3.0".freeze
 end

data/lib/knuckles/view.rb

@@ -1,28 +1,127 @@
 # frozen_string_literal: true
 
 module Knuckles
+  # An absolutely bare bones serializer. It is meant as a replacement for
+  # `ActiveModelSerializers`, but is entirely focused on being simple and
+  # explicit. Views are templates that satisfy the interface of:
+  #
+  # * root #=> symbol
+  # * data #=> hash
+  # * relations #=> hash
+  #
+  # Any object that satisfies that interface can be rendered correctly by the
+  # `Renderer` stage.
+  #
+  # @example Extending for a custom view
+  #
+  #   module TagView
+  #     extend Knuckles::View
+  #
+  #     def root
+  #       :tags
+  #     end
+  #
+  #     def data(tag, _)
+  #       {id: tag.id, name: tag.name}
+  #     end
+  #
+  #     def relations(tag, _)
+  #       {posts: has_many(tag.posts, PostView)}
+  #     end
+  #   end
+  #
   module View
     extend self
 
-    ## Callbacks
-
+    # Specifies the top level key that data will be stored under. The value
+    # will not be stringified or pluralized during rendering, so be aware of
+    # the format.
+    #
+    # @return [Symbol, nil] By default root is `nil`, it should be overridden to
+    #   return a plural symbol.
+    #
     def root
     end
 
+    # Serialize an object into a hash. This simply returns an empty hash by
+    # default, it must be overridden by submodules.
+    #
+    # @param [Object] _object The object for serializing.
+    # @param [Hash] _options The options to be used during serialization, i.e.
+    #   `:scope`
+    #
+    # @return [Hash] A hash representing the serialized object.
+    #
+    # @example Overriding data
+    #
+    #   module TagView
+    #     extend Knuckles::View
+    #
+    #     def data(tag, _)
+    #       {id: tag.id, name: tag.name}
+    #     end
+    #   end
+    #
     def data(_object, _options = {})
       {}
     end
 
+    # Extracts associations for an object. Later these are merged with the
+    # output of `data`. View relations are shallow, meaning the relations of
+    # relations are not included.
+    #
+    # @param [Object] _object The object to extract relations from
+    # @param [Hash] _options The options used during extraction, i.e. `:scope`
+    #
+    # @return [Hash] The serialized associations
+    #
+    # @example Override relations
+    #
+    #   module PostView
+    #     extend Knuckles::View
+    #
+    #     def relations(post, _)
+    #       {author: has_one(post.author, AuthorView),
+    #        comments: has_many(post.comments, CommentView)}
+    #     end
+    #   end
+    #
     def relations(_object, _options = {})
       {}
     end
 
-    ## Relations
-
+    # Renders an associated object using the specified view, wrapping the
+    # results in an array.
+    #
+    # @param [Object] object The associated object to serialize.
+    # @param [Module] view A module responding to `data`.
+    # @param [Hash] options Passed to the view for rendering.
+    #
+    # @return [Array<Hash>] A single rendered data object is always returned.
+    #
+    # @example Render a single association
+    #
+    #   PostView.has_one(post.author, AuthorView) #=> [{id: 1, name: "Author"}]
+    #
     def has_one(object, view, options = {})
-      [view.data(object, options)]
+      has_many([object], view, options)
     end
 
+    # Renders all associated objects using the specified view.
+    #
+    # @param [Array] objects Array of associated objects to serialize.
+    # @param [Module] view A module responding to `data`.
+    # @param [Hash] options The options passed to the view for rendering.
+    #
+    # @return [Array<Hash>] All rendered association data.
+    #
+    # @example Render a single association
+    #
+    #   PostView.has_one(post.authors, AuthorView) #=> [
+    #     {id: 1, name: "Me"},
+    #     {id: 2, name: "You"}
+    #   ]
+    #
     def has_many(objects, view, options = {})
       objects.map { |object| view.data(object, options) }
     end