knuckles 0.4.0 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 69ff93727664c8cb7f6c22d52542521b400f7421
-  data.tar.gz: 88f1ad40c4d32591dfcc6b9d3e8c093d1c14a80a
+  metadata.gz: 96d10e0c441bc44adc48654b19878d2a7283cd7b
+  data.tar.gz: 4a9df62148abaec7d7fb161b37bf1bab7b511327
 SHA512:
-  metadata.gz: 5bff88dba2e59d125dfa1c61ee050f191f690e1f1a12f704729e1f8e29b76bfe7a444e4bc550cd4a09d3777156114a10b8dc319f6594e92e83ca013ea17255aa
-  data.tar.gz: e51e57e0df18ccaef4304ee9f78a47ee0472063be046592b883cf2e6f732ba68cc9803f2f9636ff09ad9e9b48006603fa1c36fb25203405e6d3f2b8144e50844
+  metadata.gz: 828e4d98d6f655aee4ed20fb0792af61920d9304775b23693b0f3cd87b7a6c4d4d68ceddc20a3895408eca23f88df1220239b74f705d415cf0b3f8b4525862fe
+  data.tar.gz: 4cb0da0daf47140f5ff953214f739d79de15de96f59134dadb803c02887a70c2cc6bf35b0719f5caf1493614e2685794ac58d236917866010cd0b32d54e674a0

data/CHANGELOG.md

@@ -1,3 +1,9 @@
+## v0.5.0 - 2016-07-08
+
+* Added: Accept a `proc` or any callable object as the `keygen`. This simplifies
+  overriding the cache key on a per-instance basis.
+* Added: Lots of documentation! All code has inline documentation now.
+
 ## v0.4.0 - 2016-05-11
 
 * Added: `Knuckles::Active::Hydrator`, a hydrator specifically designed to work

data/README.md

@@ -1,6 +1,7 @@
 [![Build Status](https://travis-ci.org/sorentwo/knuckles.svg?branch=master)](https://travis-ci.org/sorentwo/knuckles)
 [![Coverage Status](https://coveralls.io/repos/github/sorentwo/knuckles/badge.svg?branch=master)](https://coveralls.io/github/sorentwo/knuckles?branch=master)
 [![Code Climate](https://codeclimate.com/github/sorentwo/knuckles/badges/gpa.svg)](https://codeclimate.com/github/sorentwo/knuckles)
+[![Inline Docs](http://inch-ci.org/github/sorentwo/knuckles.svg?branch=master)](http://inch-ci.org/github/sorentwo/knuckles)
 
 # Knuckles (Because Sonic was Taken)
 
@@ -69,6 +70,144 @@ end
 With the top level module configured it is simple to jump right into rendering,
 but we'll look at configuring the pipeline first.
 
+## Understanding and Using Pipelines
+
+Knuckles renders and serializes data through a series of stages composed into a
+pipeline. Stages can easily be added or removed to control how data is
+transformed. Here is a breakdown of the default stages and what their role is
+within the pipeline.
+
+#### Fetcher
+
+The fetcher is responsible for bulk retrieval of data from the cache. Fetching
+is done using a single `read_multi` operation, which is multiplexed in caches
+like Redis or MemCached.
+
+```ruby
+pipeline = Knuckles::Pipeline.new
+
+pipeline.call(posts)
+```
+
+#### Hydrator
+
+Models that couldn't be retrieved from the cache will then be hydrated, a
+process where the stripped down model that was given for fetching is replaced
+with a full model with preloaded associations. The behavior of the hydrator
+stage is entirely controlled by passing a Proc as the `hydrate` option. If the
+`hydrate` proc is omitted hydration will be skipped. Skipping hydration is
+useful if you want a simplified pipeline where full models and their
+associations are preloaded before starting serialization.
+
+See `Knuckles::Active::Hydrator` for an alternative `ActiveRecord` specific
+hydrator. If you are using Knuckles within a Rais app, this is probably the
+hydration stage you want to use.
+
+```ruby
+# Using the standard hydrator
+pipeline.call(posts, hydrator: -> (model) { model.fetch })
+
+# Using active hydrator with a relation that has a `prepared` scope
+pipeline.call(posts, relation: posts.prepared)
+```
+
+#### Renderer
+
+After un-cached models have been hydrated they can be rendered. Rendering is
+synonymous with converting a model to a hash, like calling `as_json` on an
+`ActiveRecord` model. Knuckles provides a minimal (but fast) view module that
+can be used with the rendering step. Alternatively, if you're migrating from
+`ActiveModelSerializers` you can pass in an AMS class instead.
+
+```ruby
+# Using Knuckles::View
+pipeline.call(models, view: PostView)
+
+# Using ActiveModelSerializer
+pipeline.call(models, view: PostSerializer)
+```
+
+#### Writer
+
+After un-cached models have been serialized they are ready to be cached for
+future retrieval. Each fully serialized model is written to the cache in a
+single `write_multi` operation if available (using Readthis, for example). Only
+previously un-cached data will be written to the cache, making the writer a
+no-op when all of the data was cached initially.
+
+#### Enhancer
+
+The enhancer modifies rendered data using proc passed through options. The
+enhancer stage is critical to customizing the final output. For example, if
+staff should have confidential data that regular users can't see you can enhance
+the final values. Another use of enhancers is personalizing an otherwise generic
+response.
+
+```ruby
+# Removing staff only content from the rendered data
+pipeline.call(posts,
+  scope: current_user,
+  enhancer: lambda do |result, options|
+    scope = options[:scope]
+
+    unless scope.staff?
+      result.delete_if { |key, _| key == "confidential" }
+    end
+
+    result
+  end
+)
+```
+
+#### Combiner
+
+The combiner stage merges all of the individually rendered results into a single
+hash. The output of this stage is a single object, ready to be serialized.
+
+#### Dumper
+
+The dumping process combines de-duplication and actual serialization. For every
+top level key that is an array all of the children will have uniqueness
+enforced. For example, if you had rendered a collection of posts that shared the
+same author, you will only have a single author object serialized. Be aware that
+the uniqueness check relies on the presence of an `id` key rather than full
+object comparisons.
+
+Dumping is the final stage of the pipeline. At this point you have a single
+serialized payload in the format of your choice (JSON by default), ready to send
+back as a response.
+
+## Customizing Pipelines
+
+Pipelines stages can be removed, swapped out or otherwise tuned. An array of
+stages can be passed when building a new pipeline. Here is an example of
+creating a customized pipeline without any caching, hydration, or enhancing:
+
+```ruby
+Knuckles::Pipeline.new(stages: [
+  Knuckles::Stages::Renderer,
+  Knuckles::Stages::Combiner,
+  Knuckles::Stages::Dumper
+])
+```
+
+Or, perhaps you want to use the active hydrator instead:
+
+```ruby
+Knuckles::Pipeline.new(stages: [
+  Knuckles::Stages::Fetcher,
+  Knuckles::Active::Hydrator,
+  Knuckles::Stages::Renderer,
+  Knuckles::Stages::Writer,
+  Knuckles::Stages::Enhancer,
+  Knuckles::Stages::Combiner,
+  Knuckles::Stages::Dumper
+])
+```
+
+Note that once the pipeline is initialized the stages are frozen to prevent
+modification.
+
 ## Defining Views for Rendering
 
 While you can use Knuckles with other serializers, you can also use the provided
@@ -95,6 +234,35 @@ end
 
 See `Knuckles::View` for more usage details.
 
+## Rendering in Rails
+
+One driving factor of Knuckles is that code should be explicit. As a result
+there isn't a default Railtie that will integrate Knuckles into the
+`ActiveController` rendering process for you. Luckily there isn't much to
+setting up a new pipeline for rendering. Add this to your
+`ApplicationController` or an API specific controller:
+
+```ruby
+def knuckles_render(relation, options)
+  Knuckles::Pipeline.new.call(relation, options)
+end
+```
+
+Now you can easily render responses:
+
+```ruby
+def index
+  posts = posts.published.paginate(pagination_params)
+
+  render json: knuckles_render(
+    posts.select(:id, :updated_at),
+    relation: posts.prepared,
+    view: PostView,
+    scope: current_user,
+  )
+end
+```
+
 ## Contributing
 
 1. Fork it ( https://github.com/sorentwo/knuckles/fork )

data/lib/knuckles.rb

@@ -33,10 +33,13 @@ module Knuckles
   autoload :Pipeline, "knuckles/pipeline"
   autoload :View, "knuckles/view"
 
+  # Top level wrapper for stages that are expected to interact with `Active*`
+  # libraries, such as `ActiveModel`.
   module Active
     autoload :Hydrator, "knuckles/active/hydrator"
   end
 
+  # Top level wrapper for standard pipleline stages.
   module Stages
     autoload :Combiner, "knuckles/stages/combiner"
     autoload :Dumper, "knuckles/stages/dumper"
@@ -104,7 +107,8 @@ module Knuckles
     yield self
   end
 
-  # @private
+  # Reset all configuration values back to `nil`, restoring them to the
+  # defaults. This is useful for testing because configuration is global.
   def reset!
     @cache = nil
     @keygen = nil

data/lib/knuckles/active/hydrator.rb

@@ -24,8 +24,8 @@ module Knuckles
       #
       # @example Hydrating missing objects
       #
-      #   prepared = [Post.new(1), Post.new(2)]
       #   relation = Post.all.preload(:author, :comments)
+      #   prepared = relation.select(:id, :updated_at)
       #
       #   Knuckles::Active::Hydrator.call(prepared, relation: relation) #=>
       #     # [{object: #Post<1>, cached?: false, ...

data/lib/knuckles/pipeline.rb

@@ -30,7 +30,7 @@ module Knuckles
     #     Knuckles::Stages::Renderer,
     #     Knuckles::Stages::Combiner,
     #     Knuckles::Stages::Dumper
-    #   ]
+    #   ])
     def initialize(stages: default_stages)
       @stages = stages.freeze
     end

data/lib/knuckles/stages/combiner.rb

@@ -2,10 +2,47 @@
 
 module Knuckles
   module Stages
+    # The combiner stage merges all of the individually rendered results into a
+    # single hash. The output of this stage is a single object with string keys
+    # and array values, ready to be serialized.
     module Combiner
       extend self
 
-      def call(prepared, _)
+      # Merge all of the rendered data into a single hash. Each
+      # resulting value will be an array, even if there was only one
+      # value in the original rendered results.
+      #
+      # @param [Enumerable] prepared The prepared collection to be combined
+      # @param [Hash] _options Options aren't used, but are accepted
+      #   to maintain a consistent interface
+      #
+      # @example Combining rendered data
+      #
+      #     prepared = [
+      #       {
+      #         result: {
+      #           author: {id: 1, name: "Michael"},
+      #           posts: [{id: 1, title: "hello"}],
+      #         }
+      #       }, {
+      #         result: {
+      #           author: {id: 1, name: "Michael"},
+      #           posts: [{id: 2, title: "there"}],
+      #         }
+      #       }
+      #     ]
+      #
+      #     Knuckles::Stage::Combiner.call(prepared, {}) #=> {
+      #       "author" => [
+      #         {id: 1, name: "Michael"}
+      #       ],
+      #       "posts" => [
+      #         {id: 1, title: "hello"},
+      #         {id: 2, title: "there"}
+      #       ]
+      #     }
+      #
+      def call(prepared, _options)
         prepared.each_with_object(array_backed_hash) do |hash, memo|
           hash[:result].each do |root, values|
             case values
@@ -16,6 +53,8 @@ module Knuckles
         end
       end
 
+      private
+
       def array_backed_hash
         Hash.new { |hash, key| hash[key] = [] }
       end

data/lib/knuckles/stages/dumper.rb

@@ -2,9 +2,23 @@
 
 module Knuckles
   module Stages
+    # The dumping process combines de-duplication and actual serialization. For
+    # every top level key that is an array all of the children will have
+    # uniqueness enforced. For example, if you had rendered a collection of
+    # posts that shared the same author, you will only have a single author
+    # object serialized. Be aware that the uniqueness check relies on the
+    # presence of an `id` key rather than full object comparisons.
     module Dumper
       extend self
 
+      # De-duplicate values in all keys and merge them into a single hash.
+      # Afterwards the complete hash is serialized using the serializer
+      # configured at `Knuckles.serializer`.
+      #
+      # @param [Enumerable<Hash>] objects A collection of hashes to be dumped
+      # @param [Hash] _options Options aren't used, but are accepted
+      #   to maintain a consistent interface
+      #
       def call(objects, _options)
         Knuckles.serializer.dump(keys_to_arrays(objects))
       end

data/lib/knuckles/stages/enhancer.rb

@@ -2,9 +2,38 @@
 
 module Knuckles
   module Stages
+    # The enhancer modifies rendered data using proc passed through options.
+    # The enhancer stage is critical to customizing the final output. For
+    # example, if staff should have confidential data that regular users can't
+    # see you can enhance the final values. Another use of enhancers is
+    # personalizing an otherwise generic response.
     module Enhancer
       extend self
 
+      # Modify all results using an `enhancer` proc.
+      #
+      # @param [Enumerable] prepared The prepared collection to be enhanced
+      # @option [Proc] :enhancer A `proc`, `lambda`, or any object that responds
+      #   to `call`. Every complete `result` in the prepared collection will be
+      #   passed to the enhancer.
+      #
+      # @example Removing tags unless the scope is staff
+      #
+      #    enhancer = lambda do |result, options|
+      #      scope = options[:scope]
+      #
+      #      unless scope.staff?
+      #        result.delete_if { |key, _| key == "tags" }
+      #      end
+      #
+      #      result
+      #    end
+      #
+      #    prepared = [{result: {"posts" => [], "tags" => []}}]
+      #
+      #    Knuckles::Stages::Enhancer.call(prepared, enhancer: enhancer) #=>
+      #     # [{result: {"posts" => []}}]
+      #
       def call(prepared, options)
         enhancer = options[:enhancer]
 

data/lib/knuckles/stages/fetcher.rb

@@ -2,16 +2,44 @@
 
 module Knuckles
   module Stages
+    # The fetcher is responsible for bulk retrieval of data from the cache.
+    # Fetching is done using a single `read_multi` operation, which is
+    # multiplexed in caches like Redis or MemCached.
+    #
+    # The underlying cache *must* support `read_multi` for the stage to work.
     module Fetcher
       extend self
 
+      # Fetch all previously cached objects from the configured store.
+      #
+      # @param [Enumerable] prepared The prepared collection to fetch
+      # @option [Module] :keygen (Knuckles.keygen) The cache key generator used
+      #   to construct an entries cache_key. It can be any object that responds
+      #   to `expand_key`
+      #
+      # @example Provide a custom keygen
+      #
+      #     keygen = Module.new do
+      #       def self.expand_key(object)
+      #         object.name
+      #       end
+      #     end
+      #
+      #     Knuckles::Stages::Fetcher.call(prepared, keygen: keygen)
+      #
+      # @example Use a lambda as a keygen
+      #
+      #     Knuckles::Stages::Fetcher.call(
+      #       prepared,
+      #       keygen: -> (object) { object.name }
+      #     )
+      #
       def call(prepared, options)
         results = get_cached(prepared, options)
 
         prepared.each do |hash|
-          result = results[hash[:key]]
-          hash[:cached?] = !result.nil?
-          hash[:result] = result
+          hash[:result] = results[hash[:key]]
+          hash[:cached?] = !hash[:result].nil?
         end
       end
 
@@ -20,11 +48,19 @@ module Knuckles
       def get_cached(prepared, options)
         kgen = options.fetch(:keygen, Knuckles.keygen)
         keys = prepared.map do |hash|
-          hash[:key] = kgen.expand_key(hash[:object])
+          hash[:key] = expand_key(kgen, hash[:object])
         end
 
         Knuckles.cache.read_multi(*keys)
       end
+
+      def expand_key(keygen, object)
+        if keygen.respond_to?(:call)
+          keygen.call(object)
+        else
+          keygen.expand_key(object)
+        end
+      end
     end
   end
 end

data/lib/knuckles/stages/hydrator.rb

@@ -2,9 +2,33 @@
 
 module Knuckles
   module Stages
+    # The hydrator converts minimal objects in a prepared collection into fully
+    # "hydrated" versions of the same record. For example, the initial `model`
+    # may only have the `id` and `updated_at` timestamp selected, which is
+    # ideal for fetching from the cache. If the object wasn't in the cache then
+    # all of the fields are needed for a complete rendering, so the hydration
+    # call will use the passed relation to fetch the full model and any
+    # associations.
+    #
+    # This is a generic hydrator suitable for any type of collection.  If you
+    # are working with `ActiveRecord` you'll want to use the
+    # `Knuckles::Active::Hydrator` module instead.
     module Hydrator
       extend self
 
+      # Convert all uncached objects into their full representation.
+      #
+      # @param [Enumerable] prepared The prepared collection for processing
+      # @option [Proc, #call] :hydrate A proc used to load missing data for
+      #   uncached objects
+      #
+      # @example Hydrating missing objects
+      #
+      #   Knuckles::Hydrator.call(
+      #     prepared,
+      #     hydrate: -> (objects) { objects.each(&:fetch!) }
+      #   )
+      #
       def call(prepared, options)
         hydrate = options[:hydrate]
 

data/lib/knuckles/stages/renderer.rb

@@ -2,9 +2,34 @@
 
 module Knuckles
   module Stages
+    # After un-cached models have been hydrated they can be rendered. Rendering
+    # is synonymous with converting a model to a hash, like calling `as_json`
+    # on an `ActiveRecord` model. Knuckles provides a minimal (but fast) view
+    # module that can be used with the rendering step. Alternatively, if you're
+    # migrating from `ActiveModelSerializers` you can pass in an AMS class
+    # instead.
     module Renderer
       extend self
 
+      # Serialize all un-cached objects into hashes.
+      #
+      # @param [Enumerable] objects The prepared collection to be rendered
+      # @option [Module] :view A `Knuckles::View` compliant module,
+      #   it will be passed the object and any options. Alternately,
+      #   a class compatible with the `ActiveModelSerializers` API.
+      #
+      # @example Using a Knuckles::View
+      #
+      #     module PostView
+      #       extend Knuckles::View
+      #
+      #       def self.data(post, _options)
+      #         {id: post.id, name: post.name}
+      #       end
+      #     end
+      #
+      #     pipeline.call(models, view: PostView)
+      #
       def call(objects, options)
         view = options.fetch(:view)
 

data/lib/knuckles/stages/writer.rb

@@ -2,10 +2,23 @@
 
 module Knuckles
   module Stages
+    # After un-cached models have been serialized they are ready to be cached
+    # for future retrieval. Each fully serialized model is written to the cache
+    # in a single `write_multi` operation if available (using Readthis, for
+    # example). Only previously un-cached data will be written to the cache,
+    # making the writer a no-op when all of the data was cached initially.
     module Writer
       extend self
 
-      def call(objects, _)
+      # Write all serialized, but previously un-cached, data to the cache.
+      #
+      # @param [Enumerable] objects A collection of hashes to be serialized,
+      #   each hash must have they keys `:key`, `:result`, and `:cached?`.
+      # @param [Hash] _options Options aren't used, but are accepted
+      #   to maintain a consistent interface
+      # @return The original enumerable is returned unchanged
+      #
+      def call(objects, _options)
         if cache.respond_to?(:write_multi)
           write_multi(objects)
         else

data/lib/knuckles/version.rb

@@ -1,5 +1,6 @@
 # frozen_string_literal: true
 
 module Knuckles
-  VERSION = "0.4.0"
+  # The current version of Knuckles
+  VERSION = "0.5.0"
 end

data/lib/knuckles/view.rb

@@ -46,8 +46,8 @@ module Knuckles
     # Convenience for combining the results of data and relations
     # into a single object.
     #
-    # @param [Object] _object The object for serializing.
-    # @param [Hash] _options The options to be used during serialization, i.e.
+    # @param [Object] object The object for serializing.
+    # @param [Hash] options The options to be used during serialization, i.e.
     #   `:scope`
     #
     # @return [Hash] A hash representing the serialized object and relations.

data/spec/knuckles/stages/fetcher_spec.rb

@@ -24,6 +24,15 @@ RSpec.describe Knuckles::Stages::Fetcher do
 
       expect(pluck(results, :key)).to eq(["alpha"])
     end
+
+    it "allows using a lambda as a keygen" do
+      results = Knuckles::Stages::Fetcher.call(
+        prepare([Tag.new(1, "alpha")]),
+        keygen: -> (object) { object.name }
+      )
+
+      expect(pluck(results, :key)).to eq(["alpha"])
+    end
   end
 
   def pluck(enum, key)

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: knuckles
 version: !ruby/object:Gem::Version
-  version: 0.4.0
+  version: 0.5.0
 platform: ruby
 authors:
 - Parker Selbert
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-05-11 00:00:00.000000000 Z
+date: 2016-07-09 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport