knuckles 0.2.0 → 0.3.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (46) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +8 -0
  3. data/README.md +40 -13
  4. data/lib/knuckles/keygen.rb +22 -0
  5. data/lib/knuckles/pipeline.rb +82 -26
  6. data/lib/knuckles/stages/combiner.rb +24 -0
  7. data/lib/knuckles/stages/dumper.rb +23 -0
  8. data/lib/knuckles/stages/enhancer.rb +21 -0
  9. data/lib/knuckles/stages/fetcher.rb +30 -0
  10. data/lib/knuckles/stages/hydrator.rb +29 -0
  11. data/lib/knuckles/stages/renderer.rb +27 -0
  12. data/lib/knuckles/stages/writer.rb +41 -0
  13. data/lib/knuckles/stages.rb +13 -0
  14. data/lib/knuckles/version.rb +1 -1
  15. data/lib/knuckles/view.rb +104 -5
  16. data/lib/knuckles.rb +55 -7
  17. data/spec/knuckles/pipeline_spec.rb +0 -40
  18. data/spec/knuckles/{combiner_spec.rb → stages/combiner_spec.rb} +2 -2
  19. data/spec/knuckles/{dumper_spec.rb → stages/dumper_spec.rb} +2 -2
  20. data/spec/knuckles/{enhancer_spec.rb → stages/enhancer_spec.rb} +3 -3
  21. data/spec/knuckles/{fetcher_spec.rb → stages/fetcher_spec.rb} +3 -3
  22. data/spec/knuckles/{hydrator_spec.rb → stages/hydrator_spec.rb} +3 -3
  23. data/spec/knuckles/{renderer_spec.rb → stages/renderer_spec.rb} +2 -2
  24. data/spec/knuckles/{writer_spec.rb → stages/writer_spec.rb} +3 -3
  25. metadata +31 -43
  26. data/.gitignore +0 -15
  27. data/.rspec +0 -2
  28. data/.rubocop.yml +0 -41
  29. data/.travis.yml +0 -10
  30. data/Gemfile +0 -11
  31. data/Rakefile +0 -9
  32. data/bench/bench_helper.rb +0 -48
  33. data/bench/fixtures/serializers.rb +0 -93
  34. data/bench/fixtures/submissions.json +0 -1
  35. data/bench/profiling.rb +0 -14
  36. data/bench/realistic.rb +0 -9
  37. data/bench/simple.rb +0 -25
  38. data/bin/rspec +0 -16
  39. data/knuckles.gemspec +0 -24
  40. data/lib/knuckles/combiner.rb +0 -26
  41. data/lib/knuckles/dumper.rb +0 -25
  42. data/lib/knuckles/enhancer.rb +0 -23
  43. data/lib/knuckles/fetcher.rb +0 -32
  44. data/lib/knuckles/hydrator.rb +0 -31
  45. data/lib/knuckles/renderer.rb +0 -29
  46. data/lib/knuckles/writer.rb +0 -43
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 19a29c342c27e5b5dd82cf59118e2145707f2d06
4
- data.tar.gz: 9660418feed584f2ae6b0b75a54ece8a31fab387
3
+ metadata.gz: ac7f83df75c4e52ee59dc85ff6d6d98c863f5963
4
+ data.tar.gz: e1ff17c2903f39eac251ee3bbaaa56ab1adfc1e5
5
5
  SHA512:
6
- metadata.gz: a29a4f519dcae90684811f5905e3641d20823e681c487dbefaea4a0157abc8e242ec368fa6a6fc5fbbcca6faddef9a9d7dadea8f41f241c6be3c7af158bcfd2f
7
- data.tar.gz: e6662ef32ed48439b934cb2997f8cbb8f21b539aba41ac28e30aa57ebdcc7847732e9b1028fd58485f1795a220cc655165ad3d759851e781b8faf75f122c3894
6
+ metadata.gz: d49c80cd7f0e4b2cefc1861188b84bf9ccd752975d6714d068c8a97798a8e95e615a6192762afe721a6461f9ec23105d66d46651b1cd809fa7c3d6abb577eb8d
7
+ data.tar.gz: 9e7bd47dd8b17edba6e3e88fa367bca797654b876fc8a4fb2f87f25189bbb503a3c72e8cca288c301ec6c0876e94a5a693ded4ca8a51031bb90e1649b5e80ec3
data/CHANGELOG.md CHANGED
@@ -1,3 +1,11 @@
1
+ ## v0.3.0 - 2016-04-07
2
+
3
+ * Added: Tons of documentation.
4
+ * Removed: Remove stage manipulation (`insert_*`, `delete`). All configuration
5
+ must be done on pipeline init.
6
+ * Change: Move stage modules into the `Stages` namespace.
7
+ * Change: Use module names rather than provide a custom name method.
8
+
1
9
  ## v0.2.0 - 2016-03-23
2
10
 
3
11
  * Add: Introduce the enhancer stage, used to augment data after it has been
data/README.md CHANGED
@@ -4,7 +4,10 @@
4
4
 
5
5
  # Knuckles (Because Sonic was Taken)
6
6
 
7
- What's it all about?
7
+ Knuckles is a performance focused data serialization pipeline. More simply, it
8
+ tries to serialize models into large JSON payloads as quickly as possible.
9
+
10
+ ### What's it all about?
8
11
 
9
12
  * Emphasis on caching as a composable operation
10
13
  * Reduced object instantiation
@@ -13,6 +16,26 @@ What's it all about?
13
16
  * Minimal runtime dependencies
14
17
  * Explicit serializer view API with as little overhead and no DSL
15
18
 
19
+ ### Is It Better?
20
+
21
+ Knuckles is absolutely faster and has a lower memory overhead than uncached
22
+ or cached usage of `ActiveModelSerializers`, and significantly faster than
23
+ cached use of `ActiveModelSerializers` with the [perforated][perforated] gem.
24
+
25
+ Here are performance and memory comparisons for an endpoint that has been cached
26
+ with Perforated and with Knuckles. All measurments were done with production
27
+ settings over the local network.
28
+
29
+ | | average | longest | shortest | allocated | retained |
30
+ | -------------- | ------- | ------- | -------- | --------- | -------- |
31
+ | perforated/ams | 230ms | 560ms | 190ms | 148,735 | 18,203 |
32
+ | knuckles/ams | 30ms | 60ms | 20ms | 19,603 | 136 |
33
+
34
+ These are measurements for a sizable payload with hundreds of associated
35
+ records.
36
+
37
+ [perforated]: https://github.com/sorentwo/perforated
38
+
16
39
  ## Installation
17
40
 
18
41
  Add this line to your application's Gemfile:
@@ -46,28 +69,32 @@ end
46
69
  With the top level module configured it is simple to jump right into rendering,
47
70
  but we'll look at configuring the pipeline first.
48
71
 
49
- ## Defining Views
72
+ ## Defining Views for Rendering
50
73
 
51
- The interface for defining serializers is largely based on Active Model
52
- Serializers, but with a few key differences.
74
+ While you can use Knuckles with other serializers, you can also use the provided
75
+ view layer. Knuckles views are simple templates that let you build up data and
76
+ relations. They look like this:
53
77
 
54
78
  ```ruby
55
- class ScoutSerializer
56
- include Knuckles::Serializer
57
-
58
- root 'scout'
79
+ module ScoutView
80
+ extend Knuckles::View
59
81
 
60
- attributes :id, :email, :display_name
82
+ def self.root
83
+ :scouts
84
+ end
61
85
 
62
- has_one :thing, serializer: ThingSerializer
63
- has_many :other, serializer: OtherThingSerializer
86
+ def self.data(object, options)
87
+ {id: object.id, email: object.email, name: object.name}
88
+ end
64
89
 
65
- def display_name(object, options)
66
- "#{object.first_name} #{object.last_name}"
90
+ def self.relations(object, options)
91
+ {things: has_many(object.things, ThingView)}
67
92
  end
68
93
  end
69
94
  ```
70
95
 
96
+ See `Knuckles::View` for more usage details.
97
+
71
98
  ## Contributing
72
99
 
73
100
  1. Fork it ( https://github.com/sorentwo/knuckles/fork )
@@ -1,9 +1,31 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Knuckles
4
+ # The Keygen module provides simple cache key generation. The global keygen
5
+ # strategy can be changed at the top level by configuring `Knuckles.keygen`.
6
+ #
7
+ # Any object that responds to `expand_key` with a single argument can be
8
+ # used instead.
9
+ #
10
+ # @example Change global keygen stragey
11
+ #
12
+ # Knuckles.keygen = MyCustomKeygen.new
13
+ #
4
14
  module Keygen
5
15
  extend self
6
16
 
17
+ # Calculates a cache key for the given object. It first attempts to use
18
+ # the object's `cache_key` method (present on `ActiveRecord` models). It
19
+ # falls back to combining the object's `id` and `updated_at` values.
20
+ #
21
+ # @param [Object] object An object to caclculate the key from
22
+ #
23
+ # @return [String] Computed cache key
24
+ #
25
+ # @example
26
+ #
27
+ # Knuckles::Keygen.expand_key(model) #=> "MyModel/1/1234567890"
28
+ #
7
29
  def expand_key(object)
8
30
  if object.respond_to?(:cache_key)
9
31
  object.cache_key
@@ -1,39 +1,81 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Knuckles
4
+ # The `Pipeline` is used to transform a collection of objects into a
5
+ # serialized result. A pipeline reduces a collection of objects through a set
6
+ # of stages, passing the result of each stage on to the next. This means that
7
+ # each stage can act in isolation and can compose with new custom stages.
8
+ # Additionally, stages are idividually instrumented for performance
9
+ # monitoring.
10
+ #
11
+ # The `Pipeline` class provides a set of default stages but can be overridden
12
+ # on initialization. Note that after initialization, the stages are frozen to
13
+ # prevent unpredictable mutation.
4
14
  class Pipeline
5
- def self.default_stages
6
- [Knuckles::Fetcher,
7
- Knuckles::Hydrator,
8
- Knuckles::Renderer,
9
- Knuckles::Writer,
10
- Knuckles::Enhancer,
11
- Knuckles::Combiner,
12
- Knuckles::Dumper]
13
- end
14
-
15
15
  attr_reader :stages
16
16
 
17
- def initialize(stages: self.class.default_stages)
18
- @stages = stages
19
- end
20
-
21
- def delete(stage)
22
- stages.delete(stage)
17
+ # Creates a new instance of `Knuckles::Pipeline`, optionally with custom
18
+ # stages.
19
+ #
20
+ # @option [Array] :stages (default_stages) An array of stages to pipe
21
+ # results through
22
+ #
23
+ # @example Create a default pipeline
24
+ #
25
+ # Knuckles::Pipeline.new
26
+ #
27
+ # @example Create a customized pipeline without any caching
28
+ #
29
+ # Knuckles::Pipeline.new(stages: [
30
+ # Knuckles::Stages::Renderer,
31
+ # Knuckles::Stages::Combiner,
32
+ # Knuckles::Stages::Dumper
33
+ # ]
34
+ def initialize(stages: default_stages)
35
+ @stages = stages.freeze
23
36
  end
24
37
 
25
- def insert_after(stage, new_stage)
26
- index = stages.index(stage)
27
-
28
- stages.insert(index + 1, new_stage)
29
- end
30
-
31
- def insert_before(stage, new_stage)
32
- index = stages.index(stage)
33
-
34
- stages.insert(index, new_stage)
38
+ # Provides default stage modules in the intended order. These are the
39
+ # stages that are used if nothing is passed during initialization. The
40
+ # defaults are defined as a method to make overriding with a subclass easy.
41
+ #
42
+ # @example Override `default_stages` within a subclass
43
+ #
44
+ # class CustomPipeline < Knuckles::Pipeline
45
+ # def default_stages
46
+ # [Knuckles::Stages::Renderer,
47
+ # Knuckles::Stages::Combiner,
48
+ # Knuckles::Stages::Dumper]
49
+ # end
50
+ # end
51
+ #
52
+ def default_stages
53
+ [Knuckles::Stages::Fetcher,
54
+ Knuckles::Stages::Hydrator,
55
+ Knuckles::Stages::Renderer,
56
+ Knuckles::Stages::Writer,
57
+ Knuckles::Stages::Enhancer,
58
+ Knuckles::Stages::Combiner,
59
+ Knuckles::Stages::Dumper]
35
60
  end
36
61
 
62
+ # Push a collection of objects through the stages of the pipeline. In
63
+ # normal usage this will render the objects out to a JSON structure.
64
+ #
65
+ # @param [Enumerable] objects A collection of objects (models) to be
66
+ # serialized and processed.
67
+ # @param [Hash] options The `call` method doesn't use any options itself,
68
+ # they are forwarded on to each stage. See the documentation for specific
69
+ # stages for the options they accept.
70
+ #
71
+ # @return [String] The final result as transformed by the pipeline,
72
+ # typically a JSON string.
73
+ #
74
+ # @example Basic pipeline rendering
75
+ #
76
+ # pipeline = Knuckles::Pipeline.new
77
+ # pipeline.call([tag_a, tag_b]) #=> '{"tags":[{"id":1},{"id":2}]}'
78
+ #
37
79
  def call(objects, options)
38
80
  prepared = prepare(objects)
39
81
 
@@ -44,6 +86,20 @@ module Knuckles
44
86
  end
45
87
  end
46
88
 
89
+ # Convert a collection of objects into a collection of hashes that enclose
90
+ # the object. The resulting hashes are populated with the keys and default
91
+ # values necessary for use with the standard pipeline stages.
92
+ #
93
+ # @param [Enumerable] objects A collection of objects to prepare
94
+ #
95
+ # @return [Array[Hash]] An array of hashes, each with the keys `:object`,
96
+ # `:key`, `:cached?`, and `:result`.
97
+ #
98
+ # @example Prepare a single object
99
+ #
100
+ # pipeline.prepare([model]) #=> [{object: model, key: nil,
101
+ # cached?: false, result: nil}]
102
+ #
47
103
  def prepare(objects)
48
104
  objects.map do |object|
49
105
  {object: object, key: nil, cached?: false, result: nil}
@@ -0,0 +1,24 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Knuckles
4
+ module Stages
5
+ module Combiner
6
+ extend self
7
+
8
+ def call(prepared, _)
9
+ prepared.each_with_object(array_backed_hash) do |hash, memo|
10
+ hash[:result].each do |root, values|
11
+ case values
12
+ when Hash then memo[root.to_s] << values
13
+ when Array then memo[root.to_s] += values
14
+ end
15
+ end
16
+ end
17
+ end
18
+
19
+ def array_backed_hash
20
+ Hash.new { |hash, key| hash[key] = [] }
21
+ end
22
+ end
23
+ end
24
+ end
@@ -0,0 +1,23 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Knuckles
4
+ module Stages
5
+ module Dumper
6
+ extend self
7
+
8
+ def call(objects, _options)
9
+ Knuckles.serializer.dump(keys_to_arrays(objects))
10
+ end
11
+
12
+ private
13
+
14
+ def keys_to_arrays(objects)
15
+ objects.each do |_, value|
16
+ if value.is_a?(Array)
17
+ value.uniq! { |hash| hash["id"] || hash[:id] }
18
+ end
19
+ end
20
+ end
21
+ end
22
+ end
23
+ end
@@ -0,0 +1,21 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Knuckles
4
+ module Stages
5
+ module Enhancer
6
+ extend self
7
+
8
+ def call(prepared, options)
9
+ enhancer = options[:enhancer]
10
+
11
+ if enhancer
12
+ prepared.each do |hash|
13
+ hash[:result] = enhancer.call(hash[:result], options)
14
+ end
15
+ end
16
+
17
+ prepared
18
+ end
19
+ end
20
+ end
21
+ end
@@ -0,0 +1,30 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Knuckles
4
+ module Stages
5
+ module Fetcher
6
+ extend self
7
+
8
+ def call(prepared, options)
9
+ results = get_cached(prepared, options)
10
+
11
+ prepared.each do |hash|
12
+ result = results[hash[:key]]
13
+ hash[:cached?] = !result.nil?
14
+ hash[:result] = result
15
+ end
16
+ end
17
+
18
+ private
19
+
20
+ def get_cached(prepared, options)
21
+ kgen = options.fetch(:keygen, Knuckles.keygen)
22
+ keys = prepared.map do |hash|
23
+ hash[:key] = kgen.expand_key(hash[:object])
24
+ end
25
+
26
+ Knuckles.cache.read_multi(*keys)
27
+ end
28
+ end
29
+ end
30
+ end
@@ -0,0 +1,29 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Knuckles
4
+ module Stages
5
+ module Hydrator
6
+ extend self
7
+
8
+ def call(prepared, options)
9
+ hydrate = options[:hydrate]
10
+
11
+ if hydrate && any_missing?(prepared)
12
+ hydrate.call(hydratable(prepared))
13
+ end
14
+
15
+ prepared
16
+ end
17
+
18
+ private
19
+
20
+ def any_missing?(prepared)
21
+ prepared.any? { |hash| !hash[:cached?] }
22
+ end
23
+
24
+ def hydratable(prepared)
25
+ prepared.reject { |hash| hash[:cached?] }
26
+ end
27
+ end
28
+ end
29
+ end
@@ -0,0 +1,27 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Knuckles
4
+ module Stages
5
+ module Renderer
6
+ extend self
7
+
8
+ def call(objects, options)
9
+ view = options.fetch(:view)
10
+
11
+ objects.each do |hash|
12
+ unless hash[:cached?]
13
+ hash[:result] = do_render(hash[:object], view, options)
14
+ end
15
+ end
16
+ end
17
+
18
+ private
19
+
20
+ def do_render(object, view, options)
21
+ view.relations(object, options).merge!(
22
+ view.root => [view.data(object, options)]
23
+ )
24
+ end
25
+ end
26
+ end
27
+ end
@@ -0,0 +1,41 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Knuckles
4
+ module Stages
5
+ module Writer
6
+ extend self
7
+
8
+ def call(objects, _)
9
+ if cache.respond_to?(:write_multi)
10
+ write_multi(objects)
11
+ else
12
+ write_each(objects)
13
+ end
14
+
15
+ objects
16
+ end
17
+
18
+ private
19
+
20
+ def cache
21
+ Knuckles.cache
22
+ end
23
+
24
+ def write_each(objects)
25
+ objects.each do |hash|
26
+ cache.write(hash[:key], hash[:result]) unless hash[:cached?]
27
+ end
28
+ end
29
+
30
+ def write_multi(objects)
31
+ writable = objects.each_with_object({}) do |hash, memo|
32
+ next if hash[:cached?]
33
+
34
+ memo[hash[:key]] = hash[:result]
35
+ end
36
+
37
+ cache.write_multi(writable) if writable.any?
38
+ end
39
+ end
40
+ end
41
+ end
@@ -0,0 +1,13 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Knuckles
4
+ module Stages
5
+ autoload :Combiner, "knuckles/stages/combiner"
6
+ autoload :Dumper, "knuckles/stages/dumper"
7
+ autoload :Enhancer, "knuckles/stages/enhancer"
8
+ autoload :Fetcher, "knuckles/stages/fetcher"
9
+ autoload :Hydrator, "knuckles/stages/hydrator"
10
+ autoload :Renderer, "knuckles/stages/renderer"
11
+ autoload :Writer, "knuckles/stages/writer"
12
+ end
13
+ end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Knuckles
4
- VERSION = "0.2.0".freeze
4
+ VERSION = "0.3.0".freeze
5
5
  end
data/lib/knuckles/view.rb CHANGED
@@ -1,28 +1,127 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Knuckles
4
+ # An absolutely bare bones serializer. It is meant as a replacement for
5
+ # `ActiveModelSerializers`, but is entirely focused on being simple and
6
+ # explicit. Views are templates that satisfy the interface of:
7
+ #
8
+ # * root #=> symbol
9
+ # * data #=> hash
10
+ # * relations #=> hash
11
+ #
12
+ # Any object that satisfies that interface can be rendered correctly by the
13
+ # `Renderer` stage.
14
+ #
15
+ # @example Extending for a custom view
16
+ #
17
+ # module TagView
18
+ # extend Knuckles::View
19
+ #
20
+ # def root
21
+ # :tags
22
+ # end
23
+ #
24
+ # def data(tag, _)
25
+ # {id: tag.id, name: tag.name}
26
+ # end
27
+ #
28
+ # def relations(tag, _)
29
+ # {posts: has_many(tag.posts, PostView)}
30
+ # end
31
+ # end
32
+ #
4
33
  module View
5
34
  extend self
6
35
 
7
- ## Callbacks
8
-
36
+ # Specifies the top level key that data will be stored under. The value
37
+ # will not be stringified or pluralized during rendering, so be aware of
38
+ # the format.
39
+ #
40
+ # @return [Symbol, nil] By default root is `nil`, it should be overridden to
41
+ # return a plural symbol.
42
+ #
9
43
  def root
10
44
  end
11
45
 
46
+ # Serialize an object into a hash. This simply returns an empty hash by
47
+ # default, it must be overridden by submodules.
48
+ #
49
+ # @param [Object] _object The object for serializing.
50
+ # @param [Hash] _options The options to be used during serialization, i.e.
51
+ # `:scope`
52
+ #
53
+ # @return [Hash] A hash representing the serialized object.
54
+ #
55
+ # @example Overriding data
56
+ #
57
+ # module TagView
58
+ # extend Knuckles::View
59
+ #
60
+ # def data(tag, _)
61
+ # {id: tag.id, name: tag.name}
62
+ # end
63
+ # end
64
+ #
12
65
  def data(_object, _options = {})
13
66
  {}
14
67
  end
15
68
 
69
+ # Extracts associations for an object. Later these are merged with the
70
+ # output of `data`. View relations are shallow, meaning the relations of
71
+ # relations are not included.
72
+ #
73
+ # @param [Object] _object The object to extract relations from
74
+ # @param [Hash] _options The options used during extraction, i.e. `:scope`
75
+ #
76
+ # @return [Hash] The serialized associations
77
+ #
78
+ # @example Override relations
79
+ #
80
+ # module PostView
81
+ # extend Knuckles::View
82
+ #
83
+ # def relations(post, _)
84
+ # {author: has_one(post.author, AuthorView),
85
+ # comments: has_many(post.comments, CommentView)}
86
+ # end
87
+ # end
88
+ #
16
89
  def relations(_object, _options = {})
17
90
  {}
18
91
  end
19
92
 
20
- ## Relations
21
-
93
+ # Renders an associated object using the specified view, wrapping the
94
+ # results in an array.
95
+ #
96
+ # @param [Object] object The associated object to serialize.
97
+ # @param [Module] view A module responding to `data`.
98
+ # @param [Hash] options Passed to the view for rendering.
99
+ #
100
+ # @return [Array<Hash>] A single rendered data object is always returned.
101
+ #
102
+ # @example Render a single association
103
+ #
104
+ # PostView.has_one(post.author, AuthorView) #=> [{id: 1, name: "Author"}]
105
+ #
22
106
  def has_one(object, view, options = {})
23
- [view.data(object, options)]
107
+ has_many([object], view, options)
24
108
  end
25
109
 
110
+ # Renders all associated objects using the specified view.
111
+ #
112
+ # @param [Array] objects Array of associated objects to serialize.
113
+ # @param [Module] view A module responding to `data`.
114
+ # @param [Hash] options The options passed to the view for rendering.
115
+ #
116
+ # @return [Array<Hash>] All rendered association data.
117
+ #
118
+ # @example Render a single association
119
+ #
120
+ # PostView.has_one(post.authors, AuthorView) #=> [
121
+ # {id: 1, name: "Me"},
122
+ # {id: 2, name: "You"}
123
+ # ]
124
+ #
26
125
  def has_many(objects, view, options = {})
27
126
  objects.map { |object| view.data(object, options) }
28
127
  end