delta 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: b03dae554a7f32d4fc3826fd35aa36f82549c2b1
+  data.tar.gz: 9b0bf35d479c98b32d2eacc38c4f71128464a2ea
+SHA512:
+  metadata.gz: 55edc3a5987c896b0c296db1171a39bf80a24ada940b7bdaa59a5d5ba5464971f4ccbccfc206bd0e4bd5e330684b8c96d21124667eaf812e4bae0ce71eabb4d7
+  data.tar.gz: ad09180100f9018bd3feee520ae5343512d7a174efc6f633bac594650ed3d15af965196321b1cc04efdc3b1bf28739210c6b4323faed9ccb6d507f261eaf69cc

data/README.md

@@ -0,0 +1,190 @@
+## Delta
+
+[![Build Status](https://travis-ci.org/tuzz/delta.svg?branch=master)](https://travis-ci.org/tuzz/delta)
+
+Calculates the delta between two collections of objects.
+
+## Usage
+
+In its simplest form, Delta calculates the additions and deletions for arbitrary
+collections of objects:
+
+```ruby
+delta = Delta.new(
+  from: [pikachu, pidgey, magikarp],
+  to: [raichu, pidgey, butterfree]
+)
+
+delta.additions
+#=> #<Enumerator: ... >
+
+delta.additions.to_a
+#=> [
+  #<Pokemon @species="Raichu", @name="Zappy", @type="Electric">,
+  #<Pokemon @species="Butterfree", @name="Flappy", @type="Flying">
+]
+
+delta.deletions.to_a
+#=> [
+  #<Pokemon @species="Pikachu", @name="Zappy", @type="Electric">,
+  #<Pokemon @species="Magikarp", @name="Splashy", @type="Water">
+]
+```
+
+## Pluck
+
+Sometimes, you'll only be interested in a few of the attributes on an object.
+Delta supports `pluck`, which selects only the attributes you're interested in,
+instead of returning full objects:
+
+```ruby
+delta = Delta.new(
+  from: [pikachu, pidgey, magikarp],
+  to: [raichu, pidgey, butterfree],
+  pluck: [:name, :species]
+)
+
+delta.additions.to_a
+#=> [
+  #<struct name="Zappy", species="Pikachu">,
+  #<struct name="Flappy", species="Butterfree">
+]
+
+delta.deletions.to_a
+#=> [
+  #<struct name="Zappy", species="Raichu">,
+  #<struct name="Splashy", species="Magikarp">
+]
+```
+
+## Modifications
+
+In most cases, it is more appropriate to think in terms of modifications rather
+than additions and deletions. In the example above, 'Pikachu' appeared as a
+deletion and 'Raichu' appeared as an addition. It might make more sense to model
+this as a modification, i.e. 'Zappy' changing its 'species'.
+
+Delta supports setting the `keys` that uniquely identify each object in the
+collection:
+
+```ruby
+delta = Delta.new(
+  from: [pikachu, pidgey, magikarp],
+  to: [raichu, pidgey, butterfree],
+  pluck: [:species, :name],
+  keys: [:name]
+)
+
+delta.additions.to_a
+#=> [#<struct species="Butterfree", name="Flappy">]
+
+delta.modifications.to_a
+#=> [#<struct species="Raichu", name="Zappy">]
+
+delta.deletions.to_a
+#=> [#<struct species="Magikarp", name="Splashy">]
+```
+
+You should specify which attributes to `pluck` so that Delta can distinguish
+between objects that have changed and objects that have not changed, but appear
+in both collections.
+
+If you do not specifiy any attributes to `pluck`, Delta will fall back to using
+object equality to determine which objects have changed. For ActiveRecord
+relations, Delta will use the database id as its fall back as it is more
+performant to do so.
+
+## Composite Keys
+
+In some cases, objects will be uniquely identified by a combination of things.
+For example, consider an application that enforces the uniqueness of names, but
+only in the context of a 'type'. This would mean that two Pokemon can have the
+same name, as long as they are not of the same type (e.g. 'Water').
+
+In these cases, you can specify multiple keys:
+
+```ruby
+delta = Delta.new(
+  from: [pikachu, pidgey, magikarp],
+  to: [raichu, pidgey, butterfree],
+  pluck: [:species, :name, :type],
+  keys: [:name, :type] # <--
+)
+
+delta.additions.to_a
+#=> [#<struct species="Butterfree", name="FANG!", type="Flying">]
+
+delta.modifications.to_a
+#=> [#<struct species="Raichu", name="Zappy", type="Electric">]
+
+delta.deletions.to_a
+#=> [#<struct species="Magikarp", name="FANG!", type="Water">]
+```
+
+Consider the alternative where 'name' is used as the only key. This would mean
+that the Pokemon with species' 'Butterfree' and 'Magikarp' would be considered
+the same. This is semantically incorrect for this particular domain.
+
+TODO: What would happen if you did that? Should it raise an error?
+
+## Many-to-one Deltas
+
+By combining the use of `pluck` and `keys`, you can build deltas that aren't
+necessarily related to a single object, but instead span multiple objects within
+the collection.
+
+This may be useful if there are objects in your collections that share some
+property. In this example, all of the Pokemon have a 'type'. We can build a
+Delta that shows the difference in types between collections, like so:
+
+```ruby
+delta = Delta.new(
+  from: [pikachu, pidgey, magikarp],
+  to: [raichu, pidgey, butterfree],
+  keys: [:type],
+  pluck: [:type]
+)
+
+delta.additions.to_a
+#=> []
+
+delta.modifications.to_a
+#=> []
+
+delta.deletions.to_a
+#=> [#<struct type="Water">]
+```
+
+In this example, both 'Electric' and 'Flying' types appear in both collections.
+The only difference is the deletion of all 'Water' type Pokemon as a result of
+removing 'Magikarp' from the first collection.
+
+## Rails Support
+
+Delta has added support for Rails applications. If the given collections are of
+class ActiveRecord::Relation, Delta will try to improve performance by reducing
+the number of select queries on the database. This isn't perfect and may not
+work with old versions of Rails.
+
+## In the Wild
+
+So far, we've talked a lot about Pokemon, but how is this useful in the
+real-world?
+
+Consider an application that is backed by an external content management system.
+You have some kind of extract-transform-load process that takes content from the
+CMS and pushes it into your database. Your application is serving live traffic
+and needs to remain available at all times.
+
+To minimise the amount of churn on your database, it makes sense to calculate
+just the things that have changed, rather than rebuilding the entire set of
+content each time. To do this, you could push the CMS content into a staging
+database where you calculate a Delta without touching your live application.
+
+Once this Delta is calculated, you can then stream the minimal set of changes
+to the database that backs your application.
+
+## Contribution
+
+Thanks for the using this gem. If you think of a great new feature or you find a
+problem, please open an issue or a pull request and I'll try to get back to you.

data/lib/delta/base.rb

@@ -0,0 +1,38 @@
+class Delta
+  def initialize(from:, to:, pluck: nil, keys: nil)
+    identifier = keys ? Identifier.new(keys) : Identifier::Null.new
+    self.set = SetOperator.adapt(a: from, b: to, identifier: identifier)
+    self.plucker = pluck ? Plucker.new(pluck) : Plucker::Null.new
+  end
+
+  def additions
+    Enumerator.new do |y|
+      set.subtract_a_from_b.each do |b|
+        y.yield plucker.pluck(b)
+      end
+    end
+  end
+
+  def modifications
+    Enumerator.new do |y|
+      set.intersection.each do |a, b|
+        a_attributes = plucker.pluck(a)
+        b_attributes = plucker.pluck(b)
+
+        y.yield b_attributes unless a_attributes == b_attributes
+      end
+    end
+  end
+
+  def deletions
+    Enumerator.new do |y|
+      set.subtract_b_from_a.each do |a|
+        y.yield plucker.pluck(a)
+      end
+    end
+  end
+
+  private
+
+  attr_accessor :set, :plucker
+end

data/lib/delta/identifier.rb

@@ -0,0 +1,37 @@
+class Delta
+  class Identifier
+    def initialize(keys)
+      self.keys = keys
+    end
+
+    def identity(object)
+      cache(object) { Hash[keys.map { |k| [k, object.public_send(k)] }] }
+    end
+
+    def identities(collection)
+      cache(collection) { Hash[keys.map { |k| [k, collection.pluck(k).uniq] }] }
+    end
+
+    private
+
+    attr_accessor :keys
+
+    def cache(key)
+      @cache ||= {}
+      @cache.key?(key) ? @cache.fetch(key) : @cache[key] = yield
+    end
+
+    class Null < Identifier
+      def initialize
+      end
+
+      def identity(object)
+        object
+      end
+
+      def identities(collection)
+        cache(collection) { { id: collection.pluck(:id) } }
+      end
+    end
+  end
+end

data/lib/delta/plucker.rb

@@ -0,0 +1,23 @@
+class Delta
+  class Plucker
+    def initialize(pluck)
+      self.array = pluck
+      self.struct = Struct.new(*array)
+    end
+
+    def pluck(object)
+      attributes = array.map { |k| object.public_send(k) }
+      struct.new(*attributes)
+    end
+
+    private
+
+    attr_accessor :array, :struct
+
+    class Null
+      def pluck(object)
+        object
+      end
+    end
+  end
+end

data/lib/delta/set_operator/active_record.rb

@@ -0,0 +1,46 @@
+class Delta
+  class SetOperator
+    class ActiveRecord < SetOperator
+      def self.compatible?(a, b)
+        a.is_a?(b.class) && a.class.name.include?("ActiveRecord")
+      end
+
+      private
+
+      def subtract(a, b)
+        a.where.not(identifier.identities(b))
+      end
+
+      def intersect(a, b)
+        keys = identity_keys(a)
+
+        a_records = left_of_intersection(a, b)
+        a_records = unique_records(a_records, keys)
+
+        Enumerator.new do |y|
+          a_records.each do |record|
+            y.yield find_pair!(record, keys)
+          end
+        end
+      end
+
+      def identity_keys(collection)
+        identifier.identities(collection).keys
+      end
+
+      def left_of_intersection(a, b)
+        b_identities = identifier.identities(b)
+        a.where(b_identities)
+      end
+
+      def unique_records(collection, keys)
+        collection.select(*keys).distinct
+      end
+
+      def find_pair!(record, keys)
+        attributes = record.slice(*keys)
+        [a.find_by!(attributes), b.find_by!(attributes)]
+      end
+    end
+  end
+end

data/lib/delta/set_operator/enumerable.rb

@@ -0,0 +1,35 @@
+class Delta
+  class SetOperator
+    class Enumerable < SetOperator
+      def self.compatible?(_a, _b)
+        true
+      end
+
+      def initialize(a:, b:, identifier:)
+        super
+
+        self.a = a.lazy
+        self.b = b.lazy
+      end
+
+      private
+
+      def subtract(a, b)
+        a.reject { |a_object| other_object(b, a_object) }
+      end
+
+      def intersect(a, b)
+        a.map { |a_object| [a_object, other_object(b, a_object)] }
+          .select { |_, b_object| b_object }
+      end
+
+      def other_object(collection, object)
+        identity = identifier.identity(object)
+
+        collection.find do |other_object|
+          identity == identifier.identity(other_object)
+        end
+      end
+    end
+  end
+end

data/lib/delta/set_operator.rb

@@ -0,0 +1,46 @@
+class Delta
+  class SetOperator
+    ADAPTERS = [ActiveRecord, Enumerable]
+
+    def self.adapt(a:, b:, identifier:)
+      adapter = ADAPTERS.find { |klass| klass.compatible?(a, b) }
+      adapter = ADAPTERS.last unless adapter
+
+      adapter.new(a: a, b: b, identifier: identifier)
+    end
+
+    def initialize(a:, b:, identifier:)
+      self.a = a
+      self.b = b
+      self.identifier = identifier
+    end
+
+    def subtract_a_from_b
+      subtract(b, a)
+    end
+
+    def subtract_b_from_a
+      subtract(a, b)
+    end
+
+    def intersection
+      intersect(a, b)
+    end
+
+    protected
+
+    attr_accessor :a, :b, :identifier
+
+    private
+
+    # a - b
+    def subtract(_a, _b)
+      fail NotImplementedError, "override me"
+    end
+
+    # a & b
+    def intersect(_a, _b)
+      fail NotImplementedError, "override me"
+    end
+  end
+end

data/lib/delta.rb

@@ -0,0 +1,6 @@
+require "delta/base"
+require "delta/set_operator/enumerable"
+require "delta/set_operator/active_record"
+require "delta/set_operator"
+require "delta/plucker"
+require "delta/identifier"

metadata

@@ -0,0 +1,135 @@
+--- !ruby/object:Gem::Specification
+name: delta
+version: !ruby/object:Gem::Version
+  version: 1.0.0
+platform: ruby
+authors:
+- Chris Patuzzo
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2015-05-24 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.4'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.4'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.2'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.2'
+- !ruby/object:Gem::Dependency
+  name: rubocop
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.31'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.31'
+- !ruby/object:Gem::Dependency
+  name: pry
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.10'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.10'
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.2'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.2'
+- !ruby/object:Gem::Dependency
+  name: sqlite3
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
+description: Calculates the delta between two collections of objects.
+email: chris@patuzzo.co.uk
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- README.md
+- lib/delta.rb
+- lib/delta/base.rb
+- lib/delta/identifier.rb
+- lib/delta/plucker.rb
+- lib/delta/set_operator.rb
+- lib/delta/set_operator/active_record.rb
+- lib/delta/set_operator/enumerable.rb
+homepage: https://github.com/tuzz/delta
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.4.5
+signing_key: 
+specification_version: 4
+summary: Delta
+test_files: []