mm_partial_update 0.1.0

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2010 Nathan Stults
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,71 @@
+= mm-partial-update
+
+mm-partial-update is a MongoMapper plugin that enables a change-only persistence strategy for MongoMapper. mm-partial-update uses the enhanced dirty tracking provided by mm-dirtier to send only changes to MognoDB, rather than the entire document on each save. *note: mm-partial update requires the rails3 branch of MongoMapper. It will not work with the master branch, the one you get when you 'gem install mongo_mapper'
+
+== Installation
+
+mm-partial-update is available as a RubyGem:
+
+     gem install mm_partial_update
+
+To activate the plugin, add 'mm_partial_update' to your gemfile
+
+   gem 'mm_partial_update'
+
+== Usage
+
+mm-partial-update does not, by default, change MongoMappers normal mode of operation. Although mm-partial-update can be configured, either at a global level or on a model by model basis, to use a partial update strategy for all persistence operations, out of the box it simply adds a "save_changes" method to both MongoMapper::Document and MongoMapper::EmbeddedDocument, which will persist, using MongoDB's atomic operators, any changes to the target document or embedded document as well as any descendents of the target document or embedded document.
+
+=== For example:
+
+     class Person
+       include MongoMapper::Document
+       key :name, String
+       many :pets
+     end
+
+     class Pet
+       include MongoMapper::EmbeddedDocument
+       key :name, String
+     end
+
+     person = Person.create! :name=>"Willard"
+     person.name = "Poe"
+     person.save! #as always, overwrites the document in the database with the in memory copy
+
+     person = Person.create! :name=>"Willard"
+     person.name = "Poe"
+     person.save_changes #only saves the changed fields (in this case name=>"Poe"
+
+However, in addition to #save_changes, partial saves can be enabled globally:
+
+     MmPartialUpdate.persistence_strategy = :changes_only
+
+Or on a model by model basis:
+
+     class Person
+       include MongoMapper::Document
+       persistence_strategy :changes_only
+       key :name, String
+       many :pets
+     end
+
+When enabled globally, all calls to save or save! will result in a partial update (with the exception of new documents). When enabled on a particular model, calls to save or save! on instances of that model and any subclasses of that model will result in partial updates. This is useful if you have a number of models but only one or two require partial saves for a particular use case, such as concurrent modification by more than one process.
+
+== Known Issues & Limitations
+
+* mm_partial_update will persist fully embedded documents on a change_only basis (i.e. issue $push, $pull and $set commands as appropriate). However, for the time being in_array_proxies behave like embedded Array keys, and persist as they always have in an all or nothing fashion. This will be changed in a future version.
+* mm_partial_update will detect and persist changes to embedded Array and Hash keys, but only at a single level of depth. This means that modifying an array or hash contained within your embedded array or hash key will not trigger the dirty tracking mechanism (i.e. my_doc.tags.meta["happy"] = 5 would not cause my_doc.tags to appear changed). This is a very solvable problem, and a fix will likely appear in a future version, particularly if someone needs it (I myself do not).
+* mm-partial-update does not (perhaps cannot?) behave in a truly atomic fashion. Because MongoDB appears to take a shotgun approach to detecting and preventing 'conflicting updates' in a single command, mm-partial-update breaks a single set of changes into multiple isolated commands to the database. Currently, the algorithm used is not very sophisticated, and results in a single update command for each unique embedded array that requires a push or a pull. In the future if this ends up presenting a performance problem, more sophisticated (but complicated) algorithms can be implemented to intelligently batch non-conflicting pushes and pulls together, resulting in a minimum of database calls. I'm holding off on that optimization until it proves necessary, however, mostly for maintainability reasons.
+
+== Note on Patches/Pull Requests
+
+* Fork the project.
+* Make your feature addition or bug fix.
+* Add tests for it. This is important so I don't break it in a future version unintentionally.
+* Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
+* Send me a pull request. Bonus points for topic branches.
+
+== Copyright
+
+Copyright (c) 2089 Nathan Stults. See LICENSE for details.

data/lib/mm_partial_update.rb

@@ -0,0 +1,32 @@
+require 'rubygems'
+require "mongo_mapper"
+require "observables"
+require "mm_dirtier"
+
+base_dir = File.dirname(__FILE__)
+[
+ 'version',
+ 'embedded_collection',
+ 'one_embedded_proxy',
+ 'extensions',
+ 'update_command',
+ 'plugins/partial_update',
+ 'plugins/document',
+ 'plugins/embedded_document'
+].each {|req| require File.join(base_dir,'mm_partial_update',req)}
+
+MongoMapper::Document.append_inclusions(MmPartialUpdate::Plugins::PartialUpdate)
+MongoMapper::Document.append_inclusions(MmPartialUpdate::Plugins::PartialUpdate::Document)
+
+MongoMapper::EmbeddedDocument.append_inclusions(MmPartialUpdate::Plugins::PartialUpdate)
+MongoMapper::EmbeddedDocument.append_inclusions(MmPartialUpdate::Plugins::PartialUpdate::EmbeddedDocument)
+
+module MmPartialUpdate
+  def self.default_persistence_strategy
+    @default_persistence_strategy ||= :full_document
+  end
+
+  def self.default_persistence_strategy=(strategy)
+    @default_persistence_strategy = strategy
+  end
+end

data/lib/mm_partial_update/embedded_collection.rb

@@ -0,0 +1,47 @@
+module MmPartialUpdate
+  module EmbeddedCollection
+
+    def save_to_collection(options)
+      super.tap { assign_database_indexes }
+    end
+
+    def add_updates_to_command(changes, command)
+      selector = association.name
+      selector = "#{proxy_owner.database_selector}.#{selector}" if
+        proxy_owner.respond_to?(:database_selector)
+
+      unless changes.blank?
+        deleted = changes[0] - changes[1]
+        deleted.each { |d| command.pull(selector, d._id) }
+      end
+
+      unless @target.blank?
+        @target.each do |child|
+          child.new? ? command.push(selector, child.to_mongo) :
+            child.add_updates_to_command(command)
+        end
+      end
+
+    end
+
+    private
+
+    def find_target
+      super.tap { |docs| assign_database_indexes(docs) }
+    end
+
+    def assign_database_indexes(docs = nil)
+      docs ||= @target
+      docs.each_with_index do |value, index|
+        value.instance_variable_set("@_database_position",index)
+      end if docs
+    end
+
+    def assign_references(*docs)
+      docs.each { |doc| doc.instance_variable_set("@_association_name", association.name) }
+      super(*docs)
+    end
+
+
+  end
+end

data/lib/mm_partial_update/extensions.rb

@@ -0,0 +1,37 @@
+class Object
+  def can_be_persistable?
+    respond_to?(:make_persistable)
+  end
+end
+
+
+module MongoMapper
+  module Plugins
+    module Associations
+
+      class EmbeddedCollection
+        def persistable?
+          kind_of?(MmPartialUpdate::EmbeddedCollection)
+        end
+
+        def make_persistable
+          class << self; include MmPartialUpdate::EmbeddedCollection; end unless persistable?
+        end
+
+      end
+
+      class OneEmbeddedProxy
+        def persistable?
+          kind_of?(MmPartialUpdate::OneEmbeddedProxy)
+        end
+
+        def make_persistable
+          class << self; include MmPartialUpdate::OneEmbeddedProxy; end unless persistable?
+        end
+
+      end
+
+
+    end
+  end
+end

data/lib/mm_partial_update/one_embedded_proxy.rb

@@ -0,0 +1,22 @@
+module MmPartialUpdate
+  module OneEmbeddedProxy
+
+    def add_updates_to_command(changes, command)
+      selector = association.name
+      selector = "#{proxy_owner.database_selector}.#{selector}" if
+        proxy_owner.respond_to?(:database_selector)
+
+      if @target.nil?
+        command.unset(selector, :nullify=>true) unless changes.blank?
+      else
+        @target.add_updates_to_command(command)
+      end
+    end
+
+    def assign_references(doc)
+      doc.instance_variable_set("@_association_name",association.name)
+      super(doc)
+    end
+
+  end
+end

data/lib/mm_partial_update/plugins/document.rb

@@ -0,0 +1,44 @@
+# encoding: UTF-8
+module MmPartialUpdate
+  module Plugins
+    module PartialUpdate
+      module Document
+
+        def self.included(model)
+          model.plugin MmPartialUpdate::Plugins::PartialUpdate::Document
+        end
+
+        module InstanceMethods
+
+          def save_to_collection(options={})
+            strategy = determine_persistence_strategy(options)
+            return super if new? || strategy == :full_document
+
+            save_changes(options)
+          end
+
+          private
+
+          def assert_valid_persistence_strategy(strategy)
+            raise "Invalid persistence strategy (#{strategy}). Valid options are :full_document or :changes_only" unless ["full_document", "changes_only"].include?(strategy.to_s)
+          end
+
+          def determine_persistence_strategy(options)
+            strategy =  options[:changes_only] ? :changes_only :
+              options[:persistence_strategy] ||
+              self.class.persistence_strategy ||
+              MmPartialUpdate.default_persistence_strategy ||
+              :full_document
+
+            strategy.tap { assert_valid_persistence_strategy(strategy) }
+          end
+
+        end
+
+
+      end
+    end
+  end
+end
+
+

data/lib/mm_partial_update/plugins/embedded_document.rb

@@ -0,0 +1,43 @@
+# encoding: UTF-8
+module MmPartialUpdate
+  module Plugins
+    module PartialUpdate
+      module EmbeddedDocument
+
+        def self.included(model)
+          model.plugin MmPartialUpdate::Plugins::PartialUpdate::EmbeddedDocument
+        end
+
+        module InstanceMethods
+
+          def database_selector
+            selector = @_association_name.to_s
+            selector = "#{_parent_document.database_selector}.#{selector}" if
+              _parent_document.respond_to?(:database_selector)
+            selector = "#{selector}.#{@_database_position}" if defined?(@_database_position)
+            selector
+          end
+
+          private
+
+          def add_create_self_to_command(selector, command)
+            if _parent_document.new?
+              _parent_document.add_updates_to_command(command)
+            else
+
+              association = _parent_document.associations[@_association_name]
+              if association && association.many?
+                command.push(selector, self.to_mongo)
+              else
+                command.set(selector, self.to_mongo, :replace=>true)
+              end
+            end
+          end
+
+        end
+
+      end
+    end
+  end
+end
+

data/lib/mm_partial_update/plugins/partial_update.rb

@@ -0,0 +1,93 @@
+# encoding: UTF-8
+module MmPartialUpdate
+  module Plugins
+    module PartialUpdate
+
+      def self.included(model)
+        model.plugin MmDirtier::Plugins::Dirtier unless
+          model.plugins.include?(MmDirtier::Plugins::Dirtier)
+        model.plugin(MmPartialUpdate::Plugins::PartialUpdate)
+      end
+
+      module ClassMethods
+
+        def inherited(descendant)
+          descendant.instance_variable_set("@_persistence_strategy",
+                                           self.persistence_strategy)
+          super
+        end
+
+        def persistence_strategy(new_strategy=nil)
+          return @_persistence_strategy ||= nil unless new_strategy
+          @_persistence_strategy = new_strategy
+        end
+
+      end
+
+      module InstanceMethods
+
+        def save_changes(options={})
+          #We can't update an embedded document if the root isn't saved
+          #The clear_changes call is added here because dirty
+          #tracking happens further up the call chain than save_to_collection
+          #under normal circumstances, so we have to inject it
+          return _root_document.save_to_collection(options).tap {clear_changes} if
+            _root_document.new?
+
+          #persist changes to self and descendents
+          update_command  = prepare_update_command
+          update_command.execute()
+
+          #clear dirty tracking
+          @_new = false
+          clear_changes
+          associations.each do |_, association|
+            proxy = get_proxy(association)
+            proxy.save_to_collection(options) if
+              proxy.proxy_respond_to?(:save_to_collection)
+          end
+        end
+
+        def prepare_update_command
+          UpdateCommand.new(self).tap { |command| add_updates_to_command(command) }
+        end
+
+        def add_updates_to_command(command)
+
+          selector = respond_to?(:database_selector) ? database_selector : nil
+
+          add_create_self_to_command(selector, command) and return if new?
+
+          field_changes = changes
+
+          associations.values.each do |association|
+            proxy = get_proxy(association)
+            association_changes = field_changes.delete(association.name)
+            proxy.add_updates_to_command(association_changes, command) if
+              proxy.respond_to?(:add_updates_to_command)
+          end
+
+          field_changes = field_changes.inject({}) do |changes,change|
+            changes[change[0]] = change[-1][-1]
+            changes
+          end
+          command.tap {|c|c.set(selector,field_changes)}
+        end
+
+        private
+
+        def add_create_self_to_command(selector, command)
+          command.tap { |c| c.set(selector, self.to_mongo, :replace=>true)}
+        end
+
+        def get_proxy(association)
+          proxy = super(association)
+          proxy.make_persistable if proxy.can_be_persistable?
+          proxy
+        end
+
+      end
+
+    end
+  end
+end

data/lib/mm_partial_update/update_command.rb

@@ -0,0 +1,114 @@
+module MmPartialUpdate
+  class UpdateCommand
+    attr_reader :target_document, :root_document, :commands
+
+    def initialize(target_document)
+      @target_document = target_document
+      @root_document = target_document.send(:_root_document)
+      @commands = BSON::OrderedHash.new { |hash,key| hash[key] = BSON::OrderedHash.new }
+    end
+
+    def document_selector
+      {:_id=>root_document._id}
+    end
+
+    def set(selector, fields, options={})
+
+      return if fields.blank?
+
+      selector = selector.to_s if selector
+
+      if selector.blank? && options[:replace]
+        commands["$set"] = fields
+      elsif selector.blank?
+        commands["$set"].merge!(fields)
+      elsif options[:replace]
+        commands["$set"][selector] = fields
+      else
+        commands["$set"].merge!(fields.keys.inject(BSON::OrderedHash.new) do |hash,field_name|
+                                  hash["#{selector}.#{field_name}"] = fields[field_name]
+                                  hash
+                                end)
+      end
+    end
+
+    def unset(selector, options={})
+      raise "'unset' requires a non-blank selector" if selector.blank?
+      selector = selector.to_s
+      options[:nullify] ? commands["$set"][selector] = nil : commands["$unset"][selector] = true
+    end
+
+    def push(selector, document)
+      raise "'push' requires a non-blank selector" if selector.blank?
+      selector = selector.to_s
+      (commands[:pushes][selector] ||= []) << document
+    end
+
+    def pull(selector, document_id)
+      raise "'pull' requires a non-blank selector" if selector.blank?
+      selector = selector.to_s
+      (commands[:pulls][selector] ||= []) << document_id
+    end
+
+    def merge(other_command)
+      commands.merge! other_command.to_h
+    end
+
+    def to_h
+      commands
+    end
+
+    def empty?
+      commands.blank?
+    end
+
+    def reset
+      commands.clear
+    end
+
+    def execute(options={})
+       #if there are no commands, there is nothing to do...
+      return if empty?
+
+      selector = document_selector.tap {|s|s.merge!("$atomic"=>true) if options[:atomic]}
+
+      dbcommands = prepare_mongodb_commands
+
+      dbcommands.each do |command|
+        root_document.collection.update(selector, command, :multi=>false,
+                                        :upsert=>true, :safe=>options[:safe])
+      end
+    end
+
+    private
+
+    def prepare_mongodb_commands
+      dbcommands = []
+
+      initial_command = commands.dup
+      pushes, pulls = initial_command.delete(:pushes), initial_command.delete(:pulls)
+
+      dbcommands << initial_command unless initial_command.blank?
+
+      while (next_op = next_pull(pulls)); dbcommands << next_op; end
+      while (next_op = next_push(pushes)); dbcommands << next_op; end
+
+      dbcommands
+    end
+
+    def next_push(pushes)
+      return nil if pushes.blank?
+      selector = pushes.keys[0]
+      docs = pushes.delete(selector)
+      {"$pushAll" => { selector => docs } }
+    end
+
+    def next_pull(pulls)
+      return nil if pulls.blank?
+      selector = pulls.keys[0]
+      doc_ids = pulls.delete(selector)
+      {"$pull" => { selector => { "_id" => { "$in" => doc_ids } } } }
+    end
+
+  end
+end