mm_partial_update 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/LICENSE ADDED
@@ -0,0 +1,20 @@
1
+ Copyright (c) 2010 Nathan Stults
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.rdoc ADDED
@@ -0,0 +1,71 @@
1
+ = mm-partial-update
2
+
3
+ mm-partial-update is a MongoMapper plugin that enables a change-only persistence strategy for MongoMapper. mm-partial-update uses the enhanced dirty tracking provided by mm-dirtier to send only changes to MognoDB, rather than the entire document on each save. *note: mm-partial update requires the rails3 branch of MongoMapper. It will not work with the master branch, the one you get when you 'gem install mongo_mapper'
4
+
5
+ == Installation
6
+
7
+ mm-partial-update is available as a RubyGem:
8
+
9
+ gem install mm_partial_update
10
+
11
+ To activate the plugin, add 'mm_partial_update' to your gemfile
12
+
13
+ gem 'mm_partial_update'
14
+
15
+ == Usage
16
+
17
+ mm-partial-update does not, by default, change MongoMappers normal mode of operation. Although mm-partial-update can be configured, either at a global level or on a model by model basis, to use a partial update strategy for all persistence operations, out of the box it simply adds a "save_changes" method to both MongoMapper::Document and MongoMapper::EmbeddedDocument, which will persist, using MongoDB's atomic operators, any changes to the target document or embedded document as well as any descendents of the target document or embedded document.
18
+
19
+ === For example:
20
+
21
+ class Person
22
+ include MongoMapper::Document
23
+ key :name, String
24
+ many :pets
25
+ end
26
+
27
+ class Pet
28
+ include MongoMapper::EmbeddedDocument
29
+ key :name, String
30
+ end
31
+
32
+ person = Person.create! :name=>"Willard"
33
+ person.name = "Poe"
34
+ person.save! #as always, overwrites the document in the database with the in memory copy
35
+
36
+ person = Person.create! :name=>"Willard"
37
+ person.name = "Poe"
38
+ person.save_changes #only saves the changed fields (in this case name=>"Poe"
39
+
40
+ However, in addition to #save_changes, partial saves can be enabled globally:
41
+
42
+ MmPartialUpdate.persistence_strategy = :changes_only
43
+
44
+ Or on a model by model basis:
45
+
46
+ class Person
47
+ include MongoMapper::Document
48
+ persistence_strategy :changes_only
49
+ key :name, String
50
+ many :pets
51
+ end
52
+
53
+ When enabled globally, all calls to save or save! will result in a partial update (with the exception of new documents). When enabled on a particular model, calls to save or save! on instances of that model and any subclasses of that model will result in partial updates. This is useful if you have a number of models but only one or two require partial saves for a particular use case, such as concurrent modification by more than one process.
54
+
55
+ == Known Issues & Limitations
56
+
57
+ * mm_partial_update will persist fully embedded documents on a change_only basis (i.e. issue $push, $pull and $set commands as appropriate). However, for the time being in_array_proxies behave like embedded Array keys, and persist as they always have in an all or nothing fashion. This will be changed in a future version.
58
+ * mm_partial_update will detect and persist changes to embedded Array and Hash keys, but only at a single level of depth. This means that modifying an array or hash contained within your embedded array or hash key will not trigger the dirty tracking mechanism (i.e. my_doc.tags.meta["happy"] = 5 would not cause my_doc.tags to appear changed). This is a very solvable problem, and a fix will likely appear in a future version, particularly if someone needs it (I myself do not).
59
+ * mm-partial-update does not (perhaps cannot?) behave in a truly atomic fashion. Because MongoDB appears to take a shotgun approach to detecting and preventing 'conflicting updates' in a single command, mm-partial-update breaks a single set of changes into multiple isolated commands to the database. Currently, the algorithm used is not very sophisticated, and results in a single update command for each unique embedded array that requires a push or a pull. In the future if this ends up presenting a performance problem, more sophisticated (but complicated) algorithms can be implemented to intelligently batch non-conflicting pushes and pulls together, resulting in a minimum of database calls. I'm holding off on that optimization until it proves necessary, however, mostly for maintainability reasons.
60
+
61
+ == Note on Patches/Pull Requests
62
+
63
+ * Fork the project.
64
+ * Make your feature addition or bug fix.
65
+ * Add tests for it. This is important so I don't break it in a future version unintentionally.
66
+ * Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
67
+ * Send me a pull request. Bonus points for topic branches.
68
+
69
+ == Copyright
70
+
71
+ Copyright (c) 2089 Nathan Stults. See LICENSE for details.
@@ -0,0 +1,32 @@
1
+ require 'rubygems'
2
+ require "mongo_mapper"
3
+ require "observables"
4
+ require "mm_dirtier"
5
+
6
+ base_dir = File.dirname(__FILE__)
7
+ [
8
+ 'version',
9
+ 'embedded_collection',
10
+ 'one_embedded_proxy',
11
+ 'extensions',
12
+ 'update_command',
13
+ 'plugins/partial_update',
14
+ 'plugins/document',
15
+ 'plugins/embedded_document'
16
+ ].each {|req| require File.join(base_dir,'mm_partial_update',req)}
17
+
18
+ MongoMapper::Document.append_inclusions(MmPartialUpdate::Plugins::PartialUpdate)
19
+ MongoMapper::Document.append_inclusions(MmPartialUpdate::Plugins::PartialUpdate::Document)
20
+
21
+ MongoMapper::EmbeddedDocument.append_inclusions(MmPartialUpdate::Plugins::PartialUpdate)
22
+ MongoMapper::EmbeddedDocument.append_inclusions(MmPartialUpdate::Plugins::PartialUpdate::EmbeddedDocument)
23
+
24
+ module MmPartialUpdate
25
+ def self.default_persistence_strategy
26
+ @default_persistence_strategy ||= :full_document
27
+ end
28
+
29
+ def self.default_persistence_strategy=(strategy)
30
+ @default_persistence_strategy = strategy
31
+ end
32
+ end
@@ -0,0 +1,47 @@
1
+ module MmPartialUpdate
2
+ module EmbeddedCollection
3
+
4
+ def save_to_collection(options)
5
+ super.tap { assign_database_indexes }
6
+ end
7
+
8
+ def add_updates_to_command(changes, command)
9
+ selector = association.name
10
+ selector = "#{proxy_owner.database_selector}.#{selector}" if
11
+ proxy_owner.respond_to?(:database_selector)
12
+
13
+ unless changes.blank?
14
+ deleted = changes[0] - changes[1]
15
+ deleted.each { |d| command.pull(selector, d._id) }
16
+ end
17
+
18
+ unless @target.blank?
19
+ @target.each do |child|
20
+ child.new? ? command.push(selector, child.to_mongo) :
21
+ child.add_updates_to_command(command)
22
+ end
23
+ end
24
+
25
+ end
26
+
27
+ private
28
+
29
+ def find_target
30
+ super.tap { |docs| assign_database_indexes(docs) }
31
+ end
32
+
33
+ def assign_database_indexes(docs = nil)
34
+ docs ||= @target
35
+ docs.each_with_index do |value, index|
36
+ value.instance_variable_set("@_database_position",index)
37
+ end if docs
38
+ end
39
+
40
+ def assign_references(*docs)
41
+ docs.each { |doc| doc.instance_variable_set("@_association_name", association.name) }
42
+ super(*docs)
43
+ end
44
+
45
+
46
+ end
47
+ end
@@ -0,0 +1,37 @@
1
+ class Object
2
+ def can_be_persistable?
3
+ respond_to?(:make_persistable)
4
+ end
5
+ end
6
+
7
+
8
+ module MongoMapper
9
+ module Plugins
10
+ module Associations
11
+
12
+ class EmbeddedCollection
13
+ def persistable?
14
+ kind_of?(MmPartialUpdate::EmbeddedCollection)
15
+ end
16
+
17
+ def make_persistable
18
+ class << self; include MmPartialUpdate::EmbeddedCollection; end unless persistable?
19
+ end
20
+
21
+ end
22
+
23
+ class OneEmbeddedProxy
24
+ def persistable?
25
+ kind_of?(MmPartialUpdate::OneEmbeddedProxy)
26
+ end
27
+
28
+ def make_persistable
29
+ class << self; include MmPartialUpdate::OneEmbeddedProxy; end unless persistable?
30
+ end
31
+
32
+ end
33
+
34
+
35
+ end
36
+ end
37
+ end
@@ -0,0 +1,22 @@
1
+ module MmPartialUpdate
2
+ module OneEmbeddedProxy
3
+
4
+ def add_updates_to_command(changes, command)
5
+ selector = association.name
6
+ selector = "#{proxy_owner.database_selector}.#{selector}" if
7
+ proxy_owner.respond_to?(:database_selector)
8
+
9
+ if @target.nil?
10
+ command.unset(selector, :nullify=>true) unless changes.blank?
11
+ else
12
+ @target.add_updates_to_command(command)
13
+ end
14
+ end
15
+
16
+ def assign_references(doc)
17
+ doc.instance_variable_set("@_association_name",association.name)
18
+ super(doc)
19
+ end
20
+
21
+ end
22
+ end
@@ -0,0 +1,44 @@
1
+ # encoding: UTF-8
2
+ module MmPartialUpdate
3
+ module Plugins
4
+ module PartialUpdate
5
+ module Document
6
+
7
+ def self.included(model)
8
+ model.plugin MmPartialUpdate::Plugins::PartialUpdate::Document
9
+ end
10
+
11
+ module InstanceMethods
12
+
13
+ def save_to_collection(options={})
14
+ strategy = determine_persistence_strategy(options)
15
+ return super if new? || strategy == :full_document
16
+
17
+ save_changes(options)
18
+ end
19
+
20
+ private
21
+
22
+ def assert_valid_persistence_strategy(strategy)
23
+ raise "Invalid persistence strategy (#{strategy}). Valid options are :full_document or :changes_only" unless ["full_document", "changes_only"].include?(strategy.to_s)
24
+ end
25
+
26
+ def determine_persistence_strategy(options)
27
+ strategy = options[:changes_only] ? :changes_only :
28
+ options[:persistence_strategy] ||
29
+ self.class.persistence_strategy ||
30
+ MmPartialUpdate.default_persistence_strategy ||
31
+ :full_document
32
+
33
+ strategy.tap { assert_valid_persistence_strategy(strategy) }
34
+ end
35
+
36
+ end
37
+
38
+
39
+ end
40
+ end
41
+ end
42
+ end
43
+
44
+
@@ -0,0 +1,43 @@
1
+ # encoding: UTF-8
2
+ module MmPartialUpdate
3
+ module Plugins
4
+ module PartialUpdate
5
+ module EmbeddedDocument
6
+
7
+ def self.included(model)
8
+ model.plugin MmPartialUpdate::Plugins::PartialUpdate::EmbeddedDocument
9
+ end
10
+
11
+ module InstanceMethods
12
+
13
+ def database_selector
14
+ selector = @_association_name.to_s
15
+ selector = "#{_parent_document.database_selector}.#{selector}" if
16
+ _parent_document.respond_to?(:database_selector)
17
+ selector = "#{selector}.#{@_database_position}" if defined?(@_database_position)
18
+ selector
19
+ end
20
+
21
+ private
22
+
23
+ def add_create_self_to_command(selector, command)
24
+ if _parent_document.new?
25
+ _parent_document.add_updates_to_command(command)
26
+ else
27
+
28
+ association = _parent_document.associations[@_association_name]
29
+ if association && association.many?
30
+ command.push(selector, self.to_mongo)
31
+ else
32
+ command.set(selector, self.to_mongo, :replace=>true)
33
+ end
34
+ end
35
+ end
36
+
37
+ end
38
+
39
+ end
40
+ end
41
+ end
42
+ end
43
+
@@ -0,0 +1,93 @@
1
+ # encoding: UTF-8
2
+ module MmPartialUpdate
3
+ module Plugins
4
+ module PartialUpdate
5
+
6
+ def self.included(model)
7
+ model.plugin MmDirtier::Plugins::Dirtier unless
8
+ model.plugins.include?(MmDirtier::Plugins::Dirtier)
9
+ model.plugin(MmPartialUpdate::Plugins::PartialUpdate)
10
+ end
11
+
12
+ module ClassMethods
13
+
14
+ def inherited(descendant)
15
+ descendant.instance_variable_set("@_persistence_strategy",
16
+ self.persistence_strategy)
17
+ super
18
+ end
19
+
20
+ def persistence_strategy(new_strategy=nil)
21
+ return @_persistence_strategy ||= nil unless new_strategy
22
+ @_persistence_strategy = new_strategy
23
+ end
24
+
25
+ end
26
+
27
+ module InstanceMethods
28
+
29
+ def save_changes(options={})
30
+ #We can't update an embedded document if the root isn't saved
31
+ #The clear_changes call is added here because dirty
32
+ #tracking happens further up the call chain than save_to_collection
33
+ #under normal circumstances, so we have to inject it
34
+ return _root_document.save_to_collection(options).tap {clear_changes} if
35
+ _root_document.new?
36
+
37
+ #persist changes to self and descendents
38
+ update_command = prepare_update_command
39
+ update_command.execute()
40
+
41
+ #clear dirty tracking
42
+ @_new = false
43
+ clear_changes
44
+ associations.each do |_, association|
45
+ proxy = get_proxy(association)
46
+ proxy.save_to_collection(options) if
47
+ proxy.proxy_respond_to?(:save_to_collection)
48
+ end
49
+ end
50
+
51
+ def prepare_update_command
52
+ UpdateCommand.new(self).tap { |command| add_updates_to_command(command) }
53
+ end
54
+
55
+ def add_updates_to_command(command)
56
+
57
+ selector = respond_to?(:database_selector) ? database_selector : nil
58
+
59
+ add_create_self_to_command(selector, command) and return if new?
60
+
61
+ field_changes = changes
62
+
63
+ associations.values.each do |association|
64
+ proxy = get_proxy(association)
65
+ association_changes = field_changes.delete(association.name)
66
+ proxy.add_updates_to_command(association_changes, command) if
67
+ proxy.respond_to?(:add_updates_to_command)
68
+ end
69
+
70
+ field_changes = field_changes.inject({}) do |changes,change|
71
+ changes[change[0]] = change[-1][-1]
72
+ changes
73
+ end
74
+ command.tap {|c|c.set(selector,field_changes)}
75
+ end
76
+
77
+ private
78
+
79
+ def add_create_self_to_command(selector, command)
80
+ command.tap { |c| c.set(selector, self.to_mongo, :replace=>true)}
81
+ end
82
+
83
+ def get_proxy(association)
84
+ proxy = super(association)
85
+ proxy.make_persistable if proxy.can_be_persistable?
86
+ proxy
87
+ end
88
+
89
+ end
90
+
91
+ end
92
+ end
93
+ end
@@ -0,0 +1,114 @@
1
+ module MmPartialUpdate
2
+ class UpdateCommand
3
+ attr_reader :target_document, :root_document, :commands
4
+
5
+ def initialize(target_document)
6
+ @target_document = target_document
7
+ @root_document = target_document.send(:_root_document)
8
+ @commands = BSON::OrderedHash.new { |hash,key| hash[key] = BSON::OrderedHash.new }
9
+ end
10
+
11
+ def document_selector
12
+ {:_id=>root_document._id}
13
+ end
14
+
15
+ def set(selector, fields, options={})
16
+
17
+ return if fields.blank?
18
+
19
+ selector = selector.to_s if selector
20
+
21
+ if selector.blank? && options[:replace]
22
+ commands["$set"] = fields
23
+ elsif selector.blank?
24
+ commands["$set"].merge!(fields)
25
+ elsif options[:replace]
26
+ commands["$set"][selector] = fields
27
+ else
28
+ commands["$set"].merge!(fields.keys.inject(BSON::OrderedHash.new) do |hash,field_name|
29
+ hash["#{selector}.#{field_name}"] = fields[field_name]
30
+ hash
31
+ end)
32
+ end
33
+ end
34
+
35
+ def unset(selector, options={})
36
+ raise "'unset' requires a non-blank selector" if selector.blank?
37
+ selector = selector.to_s
38
+ options[:nullify] ? commands["$set"][selector] = nil : commands["$unset"][selector] = true
39
+ end
40
+
41
+ def push(selector, document)
42
+ raise "'push' requires a non-blank selector" if selector.blank?
43
+ selector = selector.to_s
44
+ (commands[:pushes][selector] ||= []) << document
45
+ end
46
+
47
+ def pull(selector, document_id)
48
+ raise "'pull' requires a non-blank selector" if selector.blank?
49
+ selector = selector.to_s
50
+ (commands[:pulls][selector] ||= []) << document_id
51
+ end
52
+
53
+ def merge(other_command)
54
+ commands.merge! other_command.to_h
55
+ end
56
+
57
+ def to_h
58
+ commands
59
+ end
60
+
61
+ def empty?
62
+ commands.blank?
63
+ end
64
+
65
+ def reset
66
+ commands.clear
67
+ end
68
+
69
+ def execute(options={})
70
+ #if there are no commands, there is nothing to do...
71
+ return if empty?
72
+
73
+ selector = document_selector.tap {|s|s.merge!("$atomic"=>true) if options[:atomic]}
74
+
75
+ dbcommands = prepare_mongodb_commands
76
+
77
+ dbcommands.each do |command|
78
+ root_document.collection.update(selector, command, :multi=>false,
79
+ :upsert=>true, :safe=>options[:safe])
80
+ end
81
+ end
82
+
83
+ private
84
+
85
+ def prepare_mongodb_commands
86
+ dbcommands = []
87
+
88
+ initial_command = commands.dup
89
+ pushes, pulls = initial_command.delete(:pushes), initial_command.delete(:pulls)
90
+
91
+ dbcommands << initial_command unless initial_command.blank?
92
+
93
+ while (next_op = next_pull(pulls)); dbcommands << next_op; end
94
+ while (next_op = next_push(pushes)); dbcommands << next_op; end
95
+
96
+ dbcommands
97
+ end
98
+
99
+ def next_push(pushes)
100
+ return nil if pushes.blank?
101
+ selector = pushes.keys[0]
102
+ docs = pushes.delete(selector)
103
+ {"$pushAll" => { selector => docs } }
104
+ end
105
+
106
+ def next_pull(pulls)
107
+ return nil if pulls.blank?
108
+ selector = pulls.keys[0]
109
+ doc_ids = pulls.delete(selector)
110
+ {"$pull" => { selector => { "_id" => { "$in" => doc_ids } } } }
111
+ end
112
+
113
+ end
114
+ end