multi_db 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/LICENSE ADDED
@@ -0,0 +1,20 @@
1
+ The MIT license:
2
+ Copyright (c) 2008 Maximilian Schöfmann
3
+
4
+ Permission is hereby granted, free of charge, to any person obtaining a copy
5
+ of this software and associated documentation files (the "Software"), to deal
6
+ in the Software without restriction, including without limitation the rights
7
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
8
+ copies of the Software, and to permit persons to whom the Software is
9
+ furnished to do so, subject to the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be included in
12
+ all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
15
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
16
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
17
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
18
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
19
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
20
+ THE SOFTWARE.
@@ -0,0 +1,236 @@
1
+ = multi_db
2
+
3
+ =====-- This GEM was inspired by Rick Olson's "masochism"-Plugin
4
+
5
+ multi_db uses a connection proxy, which sends read queries to slave databases,
6
+ and all write queries to the master database (Read/Write Split).
7
+ Within transactions, while executing ActiveRecord Observers and
8
+ within "with_master" blocks (see below), even read queries are sent to the
9
+ master database.
10
+
11
+ === Important changes in 0.2.0
12
+
13
+ * As of this version, <tt>ActiveRecord::Base.connection</tt> does not return the
14
+ connection proxy by default anymore (therefore the jump to 0.2.0). Only models
15
+ inheriting from AR::B return the proxy, unless they are defined as master_models
16
+ (see below). If you want to access the connection proxy from AR::B directly,
17
+ use <tt>ActiveRecord::Base.connection_proxy</tt>.
18
+
19
+ * This version is the first attempt for thread-safety of this gem. <em>There might
20
+ still be some threading issues left!</em>. So please test your apps thoroughly
21
+ and report any issues you might encounter.
22
+
23
+ * <tt>CGI::Session::ActiveRecordStore::Session</tt> is now automatically registered
24
+ as a master model.
25
+
26
+ === Caveats
27
+
28
+ * works only with activerecord 2.1, 2.2 and 2.3
29
+
30
+ === Install
31
+
32
+ gem sources --add http://gems.github.com # only if you haven't already added github
33
+ gem install schoefmax-multi_db
34
+
35
+ When using Rails, add this to your environment.rb:
36
+
37
+ config.gem 'schoefmax-multi_db', :lib => 'multi_db', :source => 'http://gems.github.com'
38
+
39
+ === Setup
40
+
41
+ In your database.yml, add sections for the slaves, e.g.:
42
+
43
+ production: # that would be the master
44
+ adapter: mysql
45
+ database: myapp_production
46
+ username: root
47
+ password:
48
+ host: localhost
49
+
50
+ production_slave_database: # that would be a slave
51
+ adapter: mysql
52
+ database: myapp_production
53
+ username: root
54
+ password:
55
+ host: 10.0.0.2
56
+
57
+ production_slave_database_2: # another slave
58
+ ...
59
+ production_slave_database_in_india: # yet another one
60
+ ...
61
+
62
+ *NOTE*: multi_db identifies slave databases by looking for entries of the form
63
+ "<tt><environment>_slave_database<_optional_name></tt>". As a (useless) side effect you
64
+ get abstract classes named <tt>MultiDb::SlaveDatabaseInIndia</tt> etc.
65
+ The advantage of specifying the slaves explicitly, instead of the master, is that
66
+ you can use the same configuration file for scripts that don't use multi_db.
67
+ Also, when you decide to disable multi_db for some reason, you don't have to
68
+ swap hosts in your <tt>database.yml</tt> from master to slave (which is easy to forget...).
69
+
70
+ To enable the proxy globally, add this to your environment.rb, or some file in
71
+ config/initializers:
72
+
73
+ MultiDb::ConnectionProxy.setup!
74
+
75
+ If you only want to enable it for specific environments, add this to
76
+ the corresponding file in config/environments:
77
+
78
+ config.after_initialize do
79
+ MultiDb::ConnectionProxy.setup!
80
+ end
81
+
82
+ In the development and test environments, you can use identical configurations
83
+ for master and slave connections. This can help you finding (some of the) issues
84
+ your application might have with a replicated database setup without actually having
85
+ one on your development machine.
86
+
87
+ === Using with Phusion Passenger
88
+
89
+ With Passengers smart spawning method, child processes forked by the ApplicationSpawner
90
+ won't have the connection proxy set up properly.
91
+
92
+ To make it work, add this to your <tt>environment.rb</tt> or an initializer script
93
+ (e.g. <tt>config/initializers/connection_proxy.rb</tt>):
94
+
95
+ if defined?(PhusionPassenger)
96
+ PhusionPassenger.on_event(:starting_worker_process) do |forked|
97
+ if forked
98
+ # ... set MultiDb configuration options, if any ...
99
+ MultiDb::ConnectionProxy.setup!
100
+ end
101
+ end
102
+ else # not using passenger (e.g. development/testing)
103
+ # ... set MultiDb configuration options, if any ...
104
+ MultiDb::ConnectionProxy.setup!
105
+ end
106
+
107
+ Thanks to Nathan Esquenazi for testing this.
108
+
109
+ === Forcing the master for certain actions
110
+
111
+ Just add this to your controller:
112
+
113
+ around_filter(:only => :foo_action) { |c,a| ActiveRecord::Base.connection_proxy.with_master { a.call } }
114
+
115
+ === Forcing the master for certain models
116
+
117
+ In your environment.rb or an initializer, add this *before* the call to <tt>setup!</tt>:
118
+
119
+ MultiDb::ConnectionProxy.master_models = ['CGI::Session::ActiveRecordStore::Session', 'PaymentTransaction', ...]
120
+ MultiDb::ConnectionProxy.setup!
121
+
122
+ *NOTE*: You cannot safely add more master_models after calling <tt>setup!</tt>.
123
+
124
+ === Making one slave database sticky during a request
125
+
126
+ This can be useful to leverage database level query caching as all queries will
127
+ be sent to the same slave database during one web request.
128
+
129
+ To enable, add this to your environment.rb just before <tt>MultiDb::ConnectionProxy.setup!</tt>:
130
+
131
+ MultiDb::ConnectionProxy.sticky_slave = true
132
+
133
+ And add this to your ApplicationController:
134
+
135
+ after_filter { ActiveRecord::Base.connection_proxy.next_reader! }
136
+
137
+ *NOTE*: It's not possible to toggle this mode in a running process, as the dynamically
138
+ generated methods will have the initially defined "stickyness" built in.
139
+
140
+ === Usage outside of Rails
141
+
142
+ You can use multi_db together with other framworks or in standalone scripts.
143
+ Example:
144
+
145
+ require 'rubygems'
146
+ require 'active_record'
147
+ require 'multi_db'
148
+
149
+ ActiveRecord::Base.logger = Logger.new(STDOUT)
150
+ ActiveRecord::Base.configurations = {
151
+ 'development' => {
152
+ 'adapter' => 'mysql',
153
+ 'host' => 'localhost',
154
+ 'username' => 'root',
155
+ 'database' => 'multi_db_test'
156
+ },
157
+ 'development_slave_database' => {
158
+ 'adapter' => 'mysql',
159
+ 'host' => 'localhost',
160
+ 'username' => 'root',
161
+ 'database' => 'multi_db_test'
162
+ }
163
+ }
164
+ ActiveRecord::Base.establish_connection :development
165
+ MultiDb::ConnectionProxy.setup!
166
+
167
+ class MyModel < ActiveRecord::Base
168
+ # ...
169
+ end
170
+
171
+ # ...
172
+
173
+ Note that the configurations hash should contain strings as keys instead of symbols.
174
+
175
+ === Differences to "masochism":
176
+
177
+ * Supports multiple slave databases (round robin)
178
+ * It sends everything except "select ..." queries to the master, instead of
179
+ sending only specific things to the master and anything "else" to the slave.
180
+ This avoids accidential writes to the master when there are API changes in
181
+ ActiveRecord which haven't been picked up by multi_db yet.
182
+ Note that this behaviour will also always send helper methods like "+quote+" or
183
+ "<tt>add_limit!</tt>" to the master connection object, which doesn't add any
184
+ more load on the master, as these methods don't communicate with the db server
185
+ itself.
186
+ * It uses its own query cache as the slave's cache isn't emptied when there are
187
+ changes on the master
188
+ * It supports immediate failover for slave connections
189
+ * It will wait some time before trying to query a failed slave database again
190
+ * It supports nesting "with_master"-blocks, without unexpectedly switching you
191
+ back to the slave again
192
+ * It schedules a reconnect of the master connection if statements fail there.
193
+ This might help with HA setups using virtual IPs (a test setup would be nice
194
+ to verify this)
195
+ * You specify slave databases in the configuration instead of specifying an extra
196
+ master database. This makes disabling or removing multi_db less dangerous
197
+ (Update: Recent versions of masochism support this, too).
198
+ * There are no <tt>set_to_master!</tt> and <tt>set_to_slave!</tt> methods, just
199
+ <tt>with_master(&block)</tt>
200
+ * All proxied methods are dynamically generated for better performance
201
+
202
+ === See also
203
+
204
+ ==== Masochism
205
+
206
+ The original plugin:
207
+
208
+ * http://github.com/technoweenie/masochism
209
+
210
+ ==== DataFabric
211
+
212
+ A solution by FiveRuns, also based on masochism but without the "nested with_master"-issue,
213
+ threadsafe and allows sharding of data.
214
+
215
+ * http://github.com/fiveruns/data_fabric
216
+
217
+ === Contributors
218
+
219
+ * Matt Conway http://github.com/wr0ngway
220
+ * Matthias Marshall http://github.com/webops
221
+
222
+ === Ideas
223
+
224
+ See: http://github.com/schoefmax/multi_db/wikis/home
225
+
226
+ === Running specs
227
+
228
+ If you haven't already, install the rspec gem, then create an empty database
229
+ called "multi_db_test" (you might want to tweak the spec/config/database.yml).
230
+ From the plugin directory, run:
231
+
232
+ spec spec
233
+
234
+
235
+ Copyright (c) 2008, Max Schoefmann <max (a) pragmatic-it de>
236
+ Released under the MIT license
@@ -0,0 +1,6 @@
1
+ require 'tlattr_accessors'
2
+ require 'multi_db/scheduler'
3
+ require 'multi_db/active_record_extensions'
4
+ require 'multi_db/observer_extensions'
5
+ require 'multi_db/query_cache_compat'
6
+ require 'multi_db/connection_proxy'
@@ -0,0 +1,55 @@
1
+ module MultiDb
2
+ module ActiveRecordExtensions
3
+ def self.included(base)
4
+ base.send :include, InstanceMethods
5
+ base.send :extend, ClassMethods
6
+ base.alias_method_chain :reload, :master
7
+ base.cattr_accessor :connection_proxy
8
+ # handle subclasses which were defined by the framework or plugins
9
+ base.send(:subclasses).each do |child|
10
+ child.hijack_connection
11
+ end
12
+ end
13
+
14
+ module InstanceMethods
15
+ def reload_with_master(*args, &block)
16
+ self.connection_proxy.with_master { reload_without_master }
17
+ end
18
+ end
19
+
20
+ module ClassMethods
21
+ # Make sure transactions always switch to the master
22
+ def transaction(&block)
23
+ if self.connection.kind_of?(ConnectionProxy)
24
+ super
25
+ else
26
+ self.connection_proxy.with_master { super }
27
+ end
28
+ end
29
+
30
+ # make caching always use the ConnectionProxy
31
+ def cache(&block)
32
+ if ActiveRecord::Base.configurations.blank?
33
+ yield
34
+ else
35
+ self.connection_proxy.cache(&block)
36
+ end
37
+ end
38
+
39
+ def inherited(child)
40
+ super
41
+ child.hijack_connection
42
+ end
43
+
44
+ def hijack_connection
45
+ return if ConnectionProxy.master_models.include?(self.to_s)
46
+ logger.info "[MULTIDB] hijacking connection for #{self.to_s}"
47
+ class << self
48
+ def connection
49
+ self.connection_proxy
50
+ end
51
+ end
52
+ end
53
+ end
54
+ end
55
+ end
@@ -0,0 +1,189 @@
1
+ module MultiDb
2
+ class ConnectionProxy
3
+ include QueryCacheCompat
4
+ include ActiveRecord::ConnectionAdapters::QueryCache
5
+ extend ThreadLocalAccessors
6
+
7
+ # Safe methods are those that should either go to the slave ONLY or go
8
+ # to the current active connection.
9
+ SAFE_METHODS = [ :select_all, :select_one, :select_value, :select_values,
10
+ :select_rows, :select, :verify!, :raw_connection, :active?, :reconnect!,
11
+ :disconnect!, :reset_runtime, :log, :log_info ]
12
+
13
+ if ActiveRecord.const_defined?(:SessionStore) # >= Rails 2.3
14
+ DEFAULT_MASTER_MODELS = ['ActiveRecord::SessionStore::Session']
15
+ else # =< Rails 2.3
16
+ DEFAULT_MASTER_MODELS = ['CGI::Session::ActiveRecordStore::Session']
17
+ end
18
+
19
+ attr_accessor :master
20
+ tlattr_accessor :master_depth, :current, true
21
+
22
+ class << self
23
+
24
+ # defaults to RAILS_ENV if multi_db is used with Rails
25
+ # defaults to 'development' when used outside Rails
26
+ attr_accessor :environment
27
+
28
+ # a list of models that should always go directly to the master
29
+ #
30
+ # Example:
31
+ #
32
+ # MultiDb::ConnectionProxy.master_models = ['MySessionStore', 'PaymentTransaction']
33
+ attr_accessor :master_models
34
+
35
+ # decides if we should switch to the next reader automatically.
36
+ # If set to false, an after|before_filter in the ApplicationController
37
+ # has to do this.
38
+ # This will not affect failover if a master is unavailable.
39
+ attr_accessor :sticky_slave
40
+
41
+ # Replaces the connection of ActiveRecord::Base with a proxy and
42
+ # establishes the connections to the slaves.
43
+ def setup!
44
+ self.master_models ||= DEFAULT_MASTER_MODELS
45
+ self.environment ||= (defined?(RAILS_ENV) ? RAILS_ENV : 'development')
46
+ self.sticky_slave ||= false
47
+
48
+ master = ActiveRecord::Base
49
+ slaves = init_slaves
50
+ raise "No slaves databases defined for environment: #{self.environment}" if slaves.empty?
51
+ master.send :include, MultiDb::ActiveRecordExtensions
52
+ ActiveRecord::Observer.send :include, MultiDb::ObserverExtensions
53
+ master.connection_proxy = new(master, slaves)
54
+ master.logger.info("** multi_db with master and #{slaves.length} slave#{"s" if slaves.length > 1} loaded.")
55
+ end
56
+
57
+ protected
58
+
59
+ # Slave entries in the database.yml must be named like this
60
+ # development_slave_database:
61
+ # or
62
+ # development_slave_database1:
63
+ # or
64
+ # production_slave_database_someserver:
65
+ # These would be available later as MultiDb::SlaveDatabaseSomeserver
66
+ def init_slaves
67
+ returning([]) do |slaves|
68
+ ActiveRecord::Base.configurations.keys.each do |name|
69
+ if name.to_s =~ /#{self.environment}_(slave_database.*)/
70
+ MultiDb.module_eval %Q{
71
+ class #{$1.camelize} < ActiveRecord::Base
72
+ self.abstract_class = true
73
+ establish_connection :#{name}
74
+ end
75
+ }, __FILE__, __LINE__
76
+ slaves << "MultiDb::#{$1.camelize}".constantize
77
+ end
78
+ end
79
+ end
80
+ end
81
+
82
+ private :new
83
+
84
+ end
85
+
86
+ def initialize(master, slaves)
87
+ @slaves = Scheduler.new(slaves)
88
+ @master = master
89
+ @reconnect = false
90
+ self.current = @slaves.current
91
+ self.master_depth = 0
92
+ end
93
+
94
+ def slave
95
+ @slaves.current
96
+ end
97
+
98
+ def with_master
99
+ self.current = @master
100
+ self.master_depth += 1
101
+ yield
102
+ ensure
103
+ self.master_depth -= 1
104
+ self.current = slave if master_depth.zero?
105
+ end
106
+
107
+ def transaction(start_db_transaction = true, &block)
108
+ with_master { @master.retrieve_connection.transaction(start_db_transaction, &block) }
109
+ end
110
+
111
+ # Calls the method on master/slave and dynamically creates a new
112
+ # method on success to speed up subsequent calls
113
+ def method_missing(method, *args, &block)
114
+ returning(send(target_method(method), method, *args, &block)) do
115
+ create_delegation_method!(method)
116
+ end
117
+ end
118
+
119
+ # Switches to the next slave database for read operations.
120
+ # Fails over to the master database if all slaves are unavailable.
121
+ def next_reader!
122
+ return unless master_depth.zero? # don't if in with_master block
123
+ self.current = @slaves.next
124
+ rescue Scheduler::NoMoreItems
125
+ logger.warn "[MULTIDB] All slaves are blacklisted. Reading from master"
126
+ self.current = @master
127
+ end
128
+
129
+ protected
130
+
131
+ def create_delegation_method!(method)
132
+ self.instance_eval %Q{
133
+ def #{method}(*args, &block)
134
+ #{'next_reader!' unless self.class.sticky_slave || unsafe?(method)}
135
+ #{target_method(method)}(:#{method}, *args, &block)
136
+ end
137
+ }, __FILE__, __LINE__
138
+ end
139
+
140
+ def target_method(method)
141
+ unsafe?(method) ? :send_to_master : :send_to_current
142
+ end
143
+
144
+ def send_to_master(method, *args, &block)
145
+ reconnect_master! if @reconnect
146
+ @master.retrieve_connection.send(method, *args, &block)
147
+ rescue => e
148
+ raise_master_error(e)
149
+ end
150
+
151
+ def send_to_current(method, *args, &block)
152
+ reconnect_master! if @reconnect && master?
153
+ current.retrieve_connection.send(method, *args, &block)
154
+ rescue NotImplementedError, NoMethodError
155
+ raise
156
+ rescue => e # TODO don't rescue everything
157
+ raise_master_error(e) if master?
158
+ logger.warn "[MULTIDB] Error reading from slave database"
159
+ logger.error %(#{e.message}\n#{e.backtrace.join("\n")})
160
+ @slaves.blacklist!(current)
161
+ next_reader!
162
+ retry
163
+ end
164
+
165
+ def reconnect_master!
166
+ @master.retrieve_connection.reconnect!
167
+ @reconnect = false
168
+ end
169
+
170
+ def raise_master_error(error)
171
+ logger.fatal "[MULTIDB] Error accessing master database. Scheduling reconnect"
172
+ @reconnect = true
173
+ raise error
174
+ end
175
+
176
+ def unsafe?(method)
177
+ !SAFE_METHODS.include?(method)
178
+ end
179
+
180
+ def master?
181
+ current == @master
182
+ end
183
+
184
+ def logger
185
+ ActiveRecord::Base.logger
186
+ end
187
+
188
+ end
189
+ end
@@ -0,0 +1,18 @@
1
+ module MultiDb
2
+ module ObserverExtensions
3
+ def self.included(base)
4
+ base.alias_method_chain :update, :masterdb
5
+ end
6
+
7
+ # Send observed_method(object) if the method exists.
8
+ def update_with_masterdb(observed_method, object) #:nodoc:
9
+ if object.class.connection.respond_to?(:with_master)
10
+ object.class.connection.with_master do
11
+ update_without_masterdb(observed_method, object)
12
+ end
13
+ else
14
+ update_without_masterdb(observed_method, object)
15
+ end
16
+ end
17
+ end
18
+ end
@@ -0,0 +1,21 @@
1
+ module MultiDb
2
+ # Implements the methods expected by the QueryCache module
3
+ module QueryCacheCompat
4
+ def select_all(*a, &b)
5
+ next_reader! unless ConnectionProxy.sticky_slave
6
+ send_to_current(:select_all, *a, &b)
7
+ end
8
+ def columns(*a, &b)
9
+ send_to_current(:columns, *a, &b)
10
+ end
11
+ def insert(*a, &b)
12
+ send_to_master(:insert, *a, &b)
13
+ end
14
+ def update(*a, &b)
15
+ send_to_master(:update, *a, &b)
16
+ end
17
+ def delete(*a, &b)
18
+ send_to_master(:delete, *a, &b)
19
+ end
20
+ end
21
+ end
@@ -0,0 +1,41 @@
1
+ module MultiDb
2
+ class Scheduler
3
+ class NoMoreItems < Exception; end
4
+ extend ThreadLocalAccessors
5
+
6
+ attr :items
7
+ delegate :[], :[]=, :to => :items
8
+ tlattr_accessor :current_index, true
9
+
10
+ def initialize(items, blacklist_timeout = 1.minute)
11
+ @n = items.length
12
+ @items = items
13
+ @blacklist = Array.new(@n, Time.at(0))
14
+ @blacklist_timeout = blacklist_timeout
15
+ self.current_index = 0
16
+ end
17
+
18
+ def blacklist!(item)
19
+ @blacklist[@items.index(item)] = Time.now
20
+ end
21
+
22
+ def current
23
+ @items[current_index]
24
+ end
25
+
26
+ def next
27
+ previous = current_index
28
+ until(@blacklist[next_index!] < Time.now - @blacklist_timeout) do
29
+ raise NoMoreItems, 'All items are blacklisted' if current_index == previous
30
+ end
31
+ current
32
+ end
33
+
34
+ protected
35
+
36
+ def next_index!
37
+ self.current_index = (current_index + 1) % @n
38
+ end
39
+
40
+ end
41
+ end
@@ -0,0 +1,36 @@
1
+ # -*- encoding: utf-8 -*-
2
+
3
+ Gem::Specification.new do |s|
4
+ s.name = %q{multi_db}
5
+ s.version = "0.2.1"
6
+
7
+ s.required_rubygems_version = Gem::Requirement.new(">= 1.2") if s.respond_to? :required_rubygems_version=
8
+ s.authors = ["Maximilian Sch\303\266fmann"]
9
+ s.date = %q{2009-03-11}
10
+ s.description = "Connection proxy for ActiveRecord for single master / multiple slave database deployments"
11
+ s.email = "max@pragmatic-it.de"
12
+ s.extra_rdoc_files = ["LICENSE", "README.rdoc"]
13
+ s.files = ["lib/multi_db.rb", "lib/multi_db/active_record_extensions.rb", "lib/multi_db/connection_proxy.rb", "lib/multi_db/observer_extensions.rb", "lib/multi_db/query_cache_compat.rb", "lib/multi_db/scheduler.rb", "LICENSE", "README.rdoc", "spec/config/database.yml", "spec/connection_proxy_spec.rb", "spec/scheduler_spec.rb", "spec/spec_helper.rb", "multi_db.gemspec"]
14
+ s.has_rdoc = true
15
+ s.homepage = "http://github.com/schoefmax/multi_db"
16
+ s.rdoc_options = ["--line-numbers", "--inline-source", "--title", "multi_db", "--main", "README.rdoc"]
17
+ s.require_paths = ["lib"]
18
+ s.rubyforge_project = "multi_db"
19
+ s.rubygems_version = %q{1.3.1}
20
+ s.summary = "Connection proxy for ActiveRecord for single master / multiple slave database deployments"
21
+
22
+ if s.respond_to? :specification_version then
23
+ s.specification_version = 2
24
+
25
+ if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
26
+ s.add_runtime_dependency('activerecord', [">= 2.1.0"])
27
+ s.add_runtime_dependency('tlattr_accessors', [">= 0.0.3"])
28
+ else
29
+ s.add_dependency('activerecord', [">= 2.1.0"])
30
+ s.add_dependency('tlattr_accessors', [">= 0.0.3"])
31
+ end
32
+ else
33
+ s.add_dependency('activerecord', [">= 2.1.0"])
34
+ s.add_dependency('tlattr_accessors', [">= 0.0.3"])
35
+ end
36
+ end
@@ -0,0 +1,23 @@
1
+ test:
2
+ adapter: mysql
3
+ database: multi_db_test
4
+ username: root
5
+ password:
6
+ host: 127.0.0.1
7
+ pool: 5
8
+
9
+ test_slave_database_1:
10
+ adapter: mysql
11
+ database: multi_db_test
12
+ username: root
13
+ password:
14
+ host: 127.0.0.1
15
+ pool: 5
16
+
17
+ test_slave_database_2:
18
+ adapter: mysql
19
+ database: multi_db_test
20
+ username: root
21
+ password:
22
+ host: 127.0.0.1
23
+ pool: 5
@@ -0,0 +1,229 @@
1
+ require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
2
+ require MULTI_DB_SPEC_DIR + '/../lib/multi_db/query_cache_compat'
3
+ require MULTI_DB_SPEC_DIR + '/../lib/multi_db/active_record_extensions'
4
+ require MULTI_DB_SPEC_DIR + '/../lib/multi_db/observer_extensions'
5
+ require MULTI_DB_SPEC_DIR + '/../lib/multi_db/scheduler'
6
+ require MULTI_DB_SPEC_DIR + '/../lib/multi_db/connection_proxy'
7
+
8
+ RAILS_ROOT = MULTI_DB_SPEC_DIR
9
+
10
+ describe MultiDb::ConnectionProxy do
11
+
12
+ before(:all) do
13
+ ActiveRecord::Base.configurations = MULTI_DB_SPEC_CONFIG
14
+ ActiveRecord::Base.establish_connection :test
15
+ ActiveRecord::Migration.verbose = false
16
+ ActiveRecord::Migration.create_table(:master_models, :force => true) {}
17
+ class MasterModel < ActiveRecord::Base; end
18
+ ActiveRecord::Migration.create_table(:foo_models, :force => true) {|t| t.string :bar}
19
+ class FooModel < ActiveRecord::Base; end
20
+ @sql = 'SELECT 1 + 1 FROM DUAL'
21
+ end
22
+
23
+ before(:each) do
24
+ MultiDb::ConnectionProxy.master_models = ['MasterModel']
25
+ MultiDb::ConnectionProxy.setup!
26
+ @proxy = ActiveRecord::Base.connection_proxy
27
+ @master = @proxy.master.retrieve_connection
28
+ @slave1 = MultiDb::SlaveDatabase1.retrieve_connection
29
+ @slave2 = MultiDb::SlaveDatabase2.retrieve_connection
30
+ end
31
+
32
+ after(:each) do
33
+ ActiveRecord::Base.send :alias_method, :reload, :reload_without_master
34
+ end
35
+
36
+ it 'AR::B should respond to #connection_proxy' do
37
+ ActiveRecord::Base.connection_proxy.should be_kind_of(MultiDb::ConnectionProxy)
38
+ end
39
+
40
+ it 'FooModel#connection should return an instance of MultiDb::ConnectionProxy' do
41
+ FooModel.connection.should be_kind_of(MultiDb::ConnectionProxy)
42
+ end
43
+
44
+ it 'MasterModel#connection should not return an instance of MultiDb::ConnectionProxy' do
45
+ MasterModel.connection.should_not be_kind_of(MultiDb::ConnectionProxy)
46
+ end
47
+
48
+ it "should generate classes for each entry in the database.yml" do
49
+ defined?(MultiDb::SlaveDatabase1).should_not be_nil
50
+ defined?(MultiDb::SlaveDatabase2).should_not be_nil
51
+ end
52
+
53
+ it 'should handle nested with_master-blocks correctly' do
54
+ @proxy.current.should_not == @proxy.master
55
+ @proxy.with_master do
56
+ @proxy.current.should == @proxy.master
57
+ @proxy.with_master do
58
+ @proxy.current.should == @proxy.master
59
+ @proxy.with_master do
60
+ @proxy.current.should == @proxy.master
61
+ end
62
+ @proxy.current.should == @proxy.master
63
+ end
64
+ @proxy.current.should == @proxy.master
65
+ end
66
+ @proxy.current.should_not == @proxy.master
67
+ end
68
+
69
+ it 'should perform transactions on the master' do
70
+ @master.should_receive(:select_all).exactly(1) # makes sure the first one goes to a slave
71
+ @proxy.select_all(@sql)
72
+ ActiveRecord::Base.transaction do
73
+ @proxy.select_all(@sql)
74
+ end
75
+ end
76
+
77
+ it 'should switch to the next reader on selects' do
78
+ @slave1.should_receive(:select_one).exactly(2)
79
+ @slave2.should_receive(:select_one).exactly(2)
80
+ 4.times { @proxy.select_one(@sql) }
81
+ end
82
+
83
+ it 'should not switch to the next reader when whithin a with_master-block' do
84
+ @master.should_receive(:select_one).twice
85
+ @slave1.should_not_receive(:select_one)
86
+ @slave2.should_not_receive(:select_one)
87
+ @proxy.with_master do
88
+ 2.times { @proxy.select_one(@sql) }
89
+ end
90
+ end
91
+
92
+ it 'should send dangerous methods to the master' do
93
+ meths = [:insert, :update, :delete, :execute]
94
+ meths.each do |meth|
95
+ @slave1.stub!(meth).and_raise(RuntimeError)
96
+ @master.should_receive(meth).and_return(true)
97
+ @proxy.send(meth, @sql)
98
+ end
99
+ end
100
+
101
+ it 'should dynamically generate safe methods' do
102
+ @proxy.should_not respond_to(:select_value)
103
+ @proxy.select_value(@sql)
104
+ @proxy.should respond_to(:select_value)
105
+ end
106
+
107
+ it 'should cache queries using select_all' do
108
+ ActiveRecord::Base.cache do
109
+ # next_reader will be called and switch to the SlaveDatabase2
110
+ @slave2.should_receive(:select_all).exactly(1)
111
+ @slave1.should_not_receive(:select_all)
112
+ @master.should_not_receive(:select_all)
113
+ 3.times { @proxy.select_all(@sql) }
114
+ end
115
+ end
116
+
117
+ it 'should invalidate the cache on insert, delete and update' do
118
+ ActiveRecord::Base.cache do
119
+ meths = [:insert, :update, :delete]
120
+ meths.each do |meth|
121
+ @master.should_receive(meth).and_return(true)
122
+ end
123
+ @slave2.should_receive(:select_all).twice
124
+ @slave1.should_receive(:select_all).once
125
+ 3.times do |i|
126
+ @proxy.select_all(@sql)
127
+ @proxy.send(meths[i])
128
+ end
129
+ end
130
+ end
131
+
132
+ it 'should retry the next slave when one fails and finally fall back to the master' do
133
+ @slave1.should_receive(:select_all).once.and_raise(RuntimeError)
134
+ @slave2.should_receive(:select_all).once.and_raise(RuntimeError)
135
+ @master.should_receive(:select_all).and_return(true)
136
+ @proxy.select_all(@sql)
137
+ end
138
+
139
+ it 'should try to reconnect the master connection after the master has failed' do
140
+ @master.should_receive(:update).and_raise(RuntimeError)
141
+ lambda { @proxy.update(@sql) }.should raise_error
142
+ @master.should_receive(:reconnect!).and_return(true)
143
+ @master.should_receive(:insert).and_return(1)
144
+ @proxy.insert(@sql)
145
+ end
146
+
147
+ it 'should reload models from the master' do
148
+ foo = FooModel.create!(:bar => 'baz')
149
+ foo.bar = "not_saved"
150
+ @slave1.should_not_receive(:select_all)
151
+ @slave2.should_not_receive(:select_all)
152
+ foo.reload
153
+ # we didn't stub @master#select_all here, check that we actually hit the db
154
+ foo.bar.should == 'baz'
155
+ end
156
+
157
+ describe 'with sticky_slave ' do
158
+
159
+ before { MultiDb::ConnectionProxy.sticky_slave = true }
160
+ after { MultiDb::ConnectionProxy.sticky_slave = false }
161
+
162
+ it 'should not switch to the next reader automatically' do
163
+ @slave1.should_receive(:select_all).exactly(3)
164
+ @slave2.should_receive(:select_all).exactly(0)
165
+ 3.times { @proxy.select_all(@sql) }
166
+ end
167
+
168
+ it '#next_reader! should switch to the next slave' do
169
+ @slave1.should_receive(:select_one).exactly(3)
170
+ @slave2.should_receive(:select_one).exactly(7)
171
+ 3.times { @proxy.select_one(@sql) }
172
+ @proxy.next_reader!
173
+ 7.times { @proxy.select_one(@sql) }
174
+ end
175
+
176
+ end
177
+
178
+ describe '(accessed from multiple threads)' do
179
+ # NOTE: We cannot put expectations on the connection objects itself
180
+ # for the threading specs, as connection pooling will cause
181
+ # different connections being returned for different threads.
182
+
183
+ it '#current and #next_reader! should be local to the thread' do
184
+ @proxy.current.should == MultiDb::SlaveDatabase1
185
+ @proxy.next_reader!.should == MultiDb::SlaveDatabase2
186
+ Thread.new do
187
+ @proxy.current.should == MultiDb::SlaveDatabase1
188
+ @proxy.next_reader!.should == MultiDb::SlaveDatabase2
189
+ @proxy.current.should == MultiDb::SlaveDatabase2
190
+ @proxy.next_reader!.should == MultiDb::SlaveDatabase1
191
+ @proxy.current.should == MultiDb::SlaveDatabase1
192
+ end
193
+ @proxy.current.should == MultiDb::SlaveDatabase2
194
+ end
195
+
196
+ it '#with_master should be local to the thread' do
197
+ @proxy.current.should_not == @proxy.master
198
+ @proxy.with_master do
199
+ @proxy.current.should == @proxy.master
200
+ Thread.new do
201
+ @proxy.current.should_not == @proxy.master
202
+ @proxy.with_master do
203
+ @proxy.current.should == @proxy.master
204
+ end
205
+ @proxy.current.should_not == @proxy.master
206
+ end
207
+ @proxy.current.should == @proxy.master
208
+ end
209
+ @proxy.current.should_not == @proxy.master
210
+ end
211
+
212
+ it 'should switch to the next reader even whithin with_master-block in different threads' do
213
+ # Because of connection pooling in AR 2.2, the second thread will cause
214
+ # a new connection being created behind the scenes. We therefore just test
215
+ # that these connections are beting retrieved for the right databases here.
216
+ @proxy.master.should_not_receive(:retrieve_connection).and_return(@master)
217
+ MultiDb::SlaveDatabase1.should_receive(:retrieve_connection).twice.and_return(@slave1)
218
+ MultiDb::SlaveDatabase2.should_receive(:retrieve_connection).once.and_return(@slave2)
219
+ @proxy.with_master do
220
+ Thread.new do
221
+ 3.times { @proxy.select_one(@sql) }
222
+ end.join
223
+ end
224
+ end
225
+
226
+ end
227
+
228
+ end
229
+
@@ -0,0 +1,58 @@
1
+ require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
2
+ require MULTI_DB_SPEC_DIR + '/../lib/multi_db/scheduler'
3
+
4
+ describe MultiDb::Scheduler do
5
+
6
+ before do
7
+ @items = [5, 7, 4, 8]
8
+ @scheduler = MultiDb::Scheduler.new(@items.clone)
9
+ end
10
+
11
+ it "should return items in a round robin fashion" do
12
+ first = @items.shift
13
+ @scheduler.current.should == first
14
+ @items.each do |item|
15
+ @scheduler.next.should == item
16
+ end
17
+ @scheduler.next.should == first
18
+ end
19
+
20
+ it 'should not return blacklisted items' do
21
+ @scheduler.blacklist!(4)
22
+ @items.size.times do
23
+ @scheduler.next.should_not == 4
24
+ end
25
+ end
26
+
27
+ it 'should raise NoMoreItems if all are blacklisted' do
28
+ @items.each do |item|
29
+ @scheduler.blacklist!(item)
30
+ end
31
+ lambda {
32
+ @scheduler.next
33
+ }.should raise_error(MultiDb::Scheduler::NoMoreItems)
34
+ end
35
+
36
+ it 'should unblacklist items automatically' do
37
+ @scheduler = MultiDb::Scheduler.new(@items.clone, 1.second)
38
+ @scheduler.blacklist!(7)
39
+ sleep(1)
40
+ @scheduler.next.should == 7
41
+ end
42
+
43
+ describe '(accessed from multiple threads)' do
44
+
45
+ it '#current and #next should return the same item for the same thread' do
46
+ @scheduler.current.should == 5
47
+ @scheduler.next.should == 7
48
+ Thread.new do
49
+ @scheduler.current.should == 5
50
+ @scheduler.next.should == 7
51
+ end.join
52
+ @scheduler.next.should == 4
53
+ end
54
+
55
+ end
56
+
57
+ end
58
+
@@ -0,0 +1,11 @@
1
+ require 'rubygems'
2
+ gem 'activerecord', '2.2.2'
3
+ %w[tlattr_accessors active_record yaml erb spec].each {|lib| require lib}
4
+
5
+ RAILS_ENV = ENV['RAILS_ENV'] = 'test'
6
+
7
+ MULTI_DB_SPEC_DIR = File.dirname(__FILE__)
8
+ MULTI_DB_SPEC_CONFIG = YAML::load(File.open(MULTI_DB_SPEC_DIR + '/config/database.yml'))
9
+
10
+ ActiveRecord::Base.logger = Logger.new(MULTI_DB_SPEC_DIR + "/debug.log")
11
+ ActiveRecord::Base.configurations = MULTI_DB_SPEC_CONFIG
metadata ADDED
@@ -0,0 +1,92 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: multi_db
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.2.1
5
+ platform: ruby
6
+ authors:
7
+ - "Maximilian Sch\xC3\xB6fmann"
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+
12
+ date: 2009-03-11 00:00:00 +01:00
13
+ default_executable:
14
+ dependencies:
15
+ - !ruby/object:Gem::Dependency
16
+ name: activerecord
17
+ type: :runtime
18
+ version_requirement:
19
+ version_requirements: !ruby/object:Gem::Requirement
20
+ requirements:
21
+ - - ">="
22
+ - !ruby/object:Gem::Version
23
+ version: 2.1.0
24
+ version:
25
+ - !ruby/object:Gem::Dependency
26
+ name: tlattr_accessors
27
+ type: :runtime
28
+ version_requirement:
29
+ version_requirements: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - ">="
32
+ - !ruby/object:Gem::Version
33
+ version: 0.0.3
34
+ version:
35
+ description: Connection proxy for ActiveRecord for single master / multiple slave database deployments
36
+ email: max@pragmatic-it.de
37
+ executables: []
38
+
39
+ extensions: []
40
+
41
+ extra_rdoc_files:
42
+ - LICENSE
43
+ - README.rdoc
44
+ files:
45
+ - lib/multi_db.rb
46
+ - lib/multi_db/active_record_extensions.rb
47
+ - lib/multi_db/connection_proxy.rb
48
+ - lib/multi_db/observer_extensions.rb
49
+ - lib/multi_db/query_cache_compat.rb
50
+ - lib/multi_db/scheduler.rb
51
+ - LICENSE
52
+ - README.rdoc
53
+ - spec/config/database.yml
54
+ - spec/connection_proxy_spec.rb
55
+ - spec/scheduler_spec.rb
56
+ - spec/spec_helper.rb
57
+ - multi_db.gemspec
58
+ has_rdoc: true
59
+ homepage: http://github.com/schoefmax/multi_db
60
+ licenses: []
61
+
62
+ post_install_message:
63
+ rdoc_options:
64
+ - --line-numbers
65
+ - --inline-source
66
+ - --title
67
+ - multi_db
68
+ - --main
69
+ - README.rdoc
70
+ require_paths:
71
+ - lib
72
+ required_ruby_version: !ruby/object:Gem::Requirement
73
+ requirements:
74
+ - - ">="
75
+ - !ruby/object:Gem::Version
76
+ version: "0"
77
+ version:
78
+ required_rubygems_version: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - ">="
81
+ - !ruby/object:Gem::Version
82
+ version: "1.2"
83
+ version:
84
+ requirements: []
85
+
86
+ rubyforge_project: multi_db
87
+ rubygems_version: 1.3.5
88
+ signing_key:
89
+ specification_version: 2
90
+ summary: Connection proxy for ActiveRecord for single master / multiple slave database deployments
91
+ test_files: []
92
+