schoefmax-multi_db 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/LICENSE +20 -0
- data/README.rdoc +104 -0
- data/lib/multi_db.rb +5 -0
- data/lib/multi_db/active_record_extensions.rb +23 -0
- data/lib/multi_db/connection_proxy.rb +178 -0
- data/lib/multi_db/observer_extensions.rb +18 -0
- data/lib/multi_db/query_cache_compat.rb +21 -0
- data/lib/multi_db/scheduler.rb +39 -0
- data/multi_db.gemspec +33 -0
- data/spec/config/database.yml +20 -0
- data/spec/connection_proxy_spec.rb +119 -0
- data/spec/scheduler_spec.rb +44 -0
- data/spec/spec_helper.rb +9 -0
- metadata +84 -0
data/LICENSE
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
The MIT license:
|
|
2
|
+
Copyright (c) 2008 Maximilian Schöfmann
|
|
3
|
+
|
|
4
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
5
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
6
|
+
in the Software without restriction, including without limitation the rights
|
|
7
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
8
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
9
|
+
furnished to do so, subject to the following conditions:
|
|
10
|
+
|
|
11
|
+
The above copyright notice and this permission notice shall be included in
|
|
12
|
+
all copies or substantial portions of the Software.
|
|
13
|
+
|
|
14
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
15
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
16
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
17
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
18
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
19
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
20
|
+
THE SOFTWARE.
|
data/README.rdoc
ADDED
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
= multi_db
|
|
2
|
+
|
|
3
|
+
=====-- This Plugin was inspired by Rick Olson's "masochism"-Plugin
|
|
4
|
+
|
|
5
|
+
multi_db uses a connection proxy, which sends read queries to slave databases,
|
|
6
|
+
and all write queries to the master database (Read/Write Split).
|
|
7
|
+
Within transactions, while executing ActiveRecord Observers and
|
|
8
|
+
within "with_master" blocks (see below), even read queries are sent to the
|
|
9
|
+
master database.
|
|
10
|
+
|
|
11
|
+
=== Caveats
|
|
12
|
+
|
|
13
|
+
* works with activerecord 2.1 - 2.2
|
|
14
|
+
|
|
15
|
+
=== Install
|
|
16
|
+
|
|
17
|
+
gem install schoefmax-multi_db --source http://gems.github.com
|
|
18
|
+
|
|
19
|
+
In Rails 2.1, add this to your environment.rb:
|
|
20
|
+
|
|
21
|
+
config.gem 'schoefmax-multi_db', :lib => 'multi_db', :source => 'http://gems.github.com'
|
|
22
|
+
|
|
23
|
+
=== Setup
|
|
24
|
+
|
|
25
|
+
In your database.yml, add sections for the slaves, e.g.:
|
|
26
|
+
|
|
27
|
+
production: # that would be the master
|
|
28
|
+
adapter: mysql
|
|
29
|
+
database: myapp_production
|
|
30
|
+
username: root
|
|
31
|
+
password:
|
|
32
|
+
host: localhost
|
|
33
|
+
|
|
34
|
+
production_slave_database: # that would be a slave
|
|
35
|
+
adapter: mysql
|
|
36
|
+
database: myapp_production
|
|
37
|
+
username: root
|
|
38
|
+
password:
|
|
39
|
+
host: 10.0.0.2
|
|
40
|
+
|
|
41
|
+
production_slave_database2: # another slave
|
|
42
|
+
...
|
|
43
|
+
production_slave_database_some_server: # yet another one
|
|
44
|
+
...
|
|
45
|
+
|
|
46
|
+
*NOTE*: multi_db identifies slave databases by looking for "slave_database"
|
|
47
|
+
somewhere in the database name!
|
|
48
|
+
|
|
49
|
+
To enable the proxy globally, add this to your environment.rb, or some file in
|
|
50
|
+
config/initializers:
|
|
51
|
+
|
|
52
|
+
MultiDb::ConnectionProxy.setup!
|
|
53
|
+
|
|
54
|
+
If you only want to enable it for specific environments, add this to
|
|
55
|
+
the corresponding file in config/environments:
|
|
56
|
+
|
|
57
|
+
config.after_initialize do
|
|
58
|
+
MultiDb::ConnectionProxy.setup!
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
In the development and test environments, you can use the same configuration
|
|
62
|
+
for your master and slave databases.
|
|
63
|
+
|
|
64
|
+
=== Differences to "masochism":
|
|
65
|
+
|
|
66
|
+
* Support for multiple slave connections (round robin)
|
|
67
|
+
* It sends anything except "select ..." queries to the master, instead of
|
|
68
|
+
sending only specific things to the master and anything "else" to the slave,
|
|
69
|
+
which is a lot more dangerous (e.g. "execute" wasn't sent to the master in
|
|
70
|
+
earlier versions of masochism)
|
|
71
|
+
* It sends everything coming from AR-Observers to the master, to avoid race
|
|
72
|
+
conditions (idea from one of the commenters on a blog entry about masochism)
|
|
73
|
+
* It uses its own query cache (with masochism, the slave's cache isn't emptied
|
|
74
|
+
when there are changes on the master)
|
|
75
|
+
* It supports immediate failover for slave connections
|
|
76
|
+
* It will wait some time before trying to query a failed slave database again
|
|
77
|
+
* It supports nested "with_master"-blocks (in masochism, nesting such blocks
|
|
78
|
+
would unexpectedly switch you to the slave again)
|
|
79
|
+
* It schedules a reconnect on the master connection to avoid problems
|
|
80
|
+
with virtual, migrating IPs for the master (e.g. multi-master HA setups)
|
|
81
|
+
* It's possible to specify slave_database instead of master_database which
|
|
82
|
+
makes migration between with and without multi_db less dangerous
|
|
83
|
+
* It allows environment specific settings for different slave setups
|
|
84
|
+
* It doesn't come with set_to_master! and set_to_slave!, as these are
|
|
85
|
+
considered dangerous (and make no sense) in a multi-slave setup. Instead of
|
|
86
|
+
set_to_master!, use with_master { code }
|
|
87
|
+
|
|
88
|
+
|
|
89
|
+
=== Contributors
|
|
90
|
+
|
|
91
|
+
* Matt Conway http://github.com/wr0ngway
|
|
92
|
+
* Matthias Marshall http://github.com/webops
|
|
93
|
+
|
|
94
|
+
=== Running specs
|
|
95
|
+
|
|
96
|
+
If you haven't already, install the rspec gem, then create an empty database
|
|
97
|
+
called "multi_db_test" (you might want to tweak the spec/config/database.yml).
|
|
98
|
+
From the plugin directory, run:
|
|
99
|
+
|
|
100
|
+
spec spec
|
|
101
|
+
|
|
102
|
+
|
|
103
|
+
Copyright (c) 2008, Max Schoefmann <max (a) pragmatic-it de>
|
|
104
|
+
Released under the MIT license
|
data/lib/multi_db.rb
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
module MultiDb
|
|
2
|
+
module ActiveRecordExtensions
|
|
3
|
+
def self.included(base)
|
|
4
|
+
base.alias_method_chain :reload, :master
|
|
5
|
+
|
|
6
|
+
class << base
|
|
7
|
+
def connection_proxy=(proxy)
|
|
8
|
+
@@connection_proxy = proxy
|
|
9
|
+
end
|
|
10
|
+
|
|
11
|
+
# hijack the original method
|
|
12
|
+
def connection
|
|
13
|
+
@@connection_proxy
|
|
14
|
+
end
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
def reload_with_master(*args, &block)
|
|
20
|
+
connection.with_master { reload_without_master }
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
end
|
|
@@ -0,0 +1,178 @@
|
|
|
1
|
+
module MultiDb
|
|
2
|
+
class ConnectionProxy
|
|
3
|
+
include MultiDb::QueryCacheCompat
|
|
4
|
+
include ActiveRecord::ConnectionAdapters::QueryCache
|
|
5
|
+
|
|
6
|
+
# Safe methods are those that should either go to the slave ONLY or go
|
|
7
|
+
# to the current active connection.
|
|
8
|
+
SAFE_METHODS = [ :select_all, :select_one, :select_value, :select_values,
|
|
9
|
+
:select_rows, :select, :verify!, :raw_connection, :active?, :reconnect!,
|
|
10
|
+
:disconnect!, :reset_runtime, :log, :log_info ]
|
|
11
|
+
|
|
12
|
+
attr_accessor :master
|
|
13
|
+
attr_accessor :current
|
|
14
|
+
|
|
15
|
+
class << self
|
|
16
|
+
|
|
17
|
+
# Example:
|
|
18
|
+
# MultiDb::ConnectionProxy.configuration = {
|
|
19
|
+
# :production_master_database => {
|
|
20
|
+
# :adapter =>
|
|
21
|
+
# }
|
|
22
|
+
# }
|
|
23
|
+
# defaults to the content of Rails.root/config/database.yml if not set and used with Rails
|
|
24
|
+
attr_accessor :configuration
|
|
25
|
+
# defaults to RAILS_ENV if multi_db is used with Rails
|
|
26
|
+
attr_accessor :environment
|
|
27
|
+
|
|
28
|
+
def setup!
|
|
29
|
+
# if no configuration was set, we assume that rails is used and load the database.yml
|
|
30
|
+
unless self.configuration
|
|
31
|
+
raise "RAILS_ROOT not set. Set MultiDb::ConnectionProxy.configuration manually if you use multi_db outside rails" unless defined?(RAILS_ROOT)
|
|
32
|
+
self.configuration = YAML.load(ERB.new(File.read(File.join(RAILS_ROOT, 'config', 'database.yml'))).result)
|
|
33
|
+
end
|
|
34
|
+
unless self.environment
|
|
35
|
+
raise "RAILS_ENV not set. Set MultiDb::ConnectionProxy.environment manually if you use multi_db outside rails" unless defined?(RAILS_ENV)
|
|
36
|
+
self.environment = RAILS_ENV
|
|
37
|
+
end
|
|
38
|
+
master = ActiveRecord::Base
|
|
39
|
+
master.send :include, MultiDb::ActiveRecordExtensions
|
|
40
|
+
slaves = init_slaves
|
|
41
|
+
raise "No slaves defined in database configuration" if slaves.empty?
|
|
42
|
+
slaves.each {|slave| slave.send :include, MultiDb::ActiveRecordExtensions}
|
|
43
|
+
ActiveRecord::Observer.send :include, MultiDb::ObserverExtensions
|
|
44
|
+
master.connection_proxy = new(master, slaves)
|
|
45
|
+
master.logger.info("** multi_db with master and #{slaves.length} slave#{"s" if slaves.length > 1} loaded.")
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
# Slave entries in the database.yml must be named like this
|
|
49
|
+
# development_slave_database:
|
|
50
|
+
# or
|
|
51
|
+
# development_slave_database1:
|
|
52
|
+
# or
|
|
53
|
+
# production_slave_database_someserver:
|
|
54
|
+
# These would be available later as MultiDb::SlaveDatabaseSomeserver
|
|
55
|
+
def init_slaves
|
|
56
|
+
returning([]) do |slaves|
|
|
57
|
+
self.configuration.keys.each do |name|
|
|
58
|
+
if name.to_s =~ /#{self.environment}_(slave_database.*)/
|
|
59
|
+
MultiDb.module_eval %Q{
|
|
60
|
+
class #{$1.camelize} < ActiveRecord::Base
|
|
61
|
+
self.abstract_class = true
|
|
62
|
+
establish_connection :#{name}
|
|
63
|
+
end
|
|
64
|
+
}, __FILE__, __LINE__
|
|
65
|
+
slaves << "MultiDb::#{$1.camelize}".constantize
|
|
66
|
+
end
|
|
67
|
+
end
|
|
68
|
+
end
|
|
69
|
+
end
|
|
70
|
+
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
def initialize(master, slaves)
|
|
74
|
+
@slaves = Scheduler.new(slaves)
|
|
75
|
+
@master = master
|
|
76
|
+
@current = @slaves.current
|
|
77
|
+
@reconnect = false
|
|
78
|
+
@with_master = 0
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
def slave
|
|
82
|
+
@slaves.current
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
def with_master
|
|
86
|
+
@current = @master
|
|
87
|
+
@with_master += 1
|
|
88
|
+
yield
|
|
89
|
+
ensure
|
|
90
|
+
@with_master -= 1
|
|
91
|
+
@current = @slaves.current if @with_master == 0
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
def transaction(start_db_transaction = true, &block)
|
|
95
|
+
with_master { get_connection(@current).transaction(start_db_transaction, &block) }
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
# Calls the method on master/slave and dynamically creates a new
|
|
99
|
+
# method on success to speed up subsequent calls
|
|
100
|
+
def method_missing(method, *args, &block)
|
|
101
|
+
returning(send(target_method(method), method, *args, &block)) do
|
|
102
|
+
create_delegation_method!(method)
|
|
103
|
+
end
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
protected
|
|
107
|
+
|
|
108
|
+
def get_connection(db_class)
|
|
109
|
+
db_class.retrieve_connection
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
def create_delegation_method!(method)
|
|
113
|
+
self.instance_eval %Q{
|
|
114
|
+
def #{method}(*args, &block)
|
|
115
|
+
#{'next_reader!' unless unsafe?(method)}
|
|
116
|
+
#{target_method(method)}(:#{method}, *args, &block)
|
|
117
|
+
end
|
|
118
|
+
}
|
|
119
|
+
end
|
|
120
|
+
|
|
121
|
+
def target_method(method)
|
|
122
|
+
unsafe?(method) ? :send_to_master : :send_to_current
|
|
123
|
+
end
|
|
124
|
+
|
|
125
|
+
def send_to_master(method, *args, &block)
|
|
126
|
+
reconnect_master! if @reconnect
|
|
127
|
+
get_connection(@master).send(method, *args, &block)
|
|
128
|
+
rescue => e
|
|
129
|
+
raise_master_error(e)
|
|
130
|
+
end
|
|
131
|
+
|
|
132
|
+
def send_to_current(method, *args, &block)
|
|
133
|
+
reconnect_master! if @reconnect && master?
|
|
134
|
+
get_connection(@current).send(method, *args, &block)
|
|
135
|
+
rescue NotImplementedError, NoMethodError
|
|
136
|
+
raise
|
|
137
|
+
rescue => e # TODO don't rescue everything
|
|
138
|
+
raise_master_error(e) if master?
|
|
139
|
+
logger.warn "[MULTIDB] Error reading from slave database"
|
|
140
|
+
logger.error %(#{e.message}\n#{e.backtrace.join("\n")})
|
|
141
|
+
@slaves.blacklist!(@current)
|
|
142
|
+
next_reader!
|
|
143
|
+
retry
|
|
144
|
+
end
|
|
145
|
+
|
|
146
|
+
def next_reader!
|
|
147
|
+
return if @with_master > 0 # don't if in with_master block
|
|
148
|
+
@current = @slaves.next
|
|
149
|
+
rescue Scheduler::NoMoreItems
|
|
150
|
+
logger.warn "[MULTIDB] All slaves are blacklisted. Reading from master"
|
|
151
|
+
@current = @master
|
|
152
|
+
end
|
|
153
|
+
|
|
154
|
+
def reconnect_master!
|
|
155
|
+
get_connection(@master).reconnect!
|
|
156
|
+
@reconnect = false
|
|
157
|
+
end
|
|
158
|
+
|
|
159
|
+
def raise_master_error(error)
|
|
160
|
+
logger.fatal "[MULTIDB] Error accessing master database. Scheduling reconnect"
|
|
161
|
+
@reconnect = true
|
|
162
|
+
raise error
|
|
163
|
+
end
|
|
164
|
+
|
|
165
|
+
def unsafe?(method)
|
|
166
|
+
!SAFE_METHODS.include?(method)
|
|
167
|
+
end
|
|
168
|
+
|
|
169
|
+
def master?
|
|
170
|
+
@current == @master
|
|
171
|
+
end
|
|
172
|
+
|
|
173
|
+
def logger
|
|
174
|
+
ActiveRecord::Base.logger
|
|
175
|
+
end
|
|
176
|
+
|
|
177
|
+
end
|
|
178
|
+
end
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
module MultiDb
|
|
2
|
+
module ObserverExtensions
|
|
3
|
+
def self.included(base)
|
|
4
|
+
base.alias_method_chain :update, :masterdb
|
|
5
|
+
end
|
|
6
|
+
|
|
7
|
+
# Send observed_method(object) if the method exists.
|
|
8
|
+
def update_with_masterdb(observed_method, object) #:nodoc:
|
|
9
|
+
if object.class.connection.respond_to?(:with_master)
|
|
10
|
+
object.class.connection.with_master do
|
|
11
|
+
update_without_masterdb(observed_method, object)
|
|
12
|
+
end
|
|
13
|
+
else
|
|
14
|
+
update_without_masterdb(observed_method, object)
|
|
15
|
+
end
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
end
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
module MultiDb
|
|
2
|
+
# Implements the methods expected by the QueryCache module
|
|
3
|
+
module QueryCacheCompat
|
|
4
|
+
def select_all(*a, &b)
|
|
5
|
+
next_reader!
|
|
6
|
+
send_to_current(:select_all, *a, &b)
|
|
7
|
+
end
|
|
8
|
+
def columns(*a, &b)
|
|
9
|
+
send_to_current(:columns, *a, &b)
|
|
10
|
+
end
|
|
11
|
+
def insert(*a, &b)
|
|
12
|
+
send_to_master(:insert, *a, &b)
|
|
13
|
+
end
|
|
14
|
+
def update(*a, &b)
|
|
15
|
+
send_to_master(:update, *a, &b)
|
|
16
|
+
end
|
|
17
|
+
def delete(*a, &b)
|
|
18
|
+
send_to_master(:delete, *a, &b)
|
|
19
|
+
end
|
|
20
|
+
end
|
|
21
|
+
end
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
module MultiDb
|
|
2
|
+
class Scheduler
|
|
3
|
+
class NoMoreItems < Exception; end
|
|
4
|
+
|
|
5
|
+
attr :items
|
|
6
|
+
delegate :[], :[]=, :to => :items
|
|
7
|
+
|
|
8
|
+
def initialize(items, blacklist_timeout = 1.minute)
|
|
9
|
+
@n = items.length
|
|
10
|
+
@items = items
|
|
11
|
+
@blacklist = Array.new(@n, Time.at(0))
|
|
12
|
+
@current = 0
|
|
13
|
+
@blacklist_timeout = blacklist_timeout
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
def blacklist!(item)
|
|
17
|
+
@blacklist[@items.index(item)] = Time.now
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def current
|
|
21
|
+
@items[@current]
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
def next
|
|
25
|
+
previous = @current
|
|
26
|
+
until(@blacklist[next_index!] < Time.now - @blacklist_timeout) do
|
|
27
|
+
raise NoMoreItems, 'All items are blacklisted' if @current == previous
|
|
28
|
+
end
|
|
29
|
+
@items[@current]
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
protected
|
|
33
|
+
|
|
34
|
+
def next_index!
|
|
35
|
+
@current = (@current + 1) % @n
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
end
|
|
39
|
+
end
|
data/multi_db.gemspec
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# -*- encoding: utf-8 -*-
|
|
2
|
+
|
|
3
|
+
Gem::Specification.new do |s|
|
|
4
|
+
s.name = %q{multi_db}
|
|
5
|
+
s.version = "0.1.1"
|
|
6
|
+
|
|
7
|
+
s.required_rubygems_version = Gem::Requirement.new(">= 1.2") if s.respond_to? :required_rubygems_version=
|
|
8
|
+
s.authors = ["Maximilian Sch\303\266fmann"]
|
|
9
|
+
s.date = %q{2009-01-29}
|
|
10
|
+
s.description = "Connection proxy for ActiveRecord for single master / multiple slave database deployments"
|
|
11
|
+
s.email = "max@pragmatic-it.de"
|
|
12
|
+
s.extra_rdoc_files = ["lib/multi_db/active_record_extensions.rb", "lib/multi_db/connection_proxy.rb", "lib/multi_db/observer_extensions.rb", "lib/multi_db/query_cache_compat.rb", "lib/multi_db/scheduler.rb", "LICENSE", "README.rdoc"]
|
|
13
|
+
s.files = ["lib/multi_db.rb", "lib/multi_db/active_record_extensions.rb", "lib/multi_db/connection_proxy.rb", "lib/multi_db/observer_extensions.rb", "lib/multi_db/query_cache_compat.rb", "lib/multi_db/scheduler.rb", "LICENSE", "README.rdoc", "spec/config/database.yml", "spec/connection_proxy_spec.rb", "spec/scheduler_spec.rb", "spec/spec_helper.rb", "multi_db.gemspec"]
|
|
14
|
+
s.has_rdoc = true
|
|
15
|
+
s.homepage = "http://github.com/schoefmax/multi_db"
|
|
16
|
+
s.rdoc_options = ["--line-numbers", "--inline-source", "--title", "multi_db", "--main", "README.rdoc"]
|
|
17
|
+
s.require_paths = ["lib"]
|
|
18
|
+
s.rubyforge_project = "multi_db"
|
|
19
|
+
s.rubygems_version = %q{1.3.1}
|
|
20
|
+
s.summary = "Connection proxy for ActiveRecord for single master / multiple slave database deployments"
|
|
21
|
+
|
|
22
|
+
if s.respond_to? :specification_version then
|
|
23
|
+
s.specification_version = 2
|
|
24
|
+
|
|
25
|
+
if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
|
|
26
|
+
s.add_runtime_dependency('activerecord', [">= 2.1.0"])
|
|
27
|
+
else
|
|
28
|
+
s.add_dependency('activerecord', [">= 2.1.0"])
|
|
29
|
+
end
|
|
30
|
+
else
|
|
31
|
+
s.add_dependency('activerecord', [">= 2.1.0"])
|
|
32
|
+
end
|
|
33
|
+
end
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
test:
|
|
2
|
+
adapter: mysql
|
|
3
|
+
database: multi_db_test
|
|
4
|
+
username: root
|
|
5
|
+
password:
|
|
6
|
+
host: 127.0.0.1
|
|
7
|
+
|
|
8
|
+
test_slave_database_1:
|
|
9
|
+
adapter: mysql
|
|
10
|
+
database: multi_db_test
|
|
11
|
+
username: root
|
|
12
|
+
password:
|
|
13
|
+
host: 127.0.0.1
|
|
14
|
+
|
|
15
|
+
test_slave_database_2:
|
|
16
|
+
adapter: mysql
|
|
17
|
+
database: multi_db_test
|
|
18
|
+
username: root
|
|
19
|
+
password:
|
|
20
|
+
host: 127.0.0.1
|
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
|
|
2
|
+
require MULTI_DB_SPEC_DIR + '/../lib/multi_db/query_cache_compat'
|
|
3
|
+
require MULTI_DB_SPEC_DIR + '/../lib/multi_db/active_record_extensions'
|
|
4
|
+
require MULTI_DB_SPEC_DIR + '/../lib/multi_db/observer_extensions'
|
|
5
|
+
require MULTI_DB_SPEC_DIR + '/../lib/multi_db/scheduler'
|
|
6
|
+
require MULTI_DB_SPEC_DIR + '/../lib/multi_db/connection_proxy'
|
|
7
|
+
|
|
8
|
+
RAILS_ROOT = MULTI_DB_SPEC_DIR
|
|
9
|
+
|
|
10
|
+
describe MultiDb::ConnectionProxy do
|
|
11
|
+
|
|
12
|
+
before(:all) do
|
|
13
|
+
ActiveRecord::Base.establish_connection(MULTI_DB_SPEC_CONFIG['test'])
|
|
14
|
+
MultiDb::ConnectionProxy.setup!
|
|
15
|
+
@proxy = ActiveRecord::Base.connection
|
|
16
|
+
@sql = 'SELECT 1 + 1 FROM DUAL'
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
it "should generate classes for each entry in the database.yml" do
|
|
20
|
+
defined?(MultiDb::SlaveDatabase1).should_not be_nil
|
|
21
|
+
defined?(MultiDb::SlaveDatabase2).should_not be_nil
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
it 'should handle nested with_master-blocks correctly' do
|
|
25
|
+
@proxy.current.should_not == @proxy.master
|
|
26
|
+
@proxy.with_master do
|
|
27
|
+
@proxy.current.should == @proxy.master
|
|
28
|
+
@proxy.with_master do
|
|
29
|
+
@proxy.current.should == @proxy.master
|
|
30
|
+
@proxy.with_master do
|
|
31
|
+
@proxy.current.should == @proxy.master
|
|
32
|
+
end
|
|
33
|
+
@proxy.current.should == @proxy.master
|
|
34
|
+
end
|
|
35
|
+
@proxy.current.should == @proxy.master
|
|
36
|
+
end
|
|
37
|
+
@proxy.current.should_not == @proxy.master
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
it 'should perform transactions on the master' do
|
|
41
|
+
@proxy.master.retrieve_connection.should_receive(:select_all).exactly(1) # makes sure the first one goes to a slave
|
|
42
|
+
@proxy.select_all(@sql)
|
|
43
|
+
ActiveRecord::Base.transaction do
|
|
44
|
+
@proxy.select_all(@sql)
|
|
45
|
+
end
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
it 'should switch to the next reader on selects' do
|
|
49
|
+
MultiDb::SlaveDatabase1.retrieve_connection.should_receive(:select_one).twice
|
|
50
|
+
MultiDb::SlaveDatabase2.retrieve_connection.should_receive(:select_one).twice
|
|
51
|
+
4.times { @proxy.select_one(@sql) }
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
it 'should not switch to the next reader when whithin a with_master-block' do
|
|
55
|
+
@proxy.master.retrieve_connection.should_receive(:select_one).twice
|
|
56
|
+
MultiDb::SlaveDatabase1.retrieve_connection.should_not_receive(:select_one)
|
|
57
|
+
MultiDb::SlaveDatabase2.retrieve_connection.should_not_receive(:select_one)
|
|
58
|
+
@proxy.with_master do
|
|
59
|
+
2.times { @proxy.select_one(@sql) }
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
it 'should send dangerous methods to the master' do
|
|
64
|
+
meths = [:insert, :update, :delete, :execute]
|
|
65
|
+
meths.each do |meth|
|
|
66
|
+
MultiDb::SlaveDatabase1.retrieve_connection.stub!(meth).and_raise(RuntimeError)
|
|
67
|
+
@proxy.master.retrieve_connection.should_receive(meth).and_return(true)
|
|
68
|
+
@proxy.send(meth, @sql)
|
|
69
|
+
end
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
it 'should dynamically generate safe methods' do
|
|
73
|
+
@proxy.should_not respond_to(:select_value)
|
|
74
|
+
@proxy.select_value(@sql)
|
|
75
|
+
@proxy.should respond_to(:select_value)
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
it 'should cache queries using select_all' do
|
|
79
|
+
ActiveRecord::Base.cache do
|
|
80
|
+
# next_reader will be called and switch to the SlaveDatabase2
|
|
81
|
+
MultiDb::SlaveDatabase2.retrieve_connection.should_receive(:select_all).exactly(1)
|
|
82
|
+
MultiDb::SlaveDatabase1.retrieve_connection.should_not_receive(:select_all)
|
|
83
|
+
@proxy.master.retrieve_connection.should_not_receive(:select_all)
|
|
84
|
+
3.times { @proxy.select_all(@sql) }
|
|
85
|
+
end
|
|
86
|
+
end
|
|
87
|
+
|
|
88
|
+
it 'should invalidate the cache on insert, delete and update' do
|
|
89
|
+
ActiveRecord::Base.cache do
|
|
90
|
+
meths = [:insert, :update, :delete]
|
|
91
|
+
meths.each do |meth|
|
|
92
|
+
@proxy.master.retrieve_connection.should_receive(meth).and_return(true)
|
|
93
|
+
end
|
|
94
|
+
MultiDb::SlaveDatabase2.retrieve_connection.should_receive(:select_all).twice
|
|
95
|
+
MultiDb::SlaveDatabase1.retrieve_connection.should_receive(:select_all).once
|
|
96
|
+
3.times do |i|
|
|
97
|
+
@proxy.select_all(@sql)
|
|
98
|
+
@proxy.send(meths[i])
|
|
99
|
+
end
|
|
100
|
+
end
|
|
101
|
+
end
|
|
102
|
+
|
|
103
|
+
it 'should retry the next slave when one fails and finally fall back to the master' do
|
|
104
|
+
MultiDb::SlaveDatabase1.retrieve_connection.should_receive(:select_all).once.and_raise(RuntimeError)
|
|
105
|
+
MultiDb::SlaveDatabase2.retrieve_connection.should_receive(:select_all).once.and_raise(RuntimeError)
|
|
106
|
+
@proxy.master.retrieve_connection.should_receive(:select_all).and_return(true)
|
|
107
|
+
@proxy.select_all(@sql)
|
|
108
|
+
end
|
|
109
|
+
|
|
110
|
+
it 'should try to reconnect the master connection after the master has failed' do
|
|
111
|
+
@proxy.master.retrieve_connection.should_receive(:update).and_raise(RuntimeError)
|
|
112
|
+
lambda { @proxy.update(@sql) }.should raise_error
|
|
113
|
+
@proxy.master.retrieve_connection.should_receive(:reconnect!).and_return(true)
|
|
114
|
+
@proxy.master.retrieve_connection.should_receive(:insert).and_return(1)
|
|
115
|
+
@proxy.insert(@sql)
|
|
116
|
+
end
|
|
117
|
+
|
|
118
|
+
end
|
|
119
|
+
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
|
|
2
|
+
require MULTI_DB_SPEC_DIR + '/../lib/multi_db/scheduler'
|
|
3
|
+
|
|
4
|
+
describe MultiDb::Scheduler do
|
|
5
|
+
|
|
6
|
+
before do
|
|
7
|
+
@items = [5, 2, 4, 8]
|
|
8
|
+
@scheduler = MultiDb::Scheduler.new(@items.clone)
|
|
9
|
+
end
|
|
10
|
+
|
|
11
|
+
it "should return items in a round robin fashion" do
|
|
12
|
+
first = @items.shift
|
|
13
|
+
@scheduler.current.should == first
|
|
14
|
+
@items.each do |item|
|
|
15
|
+
@scheduler.next.should == item
|
|
16
|
+
end
|
|
17
|
+
@scheduler.next.should == first
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
it 'should not return blacklisted items' do
|
|
21
|
+
@scheduler.blacklist!(4)
|
|
22
|
+
@items.size.times do
|
|
23
|
+
@scheduler.next.should_not == 4
|
|
24
|
+
end
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
it 'should raise NoMoreItems if all are blacklisted' do
|
|
28
|
+
@items.each do |item|
|
|
29
|
+
@scheduler.blacklist!(item)
|
|
30
|
+
end
|
|
31
|
+
lambda {
|
|
32
|
+
@scheduler.next
|
|
33
|
+
}.should raise_error(MultiDb::Scheduler::NoMoreItems)
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
it 'should unblacklist items automatically' do
|
|
37
|
+
@scheduler = MultiDb::Scheduler.new(@items.clone, 1.second)
|
|
38
|
+
@scheduler.blacklist!(2)
|
|
39
|
+
sleep(1)
|
|
40
|
+
@scheduler.next.should == 2
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
end
|
|
44
|
+
|
data/spec/spec_helper.rb
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
%w[rubygems active_record yaml erb spec].each {|lib| require lib}
|
|
2
|
+
|
|
3
|
+
RAILS_ENV = ENV['RAILS_ENV'] = 'test'
|
|
4
|
+
|
|
5
|
+
MULTI_DB_SPEC_DIR = File.dirname(__FILE__)
|
|
6
|
+
MULTI_DB_SPEC_CONFIG = YAML::load(File.open(MULTI_DB_SPEC_DIR + '/config/database.yml'))
|
|
7
|
+
|
|
8
|
+
ActiveRecord::Base.logger = Logger.new(MULTI_DB_SPEC_DIR + "/debug.log")
|
|
9
|
+
ActiveRecord::Base.configurations = MULTI_DB_SPEC_CONFIG
|
metadata
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: schoefmax-multi_db
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.1
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- "Maximilian Sch\xC3\xB6fmann"
|
|
8
|
+
autorequire:
|
|
9
|
+
bindir: bin
|
|
10
|
+
cert_chain: []
|
|
11
|
+
|
|
12
|
+
date: 2009-01-29 00:00:00 -08:00
|
|
13
|
+
default_executable:
|
|
14
|
+
dependencies:
|
|
15
|
+
- !ruby/object:Gem::Dependency
|
|
16
|
+
name: activerecord
|
|
17
|
+
version_requirement:
|
|
18
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
19
|
+
requirements:
|
|
20
|
+
- - ">="
|
|
21
|
+
- !ruby/object:Gem::Version
|
|
22
|
+
version: 2.1.0
|
|
23
|
+
version:
|
|
24
|
+
description: Connection proxy for ActiveRecord for single master / multiple slave database deployments
|
|
25
|
+
email: max@pragmatic-it.de
|
|
26
|
+
executables: []
|
|
27
|
+
|
|
28
|
+
extensions: []
|
|
29
|
+
|
|
30
|
+
extra_rdoc_files:
|
|
31
|
+
- lib/multi_db/active_record_extensions.rb
|
|
32
|
+
- lib/multi_db/connection_proxy.rb
|
|
33
|
+
- lib/multi_db/observer_extensions.rb
|
|
34
|
+
- lib/multi_db/query_cache_compat.rb
|
|
35
|
+
- lib/multi_db/scheduler.rb
|
|
36
|
+
- LICENSE
|
|
37
|
+
- README.rdoc
|
|
38
|
+
files:
|
|
39
|
+
- lib/multi_db.rb
|
|
40
|
+
- lib/multi_db/active_record_extensions.rb
|
|
41
|
+
- lib/multi_db/connection_proxy.rb
|
|
42
|
+
- lib/multi_db/observer_extensions.rb
|
|
43
|
+
- lib/multi_db/query_cache_compat.rb
|
|
44
|
+
- lib/multi_db/scheduler.rb
|
|
45
|
+
- LICENSE
|
|
46
|
+
- README.rdoc
|
|
47
|
+
- spec/config/database.yml
|
|
48
|
+
- spec/connection_proxy_spec.rb
|
|
49
|
+
- spec/scheduler_spec.rb
|
|
50
|
+
- spec/spec_helper.rb
|
|
51
|
+
- multi_db.gemspec
|
|
52
|
+
has_rdoc: true
|
|
53
|
+
homepage: http://github.com/schoefmax/multi_db
|
|
54
|
+
post_install_message:
|
|
55
|
+
rdoc_options:
|
|
56
|
+
- --line-numbers
|
|
57
|
+
- --inline-source
|
|
58
|
+
- --title
|
|
59
|
+
- multi_db
|
|
60
|
+
- --main
|
|
61
|
+
- README.rdoc
|
|
62
|
+
require_paths:
|
|
63
|
+
- lib
|
|
64
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
65
|
+
requirements:
|
|
66
|
+
- - ">="
|
|
67
|
+
- !ruby/object:Gem::Version
|
|
68
|
+
version: "0"
|
|
69
|
+
version:
|
|
70
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
71
|
+
requirements:
|
|
72
|
+
- - ">="
|
|
73
|
+
- !ruby/object:Gem::Version
|
|
74
|
+
version: "1.2"
|
|
75
|
+
version:
|
|
76
|
+
requirements: []
|
|
77
|
+
|
|
78
|
+
rubyforge_project: multi_db
|
|
79
|
+
rubygems_version: 1.2.0
|
|
80
|
+
signing_key:
|
|
81
|
+
specification_version: 2
|
|
82
|
+
summary: Connection proxy for ActiveRecord for single master / multiple slave database deployments
|
|
83
|
+
test_files: []
|
|
84
|
+
|