makara 0.3.5 → 0.3.6
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +85 -17
- data/README.md +2 -0
- data/lib/active_record/connection_adapters/makara_abstract_adapter.rb +5 -12
- data/lib/makara.rb +6 -0
- data/lib/makara/error_handler.rb +1 -1
- data/lib/makara/pool.rb +43 -76
- data/lib/makara/proxy.rb +16 -3
- data/lib/makara/strategies/abstract.rb +29 -0
- data/lib/makara/strategies/priority_failover.rb +48 -0
- data/lib/makara/strategies/round_robin.rb +72 -0
- data/lib/makara/version.rb +1 -1
- data/spec/active_record/connection_adapters/makara_mysql2_adapter_spec.rb +1 -1
- data/spec/active_record/connection_adapters/makara_postgresql_adapter_spec.rb +1 -1
- data/spec/pool_spec.rb +8 -6
- data/spec/spec_helper.rb +1 -1
- data/spec/strategies/priority_failover_spec.rb +49 -0
- data/spec/strategies/round_robin_spec.rb +67 -0
- data/spec/support/mock_objects.rb +4 -2
- metadata +9 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 854be0f8b5615812158cd37511db68cc47310f75
|
4
|
+
data.tar.gz: 8cc3d705dd2d83415a8d4348d9ce2b949d0eb954
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 8669c261ead2cce40c1ffa8f319f975d162623071f54290fbed54e1a65159b2698c9eab709159cb6f71858b2d32b20ad9996b8472502c91a8086fdb89c08d31a
|
7
|
+
data.tar.gz: 2089f3e36f9add64c5ecdef2634665fde5aa3a2cc94ab4230f9f8099d9c438f7da057b73406b8c8f9abab8c596424ad4cd0deef627fff7831f0c0ca9f9004233
|
data/CHANGELOG.md
CHANGED
@@ -1,27 +1,95 @@
|
|
1
1
|
# Change Log
|
2
2
|
All notable changes to this project will be documented in this file.
|
3
3
|
|
4
|
-
##
|
5
|
-
### Added
|
6
|
-
- allow bypassing of stickiness
|
4
|
+
## v0.3.5 - 2016-01-08
|
7
5
|
|
8
|
-
|
9
|
-
### Added
|
10
|
-
- add postgres specific tests.
|
6
|
+
[Full Changelog](https://github.com/taskrabbit/makara/compare/v0.3.4.rc1...v0.3.5)
|
11
7
|
|
12
|
-
|
13
|
-
- change using methods for matchers to be able to monkey patch them
|
14
|
-
- follow AR naming conventions for adapter naming
|
8
|
+
Changed
|
15
9
|
|
16
|
-
|
17
|
-
|
18
|
-
|
10
|
+
- Raise `Makara::Errors::AllConnectionsBlacklisted` on timeout. [#104](https://github.com/taskrabbit/makara/pull/104) Brian Leonard
|
11
|
+
|
12
|
+
## v0.3.4.rc1 - 2016-01-06
|
13
|
+
|
14
|
+
[Full Changelog](https://github.com/taskrabbit/makara/compare/v0.3.3...v0.3.4.rc1)
|
15
|
+
|
16
|
+
Added
|
17
|
+
|
18
|
+
- Add `url` to database connections configurations. [#93](https://github.com/taskrabbit/makara/pull/93) Benjamin Fleischer
|
19
|
+
|
20
|
+
Changed
|
21
|
+
|
22
|
+
- Improve Postgresql compatibility and failover support, also fix [#78](https://github.com/taskrabbit/makara/issues/78), [#79](https://github.com/taskrabbit/makara/issues/79). [#87](https://github.com/taskrabbit/makara/pull/87) Vlad
|
23
|
+
- Update README: Specify newrelic_rpm gem versions that will have the performance issue. [#95](https://github.com/taskrabbit/makara/pull/95) Benjamin Fleischer
|
24
|
+
|
25
|
+
## v0.3.3 - 2015-05-20
|
26
|
+
|
27
|
+
[Full Changelog](https://github.com/taskrabbit/makara/compare/v0.3.2...v0.3.3)
|
28
|
+
|
29
|
+
Changed
|
30
|
+
|
31
|
+
- A context is local to the curent thread of execution. This will allow you to stick to master safely in a single thread in systems such as sidekiq, for instance. Fix [#83](https://github.com/taskrabbit/makara/issues/83). [#84](https://github.com/taskrabbit/makara/pull/84) Matt Camuto
|
32
|
+
|
33
|
+
## v0.3.2 - 2015-05-16
|
34
|
+
|
35
|
+
[Full Changelog](https://github.com/taskrabbit/makara/compare/v0.3.1...v0.3.2)
|
36
|
+
|
37
|
+
Fixed
|
38
|
+
|
39
|
+
- Fix a `ArgumentError: not delegated` error for rails 3. [#82](https://github.com/taskrabbit/makara/pull/82) Eric Saxby
|
40
|
+
|
41
|
+
Changed
|
42
|
+
|
43
|
+
- Switch log format from `:info` to `:error`. Mike Nelson
|
44
|
+
|
45
|
+
## v0.3.1 - 2015-05-08
|
46
|
+
|
47
|
+
[Full Changelog](https://github.com/taskrabbit/makara/compare/v0.3.0...v0.3.1)
|
48
|
+
|
49
|
+
Changed
|
50
|
+
|
51
|
+
- Globally move to multiline matchers. Mike Nelson
|
52
|
+
|
53
|
+
Changed
|
54
|
+
|
55
|
+
## v0.3.0 - 2015-04-27
|
56
|
+
|
57
|
+
[Full Changelog](https://github.com/taskrabbit/makara/compare/v0.3.0.rc3...v0.3.0)
|
58
|
+
|
59
|
+
Changed
|
60
|
+
|
61
|
+
- Reduce logging noise by using [the same rules as ActiveRecord uses](https://github.com/rails/rails/blob/b06f64c3480cd389d14618540d62da4978918af0/activerecord/lib/active_record/log_subscriber.rb#L33). [#76](https://github.com/taskrabbit/makara/pull/76) Andrew Kane
|
62
|
+
|
63
|
+
Fixed
|
64
|
+
|
65
|
+
- Fix an issue for postgres that would route all queries to master. [#72](https://github.com/taskrabbit/makara/pull/72) Kali Donovan
|
66
|
+
- Fix an edge case which would cause SET operations to send to all connections([#70](https://github.com/taskrabbit/makara/issues/70)). [#80](https://github.com/taskrabbit/makara/pull/80) Michael Amor Righi
|
67
|
+
- Fix performance regression with certain verions of [newrelic/rpm](https://github.com/newrelic/rpm)([#59](https://github.com/taskrabbit/makara/issues/59)). [#75](https://github.com/taskrabbit/makara/pull/75) Mike Nelson
|
68
|
+
|
69
|
+
## 0.3.0.rc3 - 2014-09-02[YANKED]
|
70
|
+
|
71
|
+
[Full Changelog](https://github.com/taskrabbit/makara/compare/v0.3.0.rc2...v0.3.0.rc3)
|
72
|
+
|
73
|
+
Added
|
74
|
+
- Allow bypassing of stickiness
|
75
|
+
|
76
|
+
## 0.3.0.rc2 - 2014-08-05
|
77
|
+
Added
|
78
|
+
- Add postgres specific tests.
|
79
|
+
|
80
|
+
Changed
|
81
|
+
- Change using methods for matchers to be able to monkey patch them.
|
82
|
+
- Follow AR naming conventions for adapter naming.
|
83
|
+
|
84
|
+
## 0.3.0.rc1 - 2014-08-05
|
85
|
+
Removed
|
86
|
+
- Remove initial connection logic. If a connection can't be made on startup, an error will be thrown rather than the node getting blacklisted.
|
19
87
|
|
20
88
|
|
21
89
|
## 0.2.2 - 2014-04-03
|
22
|
-
|
23
|
-
-
|
90
|
+
Added
|
91
|
+
- Add logging of makara operations via the Makara::Logger.
|
24
92
|
|
25
|
-
|
26
|
-
-
|
27
|
-
-
|
93
|
+
Changed
|
94
|
+
- Begin tracing the series of errors associated with blacklisting rather than just the last. This becomes apparent in error messages.
|
95
|
+
- Fix Rails.cache usage when full environment is not loaded.
|
data/README.md
CHANGED
@@ -140,6 +140,7 @@ production:
|
|
140
140
|
# the following are default values
|
141
141
|
blacklist_duration: 5
|
142
142
|
master_ttl: 5
|
143
|
+
master_strategy: round_robin
|
143
144
|
sticky: true
|
144
145
|
|
145
146
|
# list your connections with the override values (they're merged into the top-level config)
|
@@ -161,6 +162,7 @@ The makara subconfig sets up the proxy with a few of its own options, then provi
|
|
161
162
|
* blacklist_duration - the number of seconds a node is blacklisted when a connection failure occurs
|
162
163
|
* sticky - if a node should be stuck to once it's used during a specific context
|
163
164
|
* master_ttl - how long the master context is persisted. generally, this needs to be longer than any replication lag
|
165
|
+
* master_strategy - use a different strategy for picking the "current" node: `failover` will try to keep the same one until it is blacklisted. The default is `round_robin` which will cycle through available ones.
|
164
166
|
* connection_error_matchers - array of custom error matchers you want to be handled gracefully by Makara (as in, errors matching these regexes will result in blacklisting the connection as opposed to raising directly).
|
165
167
|
|
166
168
|
Connection definitions contain any extra node-specific configurations. If the node should behave as a master you must provide `role: master`. Any previous configurations can be overridden within a specific node's config. Nodes can also contain weights if you'd like to balance usage based on hardware specifications. Optionally, you can provide a name attribute which will be used in sql logging.
|
@@ -143,21 +143,14 @@ module ActiveRecord
|
|
143
143
|
protected
|
144
144
|
|
145
145
|
|
146
|
-
def appropriate_connection(method_name, args)
|
146
|
+
def appropriate_connection(method_name, args, &block)
|
147
147
|
if needed_by_all?(method_name, args)
|
148
148
|
|
149
149
|
handling_an_all_execution(method_name) do
|
150
|
-
|
151
|
-
|
152
|
-
|
153
|
-
|
154
|
-
end
|
155
|
-
end
|
156
|
-
|
157
|
-
@master_pool.provide_each do |con|
|
158
|
-
hijacked do
|
159
|
-
yield con
|
160
|
-
end
|
150
|
+
hijacked do
|
151
|
+
# slave pool must run first.
|
152
|
+
@slave_pool.send_to_all(nil, &block) # just yields to each con
|
153
|
+
@master_pool.send_to_all(nil, &block) # just yields to each con
|
161
154
|
end
|
162
155
|
end
|
163
156
|
|
data/lib/makara.rb
CHANGED
@@ -22,4 +22,10 @@ module Makara
|
|
22
22
|
autoload :Subscriber, 'makara/logging/subscriber'
|
23
23
|
end
|
24
24
|
|
25
|
+
module Strategies
|
26
|
+
autoload :Abstract, 'makara/strategies/abstract'
|
27
|
+
autoload :RoundRobin, 'makara/strategies/round_robin'
|
28
|
+
autoload :PriorityFailover, 'makara/strategies/priority_failover'
|
29
|
+
end
|
30
|
+
|
25
31
|
end
|
data/lib/makara/error_handler.rb
CHANGED
data/lib/makara/pool.rb
CHANGED
@@ -13,6 +13,7 @@ module Makara
|
|
13
13
|
attr_reader :blacklist_errors
|
14
14
|
attr_reader :role
|
15
15
|
attr_reader :connections
|
16
|
+
attr_reader :strategy
|
16
17
|
|
17
18
|
def initialize(role, proxy)
|
18
19
|
@role = role
|
@@ -20,8 +21,8 @@ module Makara
|
|
20
21
|
@context = Makara::Context.get_current
|
21
22
|
@connections = []
|
22
23
|
@blacklist_errors = []
|
23
|
-
@current_idx = 0
|
24
24
|
@disabled = false
|
25
|
+
@strategy = proxy.strategy_for(role)
|
25
26
|
end
|
26
27
|
|
27
28
|
|
@@ -47,47 +48,55 @@ module Makara
|
|
47
48
|
wrapper = Makara::ConnectionWrapper.new(@proxy, connection, config)
|
48
49
|
end
|
49
50
|
|
50
|
-
|
51
|
-
|
52
|
-
wrapper._makara_weight.times{ @connections << wrapper }
|
53
|
-
|
54
|
-
if should_shuffle?
|
55
|
-
# randomize the connections so we don't get peaks and valleys of load
|
56
|
-
@connections.shuffle!
|
57
|
-
|
58
|
-
# then start at a random spot in the list
|
59
|
-
@current_idx = rand(@connections.length)
|
60
|
-
end
|
51
|
+
@connections << wrapper
|
52
|
+
@strategy.connection_added(wrapper)
|
61
53
|
|
62
54
|
wrapper
|
63
55
|
end
|
64
56
|
|
65
57
|
# send this method to all available nodes
|
66
|
-
|
58
|
+
# send nil to just yield with each con if there is block
|
59
|
+
def send_to_all(method, *args, &block)
|
67
60
|
ret = nil
|
68
|
-
|
69
|
-
|
61
|
+
one_worked = false # actually found one that worked
|
62
|
+
errors = []
|
63
|
+
|
64
|
+
@connections.each do |con|
|
65
|
+
next if con._makara_blacklisted?
|
66
|
+
begin
|
67
|
+
if block
|
68
|
+
value = @proxy.error_handler.handle(con) do
|
69
|
+
yield con
|
70
|
+
end
|
71
|
+
end
|
72
|
+
|
73
|
+
if method
|
74
|
+
ret = con.send(method, *args)
|
75
|
+
else
|
76
|
+
ret = value
|
77
|
+
end
|
78
|
+
one_worked = true
|
79
|
+
rescue Makara::Errors::BlacklistConnection => e
|
80
|
+
errors.insert(0, e)
|
81
|
+
con._makara_blacklist!
|
82
|
+
end
|
70
83
|
end
|
71
|
-
ret
|
72
|
-
end
|
73
84
|
|
74
|
-
|
75
|
-
|
76
|
-
|
77
|
-
|
78
|
-
|
79
|
-
provide(false) do |con|
|
80
|
-
yield con
|
85
|
+
if !one_worked
|
86
|
+
if connection_made?
|
87
|
+
raise Makara::Errors::AllConnectionsBlacklisted.new(self, errors)
|
88
|
+
else
|
89
|
+
raise Makara::Errors::NoConnectionsAvailable.new(@role) unless @disabled
|
81
90
|
end
|
82
|
-
|
83
|
-
|
84
|
-
|
91
|
+
end
|
92
|
+
|
93
|
+
ret
|
85
94
|
end
|
86
95
|
|
87
96
|
# Provide a connection that is not blacklisted and connected. Handle any errors
|
88
97
|
# that may occur within the block.
|
89
|
-
def provide
|
90
|
-
provided_connection = self.next
|
98
|
+
def provide
|
99
|
+
provided_connection = self.next
|
91
100
|
|
92
101
|
# nil implies that it's blacklisted
|
93
102
|
if provided_connection
|
@@ -130,59 +139,17 @@ module Makara
|
|
130
139
|
# Get the next non-blacklisted connection. If the proxy is setup
|
131
140
|
# to be sticky, provide back the current connection assuming it is
|
132
141
|
# not blacklisted.
|
133
|
-
def next
|
134
|
-
|
135
|
-
|
136
|
-
con = safe_value(@current_idx)
|
142
|
+
def next
|
143
|
+
if @proxy.sticky && Makara::Context.get_current == @context
|
144
|
+
con = @strategy.current
|
137
145
|
return con if con
|
138
146
|
end
|
139
147
|
|
140
|
-
|
141
|
-
|
142
|
-
|
143
|
-
idx = next_index(idx)
|
144
|
-
|
145
|
-
# if we've looped all the way around, return our safe value
|
146
|
-
return safe_value(idx, true) if idx == @current_idx
|
147
|
-
|
148
|
-
# while our current safe value is dangerous
|
149
|
-
end while safe_value(idx).nil?
|
150
|
-
|
151
|
-
# store our current spot and return our safe value
|
152
|
-
safe_value(idx, true)
|
153
|
-
end
|
154
|
-
|
155
|
-
|
156
|
-
# next index within the bounds of the connections array
|
157
|
-
# loop around when the end is hit
|
158
|
-
def next_index(idx)
|
159
|
-
idx = idx + 1
|
160
|
-
idx = 0 if idx >= @connections.length
|
161
|
-
idx
|
162
|
-
end
|
163
|
-
|
164
|
-
|
165
|
-
# return the connection if it's not blacklisted
|
166
|
-
# otherwise return nil
|
167
|
-
# optionally, store the position and context we're returning
|
168
|
-
def safe_value(idx, stick = false)
|
169
|
-
con = @connections[idx]
|
170
|
-
return nil unless con
|
171
|
-
return nil if con._makara_blacklisted?
|
172
|
-
|
173
|
-
if stick
|
174
|
-
@current_idx = idx
|
148
|
+
con = @strategy.next
|
149
|
+
if con
|
175
150
|
@context = Makara::Context.get_current
|
176
151
|
end
|
177
|
-
|
178
152
|
con
|
179
153
|
end
|
180
|
-
|
181
|
-
|
182
|
-
# stub in test mode to ensure consistency
|
183
|
-
def should_shuffle?
|
184
|
-
true
|
185
|
-
end
|
186
|
-
|
187
154
|
end
|
188
155
|
end
|
data/lib/makara/proxy.rb
CHANGED
@@ -1,6 +1,7 @@
|
|
1
1
|
require 'delegate'
|
2
2
|
require 'active_support/core_ext/class/attribute'
|
3
3
|
require 'active_support/core_ext/hash/keys'
|
4
|
+
require 'active_support/core_ext/string/inflections'
|
4
5
|
|
5
6
|
# The entry point of Makara. It contains a master and slave pool which are chosen based on the invocation
|
6
7
|
# being proxied. Makara::Proxy implementations should declare which methods they are hijacking via the
|
@@ -11,7 +12,7 @@ require 'active_support/core_ext/hash/keys'
|
|
11
12
|
module Makara
|
12
13
|
class Proxy < ::SimpleDelegator
|
13
14
|
|
14
|
-
METHOD_MISSING_SKIP = [ :byebug ]
|
15
|
+
METHOD_MISSING_SKIP = [ :byebug, :puts ]
|
15
16
|
|
16
17
|
class_attribute :hijack_methods
|
17
18
|
self.hijack_methods = []
|
@@ -42,6 +43,7 @@ module Makara
|
|
42
43
|
|
43
44
|
attr_reader :error_handler
|
44
45
|
attr_reader :sticky
|
46
|
+
attr_reader :config_parser
|
45
47
|
|
46
48
|
def initialize(config)
|
47
49
|
@config = config.symbolize_keys
|
@@ -72,6 +74,17 @@ module Makara
|
|
72
74
|
Makara::Cache.write("makara::#{@master_context}-#{@id}", '1', @ttl) if write_to_cache
|
73
75
|
end
|
74
76
|
|
77
|
+
def strategy_for(role)
|
78
|
+
strategy_name = @config_parser.makara_config["#{role}_strategy".to_sym]
|
79
|
+
case strategy_name
|
80
|
+
when 'round_robin', 'roundrobin', nil, ''
|
81
|
+
strategy_name = "::Makara::Strategies::RoundRobin"
|
82
|
+
when 'failover'
|
83
|
+
strategy_name = "::Makara::Strategies::PriorityFailover"
|
84
|
+
end
|
85
|
+
strategy_name.constantize.new(self)
|
86
|
+
end
|
87
|
+
|
75
88
|
def method_missing(m, *args, &block)
|
76
89
|
if METHOD_MISSING_SKIP.include?(m)
|
77
90
|
return super(m, *args, &block)
|
@@ -125,13 +138,13 @@ module Makara
|
|
125
138
|
end
|
126
139
|
|
127
140
|
def any_connection
|
128
|
-
@master_pool.provide
|
141
|
+
@master_pool.provide do |con|
|
129
142
|
yield con
|
130
143
|
end
|
131
144
|
rescue ::Makara::Errors::AllConnectionsBlacklisted, ::Makara::Errors::NoConnectionsAvailable => e
|
132
145
|
begin
|
133
146
|
@master_pool.disabled = true
|
134
|
-
@slave_pool.provide
|
147
|
+
@slave_pool.provide do |con|
|
135
148
|
yield con
|
136
149
|
end
|
137
150
|
ensure
|
@@ -0,0 +1,29 @@
|
|
1
|
+
module Makara
|
2
|
+
module Strategies
|
3
|
+
class Abstract
|
4
|
+
attr_reader :pool
|
5
|
+
def initialize(pool)
|
6
|
+
@pool = pool
|
7
|
+
init
|
8
|
+
end
|
9
|
+
|
10
|
+
def init
|
11
|
+
# explicit constructor
|
12
|
+
end
|
13
|
+
|
14
|
+
def connection_added(wrapper)
|
15
|
+
# doesn't have to be implemented
|
16
|
+
end
|
17
|
+
|
18
|
+
def current
|
19
|
+
# it's sticky - give the "curent" one
|
20
|
+
Kernel.raise NotImplementedError
|
21
|
+
end
|
22
|
+
|
23
|
+
def next
|
24
|
+
# rotate to the "next" one if you feel like it
|
25
|
+
Kernel.raise NotImplementedError
|
26
|
+
end
|
27
|
+
end
|
28
|
+
end
|
29
|
+
end
|
@@ -0,0 +1,48 @@
|
|
1
|
+
module Makara
|
2
|
+
module Strategies
|
3
|
+
class PriorityFailover < ::Makara::Strategies::Abstract
|
4
|
+
def init
|
5
|
+
@current_idx = 0
|
6
|
+
@weighted_connections = []
|
7
|
+
end
|
8
|
+
|
9
|
+
def connection_added(wrapper)
|
10
|
+
# insert in weighted order
|
11
|
+
@weighted_connections.each_with_index do |con, index|
|
12
|
+
if wrapper._makara_weight > con._makara_weight
|
13
|
+
@weighted_connections.insert(index, wrapper)
|
14
|
+
return
|
15
|
+
end
|
16
|
+
end
|
17
|
+
|
18
|
+
# else at end
|
19
|
+
@weighted_connections << wrapper
|
20
|
+
end
|
21
|
+
|
22
|
+
def current
|
23
|
+
safe_value(@current_idx)
|
24
|
+
end
|
25
|
+
|
26
|
+
def next
|
27
|
+
@weighted_connections.each_with_index do |con, index|
|
28
|
+
check = safe_value(index)
|
29
|
+
next unless check
|
30
|
+
@current_idx = index
|
31
|
+
return check
|
32
|
+
end
|
33
|
+
|
34
|
+
nil
|
35
|
+
end
|
36
|
+
|
37
|
+
# return the connection if it's not blacklisted
|
38
|
+
# otherwise return nil
|
39
|
+
# optionally, store the position and context we're returning
|
40
|
+
def safe_value(idx)
|
41
|
+
con = @weighted_connections[idx]
|
42
|
+
return nil unless con
|
43
|
+
return nil if con._makara_blacklisted?
|
44
|
+
con
|
45
|
+
end
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
@@ -0,0 +1,72 @@
|
|
1
|
+
module Makara
|
2
|
+
module Strategies
|
3
|
+
class RoundRobin < ::Makara::Strategies::Abstract
|
4
|
+
def init
|
5
|
+
@current_idx = 0
|
6
|
+
@weighted_connections = []
|
7
|
+
end
|
8
|
+
|
9
|
+
def connection_added(wrapper)
|
10
|
+
# the weight results in N references to the connection, not N connections
|
11
|
+
wrapper._makara_weight.times{ @weighted_connections << wrapper }
|
12
|
+
|
13
|
+
if should_shuffle?
|
14
|
+
# randomize the connections so we don't get peaks and valleys of load
|
15
|
+
@weighted_connections.shuffle!
|
16
|
+
# then start at a random spot in the list
|
17
|
+
@current_idx = rand(@weighted_connections.length)
|
18
|
+
end
|
19
|
+
end
|
20
|
+
|
21
|
+
def current
|
22
|
+
safe_value(@current_idx)
|
23
|
+
end
|
24
|
+
|
25
|
+
def next
|
26
|
+
idx = @current_idx
|
27
|
+
begin
|
28
|
+
|
29
|
+
idx = next_index(idx)
|
30
|
+
|
31
|
+
# if we've looped all the way around, return our safe value
|
32
|
+
return safe_value(idx, true) if idx == @current_idx
|
33
|
+
|
34
|
+
# while our current safe value is dangerous
|
35
|
+
end while safe_value(idx).nil?
|
36
|
+
|
37
|
+
# store our current spot and return our safe value
|
38
|
+
safe_value(idx, true)
|
39
|
+
end
|
40
|
+
|
41
|
+
# next index within the bounds of the connections array
|
42
|
+
# loop around when the end is hit
|
43
|
+
def next_index(idx)
|
44
|
+
idx = idx + 1
|
45
|
+
idx = 0 if idx >= @weighted_connections.length
|
46
|
+
idx
|
47
|
+
end
|
48
|
+
|
49
|
+
|
50
|
+
# return the connection if it's not blacklisted
|
51
|
+
# otherwise return nil
|
52
|
+
# optionally, store the position and context we're returning
|
53
|
+
def safe_value(idx, stick = false)
|
54
|
+
con = @weighted_connections[idx]
|
55
|
+
return nil unless con
|
56
|
+
return nil if con._makara_blacklisted?
|
57
|
+
|
58
|
+
if stick
|
59
|
+
@current_idx = idx
|
60
|
+
end
|
61
|
+
|
62
|
+
con
|
63
|
+
end
|
64
|
+
|
65
|
+
|
66
|
+
# stub in test mode to ensure consistency
|
67
|
+
def should_shuffle?
|
68
|
+
true
|
69
|
+
end
|
70
|
+
end
|
71
|
+
end
|
72
|
+
end
|
data/lib/makara/version.rb
CHANGED
@@ -144,7 +144,7 @@ describe 'MakaraMysql2Adapter' do
|
|
144
144
|
|
145
145
|
it 'should send reads to the slave' do
|
146
146
|
# ensure the next connection will be the first one
|
147
|
-
connection.slave_pool.instance_variable_set('@current_idx', connection.slave_pool.connections.length)
|
147
|
+
connection.slave_pool.strategy.instance_variable_set('@current_idx', connection.slave_pool.connections.length)
|
148
148
|
|
149
149
|
con = connection.slave_pool.connections.first
|
150
150
|
expect(con).to receive(:execute).with('SELECT * FROM users').once
|
@@ -64,7 +64,7 @@ describe 'MakaraPostgreSQLAdapter' do
|
|
64
64
|
|
65
65
|
it 'should send reads to the slave' do
|
66
66
|
# ensure the next connection will be the first one
|
67
|
-
connection.slave_pool.instance_variable_set('@current_idx', connection.slave_pool.connections.length)
|
67
|
+
connection.slave_pool.strategy.instance_variable_set('@current_idx', connection.slave_pool.connections.length)
|
68
68
|
|
69
69
|
con = connection.slave_pool.connections.first
|
70
70
|
expect(con).to receive(:execute).with('SELECT * FROM users').once
|
data/spec/pool_spec.rb
CHANGED
@@ -9,18 +9,20 @@ describe Makara::Pool do
|
|
9
9
|
it 'should wrap connections with a ConnectionWrapper as theyre added to the pool' do
|
10
10
|
expect(pool.connections).to be_empty
|
11
11
|
|
12
|
-
connection_a = FakeConnection.new
|
13
|
-
connection_a.something = 'a'
|
12
|
+
connection_a = FakeConnection.new(something: 'a')
|
14
13
|
|
15
14
|
wrapper_a = pool.add(pool_config){ connection_a }
|
16
15
|
wrapper_b = pool.add(pool_config.merge(:weight => 2)){ FakeConnection.new }
|
17
16
|
|
18
|
-
|
17
|
+
connections = pool.connections
|
18
|
+
weighted_connections = pool.strategy.instance_variable_get("@weighted_connections")
|
19
|
+
expect(connections.length).to eq(2)
|
20
|
+
expect(weighted_connections.length).to eq(3)
|
19
21
|
|
20
22
|
expect(wrapper_a).to be_a(Makara::ConnectionWrapper)
|
21
23
|
expect(wrapper_a.irespondtothis).to eq('hey!')
|
22
24
|
|
23
|
-
as, bs =
|
25
|
+
as, bs = weighted_connections.partition{|c| c.something == 'a'}
|
24
26
|
expect(as.length).to eq(1)
|
25
27
|
expect(bs.length).to eq(2)
|
26
28
|
end
|
@@ -74,8 +76,8 @@ describe Makara::Pool do
|
|
74
76
|
|
75
77
|
it 'provides the next connection and blacklists' do
|
76
78
|
|
77
|
-
connection_a = FakeConnection.new
|
78
|
-
connection_b = FakeConnection.new
|
79
|
+
connection_a = FakeConnection.new(something: 'a')
|
80
|
+
connection_b = FakeConnection.new(something: 'b')
|
79
81
|
|
80
82
|
wrapper_a = pool.add(pool_config){ connection_a }
|
81
83
|
wrapper_b = pool.add(pool_config){ connection_b }
|
data/spec/spec_helper.rb
CHANGED
@@ -30,7 +30,7 @@ RSpec.configure do |config|
|
|
30
30
|
config.before :each do
|
31
31
|
Makara::Cache.store = :memory
|
32
32
|
change_context
|
33
|
-
allow_any_instance_of(Makara::
|
33
|
+
allow_any_instance_of(Makara::Strategies::RoundRobin).to receive(:should_shuffle?){ false }
|
34
34
|
end
|
35
35
|
|
36
36
|
def change_context
|
@@ -0,0 +1,49 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Makara::Strategies::PriorityFailover do
|
4
|
+
let(:proxy){ FakeProxy.new({:makara => pool_config.merge(makara_config).merge(:connections => [])}) }
|
5
|
+
let(:pool){ Makara::Pool.new('master', proxy) }
|
6
|
+
let(:pool_config){ {:blacklist_duration => 5} }
|
7
|
+
let(:makara_config) { { :master_strategy => 'failover' } }
|
8
|
+
let(:strategy) { pool.strategy }
|
9
|
+
|
10
|
+
it 'should use the strategy' do
|
11
|
+
expect(pool.strategy).to be_instance_of(Makara::Strategies::PriorityFailover)
|
12
|
+
end
|
13
|
+
|
14
|
+
it 'should take the top weight' do
|
15
|
+
wrapper_a = pool.add(pool_config){ FakeConnection.new(something: 'a') }
|
16
|
+
wrapper_b = pool.add(pool_config){ FakeConnection.new(something: 'b') }
|
17
|
+
wrapper_c = pool.add(pool_config.merge(weight: 2)){ FakeConnection.new(something: 'c') }
|
18
|
+
|
19
|
+
expect(strategy.current.something).to eql('c')
|
20
|
+
expect(strategy.next.something).to eql('c')
|
21
|
+
expect(strategy.next.something).to eql('c')
|
22
|
+
end
|
23
|
+
|
24
|
+
it 'should take given order if no weights' do
|
25
|
+
wrapper_a = pool.add(pool_config){ FakeConnection.new(something: 'a') }
|
26
|
+
wrapper_b = pool.add(pool_config){ FakeConnection.new(something: 'b') }
|
27
|
+
wrapper_c = pool.add(pool_config){ FakeConnection.new(something: 'c') }
|
28
|
+
|
29
|
+
expect(strategy.current.something).to eql('a')
|
30
|
+
expect(strategy.next.something).to eql('a')
|
31
|
+
end
|
32
|
+
|
33
|
+
it 'should handle failover to next one' do
|
34
|
+
wrapper_a = pool.add(pool_config){ FakeConnection.new(something: 'a') }
|
35
|
+
wrapper_b = pool.add(pool_config){ FakeConnection.new(something: 'b') }
|
36
|
+
wrapper_c = pool.add(pool_config){ FakeConnection.new(something: 'c') }
|
37
|
+
|
38
|
+
pool.provide do |connection|
|
39
|
+
if connection == wrapper_a
|
40
|
+
raise Makara::Errors::BlacklistConnection.new(wrapper_a, StandardError.new('failure'))
|
41
|
+
end
|
42
|
+
end
|
43
|
+
|
44
|
+
# skips a
|
45
|
+
expect(strategy.current.something).to eql('b')
|
46
|
+
expect(strategy.next.something).to eql('b')
|
47
|
+
end
|
48
|
+
|
49
|
+
end
|
@@ -0,0 +1,67 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Makara::Strategies::RoundRobin do
|
4
|
+
let(:proxy){ FakeProxy.new({:makara => pool_config.merge(makara_config).merge(:connections => [])}) }
|
5
|
+
let(:pool){ Makara::Pool.new('test', proxy) }
|
6
|
+
let(:pool_config){ {:blacklist_duration => 5} }
|
7
|
+
let(:makara_config) { {} }
|
8
|
+
let(:strategy) { pool.strategy }
|
9
|
+
|
10
|
+
context 'default config' do
|
11
|
+
it 'should default to the strategy' do
|
12
|
+
expect(pool.strategy).to be_instance_of(Makara::Strategies::RoundRobin)
|
13
|
+
end
|
14
|
+
end
|
15
|
+
|
16
|
+
context 'bad config' do
|
17
|
+
let(:makara_config) { { :test_strategy => 'SomethingElse::Here' } }
|
18
|
+
it 'should raise name error' do
|
19
|
+
expect {
|
20
|
+
pool
|
21
|
+
}.to raise_error(NameError)
|
22
|
+
end
|
23
|
+
end
|
24
|
+
|
25
|
+
context 'given in config' do
|
26
|
+
let(:makara_config) { { :test_strategy => 'round_robin' } }
|
27
|
+
it 'should use the strategy' do
|
28
|
+
expect(pool.strategy).to be_instance_of(Makara::Strategies::RoundRobin)
|
29
|
+
end
|
30
|
+
end
|
31
|
+
|
32
|
+
|
33
|
+
it 'should loop through with weights' do
|
34
|
+
wrapper_a = pool.add(pool_config){ FakeConnection.new(something: 'a') }
|
35
|
+
wrapper_b = pool.add(pool_config){ FakeConnection.new(something: 'b') }
|
36
|
+
wrapper_c = pool.add(pool_config.merge(weight: 2)){ FakeConnection.new(something: 'c') }
|
37
|
+
|
38
|
+
expect(strategy.current.something).to eql('a')
|
39
|
+
expect(strategy.next.something).to eql('b')
|
40
|
+
expect(strategy.current.something).to eql('b')
|
41
|
+
expect(strategy.current.something).to eql('b')
|
42
|
+
expect(strategy.next.something).to eql('c')
|
43
|
+
expect(strategy.next.something).to eql('c')
|
44
|
+
expect(strategy.next.something).to eql('a')
|
45
|
+
end
|
46
|
+
|
47
|
+
it 'should handle failover to next one' do
|
48
|
+
wrapper_a = pool.add(pool_config){ FakeConnection.new(something: 'a') }
|
49
|
+
wrapper_b = pool.add(pool_config){ FakeConnection.new(something: 'b') }
|
50
|
+
wrapper_c = pool.add(pool_config.merge(weight: 2)){ FakeConnection.new(something: 'c') }
|
51
|
+
|
52
|
+
pool.provide do |connection|
|
53
|
+
if connection == wrapper_a
|
54
|
+
raise Makara::Errors::BlacklistConnection.new(wrapper_a, StandardError.new('failure'))
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
58
|
+
# skips a
|
59
|
+
expect(strategy.current.something).to eql('b')
|
60
|
+
expect(strategy.next.something).to eql('c')
|
61
|
+
expect(strategy.next.something).to eql('c')
|
62
|
+
expect(strategy.next.something).to eql('b')
|
63
|
+
end
|
64
|
+
|
65
|
+
|
66
|
+
|
67
|
+
end
|
@@ -2,8 +2,6 @@ require 'active_record/connection_adapters/makara_abstract_adapter'
|
|
2
2
|
|
3
3
|
class FakeConnection < Struct.new(:config)
|
4
4
|
|
5
|
-
attr_accessor :something
|
6
|
-
|
7
5
|
def ping
|
8
6
|
'ping!'
|
9
7
|
end
|
@@ -23,6 +21,10 @@ class FakeConnection < Struct.new(:config)
|
|
23
21
|
def disconnect!
|
24
22
|
true
|
25
23
|
end
|
24
|
+
|
25
|
+
def something
|
26
|
+
(config || {})[:something]
|
27
|
+
end
|
26
28
|
end
|
27
29
|
|
28
30
|
class FakeDatabaseAdapter < Struct.new(:config)
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: makara
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.3.
|
4
|
+
version: 0.3.6
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Mike Nelson
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2016-
|
11
|
+
date: 2016-04-21 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activerecord
|
@@ -73,6 +73,9 @@ files:
|
|
73
73
|
- lib/makara/pool.rb
|
74
74
|
- lib/makara/proxy.rb
|
75
75
|
- lib/makara/railtie.rb
|
76
|
+
- lib/makara/strategies/abstract.rb
|
77
|
+
- lib/makara/strategies/priority_failover.rb
|
78
|
+
- lib/makara/strategies/round_robin.rb
|
76
79
|
- lib/makara/version.rb
|
77
80
|
- makara.gemspec
|
78
81
|
- spec/active_record/connection_adapters/makara_abstract_adapter_error_handling_spec.rb
|
@@ -87,6 +90,8 @@ files:
|
|
87
90
|
- spec/pool_spec.rb
|
88
91
|
- spec/proxy_spec.rb
|
89
92
|
- spec/spec_helper.rb
|
93
|
+
- spec/strategies/priority_failover_spec.rb
|
94
|
+
- spec/strategies/round_robin_spec.rb
|
90
95
|
- spec/support/configurator.rb
|
91
96
|
- spec/support/deep_dup.rb
|
92
97
|
- spec/support/mock_objects.rb
|
@@ -132,6 +137,8 @@ test_files:
|
|
132
137
|
- spec/pool_spec.rb
|
133
138
|
- spec/proxy_spec.rb
|
134
139
|
- spec/spec_helper.rb
|
140
|
+
- spec/strategies/priority_failover_spec.rb
|
141
|
+
- spec/strategies/round_robin_spec.rb
|
135
142
|
- spec/support/configurator.rb
|
136
143
|
- spec/support/deep_dup.rb
|
137
144
|
- spec/support/mock_objects.rb
|