makara 0.3.5 → 0.3.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +85 -17
- data/README.md +2 -0
- data/lib/active_record/connection_adapters/makara_abstract_adapter.rb +5 -12
- data/lib/makara.rb +6 -0
- data/lib/makara/error_handler.rb +1 -1
- data/lib/makara/pool.rb +43 -76
- data/lib/makara/proxy.rb +16 -3
- data/lib/makara/strategies/abstract.rb +29 -0
- data/lib/makara/strategies/priority_failover.rb +48 -0
- data/lib/makara/strategies/round_robin.rb +72 -0
- data/lib/makara/version.rb +1 -1
- data/spec/active_record/connection_adapters/makara_mysql2_adapter_spec.rb +1 -1
- data/spec/active_record/connection_adapters/makara_postgresql_adapter_spec.rb +1 -1
- data/spec/pool_spec.rb +8 -6
- data/spec/spec_helper.rb +1 -1
- data/spec/strategies/priority_failover_spec.rb +49 -0
- data/spec/strategies/round_robin_spec.rb +67 -0
- data/spec/support/mock_objects.rb +4 -2
- metadata +9 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 854be0f8b5615812158cd37511db68cc47310f75
|
4
|
+
data.tar.gz: 8cc3d705dd2d83415a8d4348d9ce2b949d0eb954
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 8669c261ead2cce40c1ffa8f319f975d162623071f54290fbed54e1a65159b2698c9eab709159cb6f71858b2d32b20ad9996b8472502c91a8086fdb89c08d31a
|
7
|
+
data.tar.gz: 2089f3e36f9add64c5ecdef2634665fde5aa3a2cc94ab4230f9f8099d9c438f7da057b73406b8c8f9abab8c596424ad4cd0deef627fff7831f0c0ca9f9004233
|
data/CHANGELOG.md
CHANGED
@@ -1,27 +1,95 @@
|
|
1
1
|
# Change Log
|
2
2
|
All notable changes to this project will be documented in this file.
|
3
3
|
|
4
|
-
##
|
5
|
-
### Added
|
6
|
-
- allow bypassing of stickiness
|
4
|
+
## v0.3.5 - 2016-01-08
|
7
5
|
|
8
|
-
|
9
|
-
### Added
|
10
|
-
- add postgres specific tests.
|
6
|
+
[Full Changelog](https://github.com/taskrabbit/makara/compare/v0.3.4.rc1...v0.3.5)
|
11
7
|
|
12
|
-
|
13
|
-
- change using methods for matchers to be able to monkey patch them
|
14
|
-
- follow AR naming conventions for adapter naming
|
8
|
+
Changed
|
15
9
|
|
16
|
-
|
17
|
-
|
18
|
-
|
10
|
+
- Raise `Makara::Errors::AllConnectionsBlacklisted` on timeout. [#104](https://github.com/taskrabbit/makara/pull/104) Brian Leonard
|
11
|
+
|
12
|
+
## v0.3.4.rc1 - 2016-01-06
|
13
|
+
|
14
|
+
[Full Changelog](https://github.com/taskrabbit/makara/compare/v0.3.3...v0.3.4.rc1)
|
15
|
+
|
16
|
+
Added
|
17
|
+
|
18
|
+
- Add `url` to database connections configurations. [#93](https://github.com/taskrabbit/makara/pull/93) Benjamin Fleischer
|
19
|
+
|
20
|
+
Changed
|
21
|
+
|
22
|
+
- Improve Postgresql compatibility and failover support, also fix [#78](https://github.com/taskrabbit/makara/issues/78), [#79](https://github.com/taskrabbit/makara/issues/79). [#87](https://github.com/taskrabbit/makara/pull/87) Vlad
|
23
|
+
- Update README: Specify newrelic_rpm gem versions that will have the performance issue. [#95](https://github.com/taskrabbit/makara/pull/95) Benjamin Fleischer
|
24
|
+
|
25
|
+
## v0.3.3 - 2015-05-20
|
26
|
+
|
27
|
+
[Full Changelog](https://github.com/taskrabbit/makara/compare/v0.3.2...v0.3.3)
|
28
|
+
|
29
|
+
Changed
|
30
|
+
|
31
|
+
- A context is local to the curent thread of execution. This will allow you to stick to master safely in a single thread in systems such as sidekiq, for instance. Fix [#83](https://github.com/taskrabbit/makara/issues/83). [#84](https://github.com/taskrabbit/makara/pull/84) Matt Camuto
|
32
|
+
|
33
|
+
## v0.3.2 - 2015-05-16
|
34
|
+
|
35
|
+
[Full Changelog](https://github.com/taskrabbit/makara/compare/v0.3.1...v0.3.2)
|
36
|
+
|
37
|
+
Fixed
|
38
|
+
|
39
|
+
- Fix a `ArgumentError: not delegated` error for rails 3. [#82](https://github.com/taskrabbit/makara/pull/82) Eric Saxby
|
40
|
+
|
41
|
+
Changed
|
42
|
+
|
43
|
+
- Switch log format from `:info` to `:error`. Mike Nelson
|
44
|
+
|
45
|
+
## v0.3.1 - 2015-05-08
|
46
|
+
|
47
|
+
[Full Changelog](https://github.com/taskrabbit/makara/compare/v0.3.0...v0.3.1)
|
48
|
+
|
49
|
+
Changed
|
50
|
+
|
51
|
+
- Globally move to multiline matchers. Mike Nelson
|
52
|
+
|
53
|
+
Changed
|
54
|
+
|
55
|
+
## v0.3.0 - 2015-04-27
|
56
|
+
|
57
|
+
[Full Changelog](https://github.com/taskrabbit/makara/compare/v0.3.0.rc3...v0.3.0)
|
58
|
+
|
59
|
+
Changed
|
60
|
+
|
61
|
+
- Reduce logging noise by using [the same rules as ActiveRecord uses](https://github.com/rails/rails/blob/b06f64c3480cd389d14618540d62da4978918af0/activerecord/lib/active_record/log_subscriber.rb#L33). [#76](https://github.com/taskrabbit/makara/pull/76) Andrew Kane
|
62
|
+
|
63
|
+
Fixed
|
64
|
+
|
65
|
+
- Fix an issue for postgres that would route all queries to master. [#72](https://github.com/taskrabbit/makara/pull/72) Kali Donovan
|
66
|
+
- Fix an edge case which would cause SET operations to send to all connections([#70](https://github.com/taskrabbit/makara/issues/70)). [#80](https://github.com/taskrabbit/makara/pull/80) Michael Amor Righi
|
67
|
+
- Fix performance regression with certain verions of [newrelic/rpm](https://github.com/newrelic/rpm)([#59](https://github.com/taskrabbit/makara/issues/59)). [#75](https://github.com/taskrabbit/makara/pull/75) Mike Nelson
|
68
|
+
|
69
|
+
## 0.3.0.rc3 - 2014-09-02[YANKED]
|
70
|
+
|
71
|
+
[Full Changelog](https://github.com/taskrabbit/makara/compare/v0.3.0.rc2...v0.3.0.rc3)
|
72
|
+
|
73
|
+
Added
|
74
|
+
- Allow bypassing of stickiness
|
75
|
+
|
76
|
+
## 0.3.0.rc2 - 2014-08-05
|
77
|
+
Added
|
78
|
+
- Add postgres specific tests.
|
79
|
+
|
80
|
+
Changed
|
81
|
+
- Change using methods for matchers to be able to monkey patch them.
|
82
|
+
- Follow AR naming conventions for adapter naming.
|
83
|
+
|
84
|
+
## 0.3.0.rc1 - 2014-08-05
|
85
|
+
Removed
|
86
|
+
- Remove initial connection logic. If a connection can't be made on startup, an error will be thrown rather than the node getting blacklisted.
|
19
87
|
|
20
88
|
|
21
89
|
## 0.2.2 - 2014-04-03
|
22
|
-
|
23
|
-
-
|
90
|
+
Added
|
91
|
+
- Add logging of makara operations via the Makara::Logger.
|
24
92
|
|
25
|
-
|
26
|
-
-
|
27
|
-
-
|
93
|
+
Changed
|
94
|
+
- Begin tracing the series of errors associated with blacklisting rather than just the last. This becomes apparent in error messages.
|
95
|
+
- Fix Rails.cache usage when full environment is not loaded.
|
data/README.md
CHANGED
@@ -140,6 +140,7 @@ production:
|
|
140
140
|
# the following are default values
|
141
141
|
blacklist_duration: 5
|
142
142
|
master_ttl: 5
|
143
|
+
master_strategy: round_robin
|
143
144
|
sticky: true
|
144
145
|
|
145
146
|
# list your connections with the override values (they're merged into the top-level config)
|
@@ -161,6 +162,7 @@ The makara subconfig sets up the proxy with a few of its own options, then provi
|
|
161
162
|
* blacklist_duration - the number of seconds a node is blacklisted when a connection failure occurs
|
162
163
|
* sticky - if a node should be stuck to once it's used during a specific context
|
163
164
|
* master_ttl - how long the master context is persisted. generally, this needs to be longer than any replication lag
|
165
|
+
* master_strategy - use a different strategy for picking the "current" node: `failover` will try to keep the same one until it is blacklisted. The default is `round_robin` which will cycle through available ones.
|
164
166
|
* connection_error_matchers - array of custom error matchers you want to be handled gracefully by Makara (as in, errors matching these regexes will result in blacklisting the connection as opposed to raising directly).
|
165
167
|
|
166
168
|
Connection definitions contain any extra node-specific configurations. If the node should behave as a master you must provide `role: master`. Any previous configurations can be overridden within a specific node's config. Nodes can also contain weights if you'd like to balance usage based on hardware specifications. Optionally, you can provide a name attribute which will be used in sql logging.
|
@@ -143,21 +143,14 @@ module ActiveRecord
|
|
143
143
|
protected
|
144
144
|
|
145
145
|
|
146
|
-
def appropriate_connection(method_name, args)
|
146
|
+
def appropriate_connection(method_name, args, &block)
|
147
147
|
if needed_by_all?(method_name, args)
|
148
148
|
|
149
149
|
handling_an_all_execution(method_name) do
|
150
|
-
|
151
|
-
|
152
|
-
|
153
|
-
|
154
|
-
end
|
155
|
-
end
|
156
|
-
|
157
|
-
@master_pool.provide_each do |con|
|
158
|
-
hijacked do
|
159
|
-
yield con
|
160
|
-
end
|
150
|
+
hijacked do
|
151
|
+
# slave pool must run first.
|
152
|
+
@slave_pool.send_to_all(nil, &block) # just yields to each con
|
153
|
+
@master_pool.send_to_all(nil, &block) # just yields to each con
|
161
154
|
end
|
162
155
|
end
|
163
156
|
|
data/lib/makara.rb
CHANGED
@@ -22,4 +22,10 @@ module Makara
|
|
22
22
|
autoload :Subscriber, 'makara/logging/subscriber'
|
23
23
|
end
|
24
24
|
|
25
|
+
module Strategies
|
26
|
+
autoload :Abstract, 'makara/strategies/abstract'
|
27
|
+
autoload :RoundRobin, 'makara/strategies/round_robin'
|
28
|
+
autoload :PriorityFailover, 'makara/strategies/priority_failover'
|
29
|
+
end
|
30
|
+
|
25
31
|
end
|
data/lib/makara/error_handler.rb
CHANGED
data/lib/makara/pool.rb
CHANGED
@@ -13,6 +13,7 @@ module Makara
|
|
13
13
|
attr_reader :blacklist_errors
|
14
14
|
attr_reader :role
|
15
15
|
attr_reader :connections
|
16
|
+
attr_reader :strategy
|
16
17
|
|
17
18
|
def initialize(role, proxy)
|
18
19
|
@role = role
|
@@ -20,8 +21,8 @@ module Makara
|
|
20
21
|
@context = Makara::Context.get_current
|
21
22
|
@connections = []
|
22
23
|
@blacklist_errors = []
|
23
|
-
@current_idx = 0
|
24
24
|
@disabled = false
|
25
|
+
@strategy = proxy.strategy_for(role)
|
25
26
|
end
|
26
27
|
|
27
28
|
|
@@ -47,47 +48,55 @@ module Makara
|
|
47
48
|
wrapper = Makara::ConnectionWrapper.new(@proxy, connection, config)
|
48
49
|
end
|
49
50
|
|
50
|
-
|
51
|
-
|
52
|
-
wrapper._makara_weight.times{ @connections << wrapper }
|
53
|
-
|
54
|
-
if should_shuffle?
|
55
|
-
# randomize the connections so we don't get peaks and valleys of load
|
56
|
-
@connections.shuffle!
|
57
|
-
|
58
|
-
# then start at a random spot in the list
|
59
|
-
@current_idx = rand(@connections.length)
|
60
|
-
end
|
51
|
+
@connections << wrapper
|
52
|
+
@strategy.connection_added(wrapper)
|
61
53
|
|
62
54
|
wrapper
|
63
55
|
end
|
64
56
|
|
65
57
|
# send this method to all available nodes
|
66
|
-
|
58
|
+
# send nil to just yield with each con if there is block
|
59
|
+
def send_to_all(method, *args, &block)
|
67
60
|
ret = nil
|
68
|
-
|
69
|
-
|
61
|
+
one_worked = false # actually found one that worked
|
62
|
+
errors = []
|
63
|
+
|
64
|
+
@connections.each do |con|
|
65
|
+
next if con._makara_blacklisted?
|
66
|
+
begin
|
67
|
+
if block
|
68
|
+
value = @proxy.error_handler.handle(con) do
|
69
|
+
yield con
|
70
|
+
end
|
71
|
+
end
|
72
|
+
|
73
|
+
if method
|
74
|
+
ret = con.send(method, *args)
|
75
|
+
else
|
76
|
+
ret = value
|
77
|
+
end
|
78
|
+
one_worked = true
|
79
|
+
rescue Makara::Errors::BlacklistConnection => e
|
80
|
+
errors.insert(0, e)
|
81
|
+
con._makara_blacklist!
|
82
|
+
end
|
70
83
|
end
|
71
|
-
ret
|
72
|
-
end
|
73
84
|
|
74
|
-
|
75
|
-
|
76
|
-
|
77
|
-
|
78
|
-
|
79
|
-
provide(false) do |con|
|
80
|
-
yield con
|
85
|
+
if !one_worked
|
86
|
+
if connection_made?
|
87
|
+
raise Makara::Errors::AllConnectionsBlacklisted.new(self, errors)
|
88
|
+
else
|
89
|
+
raise Makara::Errors::NoConnectionsAvailable.new(@role) unless @disabled
|
81
90
|
end
|
82
|
-
|
83
|
-
|
84
|
-
|
91
|
+
end
|
92
|
+
|
93
|
+
ret
|
85
94
|
end
|
86
95
|
|
87
96
|
# Provide a connection that is not blacklisted and connected. Handle any errors
|
88
97
|
# that may occur within the block.
|
89
|
-
def provide
|
90
|
-
provided_connection = self.next
|
98
|
+
def provide
|
99
|
+
provided_connection = self.next
|
91
100
|
|
92
101
|
# nil implies that it's blacklisted
|
93
102
|
if provided_connection
|
@@ -130,59 +139,17 @@ module Makara
|
|
130
139
|
# Get the next non-blacklisted connection. If the proxy is setup
|
131
140
|
# to be sticky, provide back the current connection assuming it is
|
132
141
|
# not blacklisted.
|
133
|
-
def next
|
134
|
-
|
135
|
-
|
136
|
-
con = safe_value(@current_idx)
|
142
|
+
def next
|
143
|
+
if @proxy.sticky && Makara::Context.get_current == @context
|
144
|
+
con = @strategy.current
|
137
145
|
return con if con
|
138
146
|
end
|
139
147
|
|
140
|
-
|
141
|
-
|
142
|
-
|
143
|
-
idx = next_index(idx)
|
144
|
-
|
145
|
-
# if we've looped all the way around, return our safe value
|
146
|
-
return safe_value(idx, true) if idx == @current_idx
|
147
|
-
|
148
|
-
# while our current safe value is dangerous
|
149
|
-
end while safe_value(idx).nil?
|
150
|
-
|
151
|
-
# store our current spot and return our safe value
|
152
|
-
safe_value(idx, true)
|
153
|
-
end
|
154
|
-
|
155
|
-
|
156
|
-
# next index within the bounds of the connections array
|
157
|
-
# loop around when the end is hit
|
158
|
-
def next_index(idx)
|
159
|
-
idx = idx + 1
|
160
|
-
idx = 0 if idx >= @connections.length
|
161
|
-
idx
|
162
|
-
end
|
163
|
-
|
164
|
-
|
165
|
-
# return the connection if it's not blacklisted
|
166
|
-
# otherwise return nil
|
167
|
-
# optionally, store the position and context we're returning
|
168
|
-
def safe_value(idx, stick = false)
|
169
|
-
con = @connections[idx]
|
170
|
-
return nil unless con
|
171
|
-
return nil if con._makara_blacklisted?
|
172
|
-
|
173
|
-
if stick
|
174
|
-
@current_idx = idx
|
148
|
+
con = @strategy.next
|
149
|
+
if con
|
175
150
|
@context = Makara::Context.get_current
|
176
151
|
end
|
177
|
-
|
178
152
|
con
|
179
153
|
end
|
180
|
-
|
181
|
-
|
182
|
-
# stub in test mode to ensure consistency
|
183
|
-
def should_shuffle?
|
184
|
-
true
|
185
|
-
end
|
186
|
-
|
187
154
|
end
|
188
155
|
end
|
data/lib/makara/proxy.rb
CHANGED
@@ -1,6 +1,7 @@
|
|
1
1
|
require 'delegate'
|
2
2
|
require 'active_support/core_ext/class/attribute'
|
3
3
|
require 'active_support/core_ext/hash/keys'
|
4
|
+
require 'active_support/core_ext/string/inflections'
|
4
5
|
|
5
6
|
# The entry point of Makara. It contains a master and slave pool which are chosen based on the invocation
|
6
7
|
# being proxied. Makara::Proxy implementations should declare which methods they are hijacking via the
|
@@ -11,7 +12,7 @@ require 'active_support/core_ext/hash/keys'
|
|
11
12
|
module Makara
|
12
13
|
class Proxy < ::SimpleDelegator
|
13
14
|
|
14
|
-
METHOD_MISSING_SKIP = [ :byebug ]
|
15
|
+
METHOD_MISSING_SKIP = [ :byebug, :puts ]
|
15
16
|
|
16
17
|
class_attribute :hijack_methods
|
17
18
|
self.hijack_methods = []
|
@@ -42,6 +43,7 @@ module Makara
|
|
42
43
|
|
43
44
|
attr_reader :error_handler
|
44
45
|
attr_reader :sticky
|
46
|
+
attr_reader :config_parser
|
45
47
|
|
46
48
|
def initialize(config)
|
47
49
|
@config = config.symbolize_keys
|
@@ -72,6 +74,17 @@ module Makara
|
|
72
74
|
Makara::Cache.write("makara::#{@master_context}-#{@id}", '1', @ttl) if write_to_cache
|
73
75
|
end
|
74
76
|
|
77
|
+
def strategy_for(role)
|
78
|
+
strategy_name = @config_parser.makara_config["#{role}_strategy".to_sym]
|
79
|
+
case strategy_name
|
80
|
+
when 'round_robin', 'roundrobin', nil, ''
|
81
|
+
strategy_name = "::Makara::Strategies::RoundRobin"
|
82
|
+
when 'failover'
|
83
|
+
strategy_name = "::Makara::Strategies::PriorityFailover"
|
84
|
+
end
|
85
|
+
strategy_name.constantize.new(self)
|
86
|
+
end
|
87
|
+
|
75
88
|
def method_missing(m, *args, &block)
|
76
89
|
if METHOD_MISSING_SKIP.include?(m)
|
77
90
|
return super(m, *args, &block)
|
@@ -125,13 +138,13 @@ module Makara
|
|
125
138
|
end
|
126
139
|
|
127
140
|
def any_connection
|
128
|
-
@master_pool.provide
|
141
|
+
@master_pool.provide do |con|
|
129
142
|
yield con
|
130
143
|
end
|
131
144
|
rescue ::Makara::Errors::AllConnectionsBlacklisted, ::Makara::Errors::NoConnectionsAvailable => e
|
132
145
|
begin
|
133
146
|
@master_pool.disabled = true
|
134
|
-
@slave_pool.provide
|
147
|
+
@slave_pool.provide do |con|
|
135
148
|
yield con
|
136
149
|
end
|
137
150
|
ensure
|
@@ -0,0 +1,29 @@
|
|
1
|
+
module Makara
|
2
|
+
module Strategies
|
3
|
+
class Abstract
|
4
|
+
attr_reader :pool
|
5
|
+
def initialize(pool)
|
6
|
+
@pool = pool
|
7
|
+
init
|
8
|
+
end
|
9
|
+
|
10
|
+
def init
|
11
|
+
# explicit constructor
|
12
|
+
end
|
13
|
+
|
14
|
+
def connection_added(wrapper)
|
15
|
+
# doesn't have to be implemented
|
16
|
+
end
|
17
|
+
|
18
|
+
def current
|
19
|
+
# it's sticky - give the "curent" one
|
20
|
+
Kernel.raise NotImplementedError
|
21
|
+
end
|
22
|
+
|
23
|
+
def next
|
24
|
+
# rotate to the "next" one if you feel like it
|
25
|
+
Kernel.raise NotImplementedError
|
26
|
+
end
|
27
|
+
end
|
28
|
+
end
|
29
|
+
end
|
@@ -0,0 +1,48 @@
|
|
1
|
+
module Makara
|
2
|
+
module Strategies
|
3
|
+
class PriorityFailover < ::Makara::Strategies::Abstract
|
4
|
+
def init
|
5
|
+
@current_idx = 0
|
6
|
+
@weighted_connections = []
|
7
|
+
end
|
8
|
+
|
9
|
+
def connection_added(wrapper)
|
10
|
+
# insert in weighted order
|
11
|
+
@weighted_connections.each_with_index do |con, index|
|
12
|
+
if wrapper._makara_weight > con._makara_weight
|
13
|
+
@weighted_connections.insert(index, wrapper)
|
14
|
+
return
|
15
|
+
end
|
16
|
+
end
|
17
|
+
|
18
|
+
# else at end
|
19
|
+
@weighted_connections << wrapper
|
20
|
+
end
|
21
|
+
|
22
|
+
def current
|
23
|
+
safe_value(@current_idx)
|
24
|
+
end
|
25
|
+
|
26
|
+
def next
|
27
|
+
@weighted_connections.each_with_index do |con, index|
|
28
|
+
check = safe_value(index)
|
29
|
+
next unless check
|
30
|
+
@current_idx = index
|
31
|
+
return check
|
32
|
+
end
|
33
|
+
|
34
|
+
nil
|
35
|
+
end
|
36
|
+
|
37
|
+
# return the connection if it's not blacklisted
|
38
|
+
# otherwise return nil
|
39
|
+
# optionally, store the position and context we're returning
|
40
|
+
def safe_value(idx)
|
41
|
+
con = @weighted_connections[idx]
|
42
|
+
return nil unless con
|
43
|
+
return nil if con._makara_blacklisted?
|
44
|
+
con
|
45
|
+
end
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
@@ -0,0 +1,72 @@
|
|
1
|
+
module Makara
|
2
|
+
module Strategies
|
3
|
+
class RoundRobin < ::Makara::Strategies::Abstract
|
4
|
+
def init
|
5
|
+
@current_idx = 0
|
6
|
+
@weighted_connections = []
|
7
|
+
end
|
8
|
+
|
9
|
+
def connection_added(wrapper)
|
10
|
+
# the weight results in N references to the connection, not N connections
|
11
|
+
wrapper._makara_weight.times{ @weighted_connections << wrapper }
|
12
|
+
|
13
|
+
if should_shuffle?
|
14
|
+
# randomize the connections so we don't get peaks and valleys of load
|
15
|
+
@weighted_connections.shuffle!
|
16
|
+
# then start at a random spot in the list
|
17
|
+
@current_idx = rand(@weighted_connections.length)
|
18
|
+
end
|
19
|
+
end
|
20
|
+
|
21
|
+
def current
|
22
|
+
safe_value(@current_idx)
|
23
|
+
end
|
24
|
+
|
25
|
+
def next
|
26
|
+
idx = @current_idx
|
27
|
+
begin
|
28
|
+
|
29
|
+
idx = next_index(idx)
|
30
|
+
|
31
|
+
# if we've looped all the way around, return our safe value
|
32
|
+
return safe_value(idx, true) if idx == @current_idx
|
33
|
+
|
34
|
+
# while our current safe value is dangerous
|
35
|
+
end while safe_value(idx).nil?
|
36
|
+
|
37
|
+
# store our current spot and return our safe value
|
38
|
+
safe_value(idx, true)
|
39
|
+
end
|
40
|
+
|
41
|
+
# next index within the bounds of the connections array
|
42
|
+
# loop around when the end is hit
|
43
|
+
def next_index(idx)
|
44
|
+
idx = idx + 1
|
45
|
+
idx = 0 if idx >= @weighted_connections.length
|
46
|
+
idx
|
47
|
+
end
|
48
|
+
|
49
|
+
|
50
|
+
# return the connection if it's not blacklisted
|
51
|
+
# otherwise return nil
|
52
|
+
# optionally, store the position and context we're returning
|
53
|
+
def safe_value(idx, stick = false)
|
54
|
+
con = @weighted_connections[idx]
|
55
|
+
return nil unless con
|
56
|
+
return nil if con._makara_blacklisted?
|
57
|
+
|
58
|
+
if stick
|
59
|
+
@current_idx = idx
|
60
|
+
end
|
61
|
+
|
62
|
+
con
|
63
|
+
end
|
64
|
+
|
65
|
+
|
66
|
+
# stub in test mode to ensure consistency
|
67
|
+
def should_shuffle?
|
68
|
+
true
|
69
|
+
end
|
70
|
+
end
|
71
|
+
end
|
72
|
+
end
|
data/lib/makara/version.rb
CHANGED
@@ -144,7 +144,7 @@ describe 'MakaraMysql2Adapter' do
|
|
144
144
|
|
145
145
|
it 'should send reads to the slave' do
|
146
146
|
# ensure the next connection will be the first one
|
147
|
-
connection.slave_pool.instance_variable_set('@current_idx', connection.slave_pool.connections.length)
|
147
|
+
connection.slave_pool.strategy.instance_variable_set('@current_idx', connection.slave_pool.connections.length)
|
148
148
|
|
149
149
|
con = connection.slave_pool.connections.first
|
150
150
|
expect(con).to receive(:execute).with('SELECT * FROM users').once
|
@@ -64,7 +64,7 @@ describe 'MakaraPostgreSQLAdapter' do
|
|
64
64
|
|
65
65
|
it 'should send reads to the slave' do
|
66
66
|
# ensure the next connection will be the first one
|
67
|
-
connection.slave_pool.instance_variable_set('@current_idx', connection.slave_pool.connections.length)
|
67
|
+
connection.slave_pool.strategy.instance_variable_set('@current_idx', connection.slave_pool.connections.length)
|
68
68
|
|
69
69
|
con = connection.slave_pool.connections.first
|
70
70
|
expect(con).to receive(:execute).with('SELECT * FROM users').once
|
data/spec/pool_spec.rb
CHANGED
@@ -9,18 +9,20 @@ describe Makara::Pool do
|
|
9
9
|
it 'should wrap connections with a ConnectionWrapper as theyre added to the pool' do
|
10
10
|
expect(pool.connections).to be_empty
|
11
11
|
|
12
|
-
connection_a = FakeConnection.new
|
13
|
-
connection_a.something = 'a'
|
12
|
+
connection_a = FakeConnection.new(something: 'a')
|
14
13
|
|
15
14
|
wrapper_a = pool.add(pool_config){ connection_a }
|
16
15
|
wrapper_b = pool.add(pool_config.merge(:weight => 2)){ FakeConnection.new }
|
17
16
|
|
18
|
-
|
17
|
+
connections = pool.connections
|
18
|
+
weighted_connections = pool.strategy.instance_variable_get("@weighted_connections")
|
19
|
+
expect(connections.length).to eq(2)
|
20
|
+
expect(weighted_connections.length).to eq(3)
|
19
21
|
|
20
22
|
expect(wrapper_a).to be_a(Makara::ConnectionWrapper)
|
21
23
|
expect(wrapper_a.irespondtothis).to eq('hey!')
|
22
24
|
|
23
|
-
as, bs =
|
25
|
+
as, bs = weighted_connections.partition{|c| c.something == 'a'}
|
24
26
|
expect(as.length).to eq(1)
|
25
27
|
expect(bs.length).to eq(2)
|
26
28
|
end
|
@@ -74,8 +76,8 @@ describe Makara::Pool do
|
|
74
76
|
|
75
77
|
it 'provides the next connection and blacklists' do
|
76
78
|
|
77
|
-
connection_a = FakeConnection.new
|
78
|
-
connection_b = FakeConnection.new
|
79
|
+
connection_a = FakeConnection.new(something: 'a')
|
80
|
+
connection_b = FakeConnection.new(something: 'b')
|
79
81
|
|
80
82
|
wrapper_a = pool.add(pool_config){ connection_a }
|
81
83
|
wrapper_b = pool.add(pool_config){ connection_b }
|
data/spec/spec_helper.rb
CHANGED
@@ -30,7 +30,7 @@ RSpec.configure do |config|
|
|
30
30
|
config.before :each do
|
31
31
|
Makara::Cache.store = :memory
|
32
32
|
change_context
|
33
|
-
allow_any_instance_of(Makara::
|
33
|
+
allow_any_instance_of(Makara::Strategies::RoundRobin).to receive(:should_shuffle?){ false }
|
34
34
|
end
|
35
35
|
|
36
36
|
def change_context
|
@@ -0,0 +1,49 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Makara::Strategies::PriorityFailover do
|
4
|
+
let(:proxy){ FakeProxy.new({:makara => pool_config.merge(makara_config).merge(:connections => [])}) }
|
5
|
+
let(:pool){ Makara::Pool.new('master', proxy) }
|
6
|
+
let(:pool_config){ {:blacklist_duration => 5} }
|
7
|
+
let(:makara_config) { { :master_strategy => 'failover' } }
|
8
|
+
let(:strategy) { pool.strategy }
|
9
|
+
|
10
|
+
it 'should use the strategy' do
|
11
|
+
expect(pool.strategy).to be_instance_of(Makara::Strategies::PriorityFailover)
|
12
|
+
end
|
13
|
+
|
14
|
+
it 'should take the top weight' do
|
15
|
+
wrapper_a = pool.add(pool_config){ FakeConnection.new(something: 'a') }
|
16
|
+
wrapper_b = pool.add(pool_config){ FakeConnection.new(something: 'b') }
|
17
|
+
wrapper_c = pool.add(pool_config.merge(weight: 2)){ FakeConnection.new(something: 'c') }
|
18
|
+
|
19
|
+
expect(strategy.current.something).to eql('c')
|
20
|
+
expect(strategy.next.something).to eql('c')
|
21
|
+
expect(strategy.next.something).to eql('c')
|
22
|
+
end
|
23
|
+
|
24
|
+
it 'should take given order if no weights' do
|
25
|
+
wrapper_a = pool.add(pool_config){ FakeConnection.new(something: 'a') }
|
26
|
+
wrapper_b = pool.add(pool_config){ FakeConnection.new(something: 'b') }
|
27
|
+
wrapper_c = pool.add(pool_config){ FakeConnection.new(something: 'c') }
|
28
|
+
|
29
|
+
expect(strategy.current.something).to eql('a')
|
30
|
+
expect(strategy.next.something).to eql('a')
|
31
|
+
end
|
32
|
+
|
33
|
+
it 'should handle failover to next one' do
|
34
|
+
wrapper_a = pool.add(pool_config){ FakeConnection.new(something: 'a') }
|
35
|
+
wrapper_b = pool.add(pool_config){ FakeConnection.new(something: 'b') }
|
36
|
+
wrapper_c = pool.add(pool_config){ FakeConnection.new(something: 'c') }
|
37
|
+
|
38
|
+
pool.provide do |connection|
|
39
|
+
if connection == wrapper_a
|
40
|
+
raise Makara::Errors::BlacklistConnection.new(wrapper_a, StandardError.new('failure'))
|
41
|
+
end
|
42
|
+
end
|
43
|
+
|
44
|
+
# skips a
|
45
|
+
expect(strategy.current.something).to eql('b')
|
46
|
+
expect(strategy.next.something).to eql('b')
|
47
|
+
end
|
48
|
+
|
49
|
+
end
|
@@ -0,0 +1,67 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Makara::Strategies::RoundRobin do
|
4
|
+
let(:proxy){ FakeProxy.new({:makara => pool_config.merge(makara_config).merge(:connections => [])}) }
|
5
|
+
let(:pool){ Makara::Pool.new('test', proxy) }
|
6
|
+
let(:pool_config){ {:blacklist_duration => 5} }
|
7
|
+
let(:makara_config) { {} }
|
8
|
+
let(:strategy) { pool.strategy }
|
9
|
+
|
10
|
+
context 'default config' do
|
11
|
+
it 'should default to the strategy' do
|
12
|
+
expect(pool.strategy).to be_instance_of(Makara::Strategies::RoundRobin)
|
13
|
+
end
|
14
|
+
end
|
15
|
+
|
16
|
+
context 'bad config' do
|
17
|
+
let(:makara_config) { { :test_strategy => 'SomethingElse::Here' } }
|
18
|
+
it 'should raise name error' do
|
19
|
+
expect {
|
20
|
+
pool
|
21
|
+
}.to raise_error(NameError)
|
22
|
+
end
|
23
|
+
end
|
24
|
+
|
25
|
+
context 'given in config' do
|
26
|
+
let(:makara_config) { { :test_strategy => 'round_robin' } }
|
27
|
+
it 'should use the strategy' do
|
28
|
+
expect(pool.strategy).to be_instance_of(Makara::Strategies::RoundRobin)
|
29
|
+
end
|
30
|
+
end
|
31
|
+
|
32
|
+
|
33
|
+
it 'should loop through with weights' do
|
34
|
+
wrapper_a = pool.add(pool_config){ FakeConnection.new(something: 'a') }
|
35
|
+
wrapper_b = pool.add(pool_config){ FakeConnection.new(something: 'b') }
|
36
|
+
wrapper_c = pool.add(pool_config.merge(weight: 2)){ FakeConnection.new(something: 'c') }
|
37
|
+
|
38
|
+
expect(strategy.current.something).to eql('a')
|
39
|
+
expect(strategy.next.something).to eql('b')
|
40
|
+
expect(strategy.current.something).to eql('b')
|
41
|
+
expect(strategy.current.something).to eql('b')
|
42
|
+
expect(strategy.next.something).to eql('c')
|
43
|
+
expect(strategy.next.something).to eql('c')
|
44
|
+
expect(strategy.next.something).to eql('a')
|
45
|
+
end
|
46
|
+
|
47
|
+
it 'should handle failover to next one' do
|
48
|
+
wrapper_a = pool.add(pool_config){ FakeConnection.new(something: 'a') }
|
49
|
+
wrapper_b = pool.add(pool_config){ FakeConnection.new(something: 'b') }
|
50
|
+
wrapper_c = pool.add(pool_config.merge(weight: 2)){ FakeConnection.new(something: 'c') }
|
51
|
+
|
52
|
+
pool.provide do |connection|
|
53
|
+
if connection == wrapper_a
|
54
|
+
raise Makara::Errors::BlacklistConnection.new(wrapper_a, StandardError.new('failure'))
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
58
|
+
# skips a
|
59
|
+
expect(strategy.current.something).to eql('b')
|
60
|
+
expect(strategy.next.something).to eql('c')
|
61
|
+
expect(strategy.next.something).to eql('c')
|
62
|
+
expect(strategy.next.something).to eql('b')
|
63
|
+
end
|
64
|
+
|
65
|
+
|
66
|
+
|
67
|
+
end
|
@@ -2,8 +2,6 @@ require 'active_record/connection_adapters/makara_abstract_adapter'
|
|
2
2
|
|
3
3
|
class FakeConnection < Struct.new(:config)
|
4
4
|
|
5
|
-
attr_accessor :something
|
6
|
-
|
7
5
|
def ping
|
8
6
|
'ping!'
|
9
7
|
end
|
@@ -23,6 +21,10 @@ class FakeConnection < Struct.new(:config)
|
|
23
21
|
def disconnect!
|
24
22
|
true
|
25
23
|
end
|
24
|
+
|
25
|
+
def something
|
26
|
+
(config || {})[:something]
|
27
|
+
end
|
26
28
|
end
|
27
29
|
|
28
30
|
class FakeDatabaseAdapter < Struct.new(:config)
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: makara
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.3.
|
4
|
+
version: 0.3.6
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Mike Nelson
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2016-
|
11
|
+
date: 2016-04-21 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activerecord
|
@@ -73,6 +73,9 @@ files:
|
|
73
73
|
- lib/makara/pool.rb
|
74
74
|
- lib/makara/proxy.rb
|
75
75
|
- lib/makara/railtie.rb
|
76
|
+
- lib/makara/strategies/abstract.rb
|
77
|
+
- lib/makara/strategies/priority_failover.rb
|
78
|
+
- lib/makara/strategies/round_robin.rb
|
76
79
|
- lib/makara/version.rb
|
77
80
|
- makara.gemspec
|
78
81
|
- spec/active_record/connection_adapters/makara_abstract_adapter_error_handling_spec.rb
|
@@ -87,6 +90,8 @@ files:
|
|
87
90
|
- spec/pool_spec.rb
|
88
91
|
- spec/proxy_spec.rb
|
89
92
|
- spec/spec_helper.rb
|
93
|
+
- spec/strategies/priority_failover_spec.rb
|
94
|
+
- spec/strategies/round_robin_spec.rb
|
90
95
|
- spec/support/configurator.rb
|
91
96
|
- spec/support/deep_dup.rb
|
92
97
|
- spec/support/mock_objects.rb
|
@@ -132,6 +137,8 @@ test_files:
|
|
132
137
|
- spec/pool_spec.rb
|
133
138
|
- spec/proxy_spec.rb
|
134
139
|
- spec/spec_helper.rb
|
140
|
+
- spec/strategies/priority_failover_spec.rb
|
141
|
+
- spec/strategies/round_robin_spec.rb
|
135
142
|
- spec/support/configurator.rb
|
136
143
|
- spec/support/deep_dup.rb
|
137
144
|
- spec/support/mock_objects.rb
|