redis_failover 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/.gitignore ADDED
@@ -0,0 +1,17 @@
1
+ *.gem
2
+ *.rbc
3
+ .bundle
4
+ .config
5
+ .yardoc
6
+ Gemfile.lock
7
+ InstalledFiles
8
+ _yardoc
9
+ coverage
10
+ doc/
11
+ lib/bundler/man
12
+ pkg
13
+ rdoc
14
+ spec/reports
15
+ test/tmp
16
+ test/version_tmp
17
+ tmp
data/Changes.md ADDED
@@ -0,0 +1,4 @@
1
+ 0.1.0
2
+ -----------
3
+
4
+ - First release
data/Gemfile ADDED
@@ -0,0 +1,2 @@
1
+ source 'https://rubygems.org'
2
+ gemspec
data/LICENSE ADDED
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2012 Ryan LeCompte
2
+
3
+ MIT License
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining
6
+ a copy of this software and associated documentation files (the
7
+ "Software"), to deal in the Software without restriction, including
8
+ without limitation the rights to use, copy, modify, merge, publish,
9
+ distribute, sublicense, and/or sell copies of the Software, and to
10
+ permit persons to whom the Software is furnished to do so, subject to
11
+ the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be
14
+ included in all copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,106 @@
1
+ # Redis Failover Client/Server
2
+
3
+ Redis Failover attempts to provides a full automatic master/slave failover solution for Ruby. Redis does not provide
4
+ an automatic failover capability when configured for master/slave replication. When the master node dies,
5
+ a new master must be manually brought online and assigned as the slave's new master. This manual
6
+ switch-over is not desirable in high traffic sites where Redis is a critical part of the overall
7
+ architecture. The existing standard Redis client for Ruby also only supports configuration for a single
8
+ Redis server. When using master/slave replication, it is desirable to have all writes go to the
9
+ master, and all reads go to one of the N configured slaves.
10
+
11
+ This gem attempts to address both the server and client problems. A redis failover server runs as a background
12
+ daemon and monitors all of your configured master/slave nodes. When the server starts up, it
13
+ automatically discovers who is the master and who are the slaves. Watchers are setup for each of
14
+ the redis nodes. As soon as a node is detected as being offline, it will be moved to an "unreachable" state.
15
+ If the node that went offline was the master, then one of the slaves will be promoted as the new master.
16
+ All existing slaves will be automatically reconfigured to point to the new master for replication.
17
+ All nodes marked as unreachable will be periodically checked to see if they have been brought back online.
18
+ If so, the newly reachable nodes will be configured as slaves and brought back into the list of live
19
+ servers. Note that detection of a node going down should be nearly instantaneous, since the mechanism
20
+ used to keep tabs on a node is via a blocking Redis BLPOP call (no polling). This call fails nearly
21
+ immediately when the node actually goes offline.
22
+
23
+ This gem provides a RedisFailover::Client wrapper that is master/slave aware. The client is configured
24
+ with a single host/port pair that points to redis failover server. The client will automatically
25
+ connect to the server to find out the current state of the world (i.e., who's the current master and
26
+ who are the current slaves). The client also acts as a load balancer in that it will automatically
27
+ dispatch Redis read operations to one of N slaves, and Redis write operations to the master.
28
+ If it fails to communicate with any node, it will go back and ask the server for the current list of
29
+ available servers, and then optionally retry the operation.
30
+
31
+ ## Installation
32
+
33
+ Add this line to your application's Gemfile:
34
+
35
+ gem 'redis_failover'
36
+
37
+ And then execute:
38
+
39
+ $ bundle
40
+
41
+ Or install it yourself as:
42
+
43
+ $ gem install redis_failover
44
+
45
+ ## Server Usage
46
+
47
+ The redis failover server is a simple process that should be run as a background daemon. The server supports the
48
+ following options:
49
+
50
+ Usage: redis_failover_server [OPTIONS]
51
+ -P, --port port Server port
52
+ -p, --password password Redis password
53
+ -n, --nodes nodes Comma-separated redis host:port pairs
54
+ --max-failures count Max failures before server marks node unreachable (default 3)
55
+ -h, --help Display all options
56
+
57
+ To start the server for a simple master/slave configuration, use the following:
58
+
59
+ redis_failover_server -P 3000 -n localhost:6379,localhost:6380
60
+
61
+ The server will automatically figure out who is the master and who is the slave upon startup. Note that it is
62
+ a good idea to monitor the redis failover server process with a tool like Monit to ensure that it is restarted
63
+ in the case of a failure.
64
+
65
+ ## Client Usage
66
+
67
+ The redis failover client must be used in conjunction with a running redis failover server. The
68
+ client supports various configuration options, however the two mandatory options are the host
69
+ and port of the redis failover server:
70
+
71
+ client = RedisFailover::Client.new(:host => 'localhost', :port => 3000)
72
+
73
+ The client actually employs the common redis and redis-namespace gems underneath, so this should be
74
+ a drop-in replacement for your existing pure redis client usage.
75
+
76
+ The full set of options that can be passed to RedisFailover::Client are:
77
+
78
+ :host - redis failover server host (required)
79
+ :port - redis failover server port (required)
80
+ :password - optional password for redis nodes
81
+ :namespace - optional namespace for redis nodes
82
+ :logger - optional logger override
83
+ :retry_failure - indicate if failures should be retried (default true)
84
+ :max_retries - max retries for a failure (default 3)
85
+
86
+ ## Resources
87
+
88
+ To learn more about Redis master/slave replication, see the [Redis documentation](http://redis.io/topics/replication).
89
+
90
+ ## License
91
+
92
+ Please see LICENSE for licensing details.
93
+
94
+ ## Author
95
+
96
+ Ryan LeCompte
97
+
98
+ [@ryanlecompte](https://twitter.com/ryanlecompte)
99
+
100
+ ## Contributing
101
+
102
+ 1. Fork it
103
+ 2. Create your feature branch (`git checkout -b my-new-feature`)
104
+ 3. Commit your changes (`git commit -am 'Added some feature'`)
105
+ 4. Push to the branch (`git push origin my-new-feature`)
106
+ 5. Create new Pull Request
data/Rakefile ADDED
@@ -0,0 +1,9 @@
1
+ #!/usr/bin/env rake
2
+ require "bundler/gem_tasks"
3
+ require 'rspec/core/rake_task'
4
+
5
+ RSpec::Core::RakeTask.new(:spec) do |t|
6
+ t.rspec_opts = %w(--format progress)
7
+ end
8
+
9
+ task :default => [:spec]
@@ -0,0 +1,6 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require 'redis_failover'
4
+
5
+ RedisFailover::Runner.run(ARGV)
6
+
@@ -0,0 +1,16 @@
1
+ require 'redis'
2
+ require 'thread'
3
+ require 'logger'
4
+ require 'multi_json'
5
+ require 'securerandom'
6
+
7
+ require 'redis_failover/cli'
8
+ require 'redis_failover/util'
9
+ require 'redis_failover/node'
10
+ require 'redis_failover/errors'
11
+ require 'redis_failover/client'
12
+ require 'redis_failover/server'
13
+ require 'redis_failover/runner'
14
+ require 'redis_failover/version'
15
+ require 'redis_failover/node_manager'
16
+ require 'redis_failover/node_watcher'
@@ -0,0 +1,47 @@
1
+ module RedisFailover
2
+ # Parses server command-line arguments.
3
+ class CLI
4
+ def self.parse(source)
5
+ return {} if source.empty?
6
+
7
+ options = {}
8
+ parser = OptionParser.new do |opts|
9
+ opts.banner = "Usage: redis_failover_server [OPTIONS]"
10
+
11
+ opts.on('-P', '--port port', 'Server port') do |port|
12
+ options[:port] = Integer(port)
13
+ end
14
+
15
+ opts.on('-p', '--password password', 'Redis password') do |password|
16
+ options[:password] = password.strip
17
+ end
18
+
19
+ opts.on('-n', '--nodes nodes', 'Comma-separated redis host:port pairs') do |nodes|
20
+ # turns 'host1:port,host2:port' => [{:host => host, :port => port}, ...]
21
+ options[:nodes] = nodes.split(',').map do |node|
22
+ Hash[[:host, :port].zip(node.strip.split(':'))]
23
+ end
24
+ end
25
+
26
+ opts.on('--max-failures count',
27
+ 'Max failures before server marks node unreachable (default 3)') do |max|
28
+ options[:max_failures] = Integer(max)
29
+ end
30
+
31
+ opts.on('-h', '--help', 'Display all options') do
32
+ puts opts
33
+ exit
34
+ end
35
+ end
36
+
37
+ parser.parse(source)
38
+
39
+ # assume password is same for all redis nodes
40
+ if password = options[:password]
41
+ options[:nodes].each { |opts| opts.update(:password => password) }
42
+ end
43
+
44
+ options
45
+ end
46
+ end
47
+ end
@@ -0,0 +1,198 @@
1
+ require 'set'
2
+ require 'open-uri'
3
+
4
+ module RedisFailover
5
+ # Redis failover-aware client.
6
+ class Client
7
+ include Util
8
+
9
+ REDIS_ERRORS = Errno.constants.map { |c| Errno.const_get(c) }.freeze
10
+ REDIS_READ_OPS = Set[
11
+ :dbsize,
12
+ :echo,
13
+ :exists,
14
+ :get,
15
+ :getbit,
16
+ :getrange,
17
+ :hexists,
18
+ :hget,
19
+ :hgetall,
20
+ :hkeys,
21
+ :hlen,
22
+ :hmget,
23
+ :hvals,
24
+ :info,
25
+ :keys,
26
+ :lastsave,
27
+ :lindex,
28
+ :llen,
29
+ :lrange,
30
+ :mapped_hmget,
31
+ :mapped_mget,
32
+ :mget,
33
+ :ping,
34
+ :scard,
35
+ :sdiff,
36
+ :select,
37
+ :sinter,
38
+ :sismember,
39
+ :smembers,
40
+ :srandmember,
41
+ :strlen,
42
+ :sunion,
43
+ :ttl,
44
+ :type,
45
+ :zcard,
46
+ :zcount,
47
+ :zrange,
48
+ :zrangebyscore,
49
+ :zrank,
50
+ :zrevrange,
51
+ :zrevrangebyscore,
52
+ :zrevrank,
53
+ :zscore
54
+ ].freeze
55
+
56
+ # Performance optimization: to avoid unnecessary method_missing calls,
57
+ # we proactively define methods that dispatch to the underlying redis
58
+ # calls.
59
+ Redis.public_instance_methods(false).each do |method|
60
+ define_method(method) do |*args, &block|
61
+ dispatch(method, *args, &block)
62
+ end
63
+ end
64
+
65
+ # Creates a new failover redis client.
66
+ #
67
+ # Options:
68
+ #
69
+ # :host - redis failover server host (required)
70
+ # :port - redis failover server port (required)
71
+ # :password - optional password for redis nodes
72
+ # :namespace - optional namespace for redis nodes
73
+ # :logger - optional logger override
74
+ # :retry_failure - indicate if failures should be retried (default true)
75
+ # :max_retries - max retries for a failure (default 5)
76
+ #
77
+ def initialize(options = {})
78
+ unless options.values_at(:host, :port).all?
79
+ raise ArgumentError, ':host and :port options required'
80
+ end
81
+
82
+ Util.logger = options[:logger] if options[:logger]
83
+ @namespace = options[:namespace]
84
+ @password = options[:password]
85
+ @retry = options[:retry_failure] || true
86
+ @max_retries = options[:max_retries] || 3
87
+ @registry_url = "http://#{options[:host]}:#{options[:port]}/redis_servers"
88
+ @redis_servers = nil
89
+ @master = nil
90
+ @slaves = []
91
+ @lock = Mutex.new
92
+ build_clients
93
+ end
94
+
95
+ def method_missing(method, *args, &block)
96
+ if redis_operation?(method)
97
+ dispatch(method, *args, &block)
98
+ else
99
+ super
100
+ end
101
+ end
102
+
103
+ def respond_to?(method)
104
+ redis_operation?(method) || super
105
+ end
106
+
107
+ def inspect
108
+ "#<RedisFailover::Client - master: #{master_info}, slaves: #{slaves_info})>"
109
+ end
110
+ alias_method :to_s, :inspect
111
+
112
+ private
113
+
114
+ def redis_operation?(method)
115
+ Redis.public_instance_methods(false).include?(method)
116
+ end
117
+
118
+ def dispatch(method, *args, &block)
119
+ tries = 0
120
+ begin
121
+ if REDIS_READ_OPS.include?(method)
122
+ # send read operations to a slave
123
+ slave.send(method, *args, &block)
124
+ else
125
+ # direct everything else to master
126
+ master.send(method, *args, &block)
127
+ end
128
+ rescue NoMasterError, *REDIS_ERRORS
129
+ logger.error("No suitable node available for operation `#{method}.`")
130
+ sleep(3)
131
+ build_clients
132
+
133
+ if @retry && tries < @max_retries
134
+ tries += 1
135
+ retry
136
+ end
137
+
138
+ raise
139
+ end
140
+ end
141
+
142
+ def master
143
+ @master or raise NoMasterError
144
+ end
145
+
146
+ def slave
147
+ # pick a slave, if none available fallback to master
148
+ @slaves.sample || master
149
+ end
150
+
151
+ def build_clients
152
+ @lock.synchronize do
153
+ begin
154
+ logger.info('Attempting to fetch nodes and build redis clients.')
155
+ servers = fetch_redis_servers
156
+ master = new_clients_for(servers[:master]).first if servers[:master]
157
+ slaves = new_clients_for(*servers[:slaves])
158
+
159
+ # once clients are successfully created, swap the references
160
+ @master = master
161
+ @slaves = slaves
162
+ rescue => ex
163
+ logger.error("Failed to fetch servers from #{@registry_url} - #{ex.message}")
164
+ logger.error(ex.backtrace.join("\n"))
165
+ end
166
+ end
167
+ end
168
+
169
+ def fetch_redis_servers
170
+ open(@registry_url) do |io|
171
+ servers = symbolize_keys(MultiJson.decode(io))
172
+ logger.info("Fetched servers: #{servers}")
173
+ servers
174
+ end
175
+ end
176
+
177
+ def new_clients_for(*nodes)
178
+ nodes.map do |node|
179
+ host, port = node.split(':')
180
+ client = Redis.new(:host => host, :port => port, :password => @password)
181
+ if @namespace
182
+ client = Redis::Namespace.new(@namespace, :redis => client)
183
+ end
184
+ client
185
+ end
186
+ end
187
+
188
+ def master_info
189
+ return "none" unless @master
190
+ "#{@master.client.host}:#{@master.client.port}"
191
+ end
192
+
193
+ def slaves_info
194
+ return "none" if @slaves.empty?
195
+ @slaves.map { |s| "#{s.client.host}:#{s.client.port}" }.join(', ')
196
+ end
197
+ end
198
+ end
@@ -0,0 +1,25 @@
1
+ module RedisFailover
2
+ class Error < StandardError
3
+ end
4
+
5
+ class InvalidNodeError < Error
6
+ end
7
+
8
+ class InvalidNodeStateError < Error
9
+ def initialize(node, state)
10
+ super("Invalid state change `#{state}` for node #{node}")
11
+ end
12
+ end
13
+
14
+ class NodeUnreachableError < Error
15
+ def initialize(node)
16
+ super("Node: #{node}")
17
+ end
18
+ end
19
+
20
+ class NoMasterError < Error
21
+ end
22
+
23
+ class NoSlaveError < Error
24
+ end
25
+ end