ar_mysql_flexmaster 1.0.2 → 1.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.travis.yml +9 -10
- data/README.md +60 -54
- data/Rakefile +1 -1
- data/ar_mysql_flexmaster.gemspec +11 -8
- data/bin/master_cut +6 -9
- data/gemfiles/rails3.2.gemfile.lock +42 -42
- data/gemfiles/rails4.2.gemfile +2 -2
- data/gemfiles/rails4.2.gemfile.lock +65 -64
- data/gemfiles/rails5.0.gemfile +2 -2
- data/gemfiles/rails5.0.gemfile.lock +76 -123
- data/lib/active_record/connection_adapters/mysql_flexmaster_adapter.rb +12 -10
- data/test/ar_flexmaster_test.rb +10 -7
- data/test/boot_mysql_env.rb +5 -5
- data/test/boot_slave +4 -4
- data/test/integration/no_traffic_test.rb +0 -1
- data/test/integration/with_queries_to_be_killed_test.rb +1 -1
- data/test/integration/wrong_setup_test.rb +1 -1
- data/test/integration_helper.rb +1 -2
- metadata +32 -15
- data/gemfiles/rails4.0.gemfile +0 -6
- data/gemfiles/rails4.0.gemfile.lock +0 -104
- data/gemfiles/rails4.1.gemfile +0 -6
- data/gemfiles/rails4.1.gemfile.lock +0 -109
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 003ae600df9f49fa3ed61a3f24b50be9a1c1578f
|
4
|
+
data.tar.gz: 964eea239d02d3ac8984f690e73b0d9dcda4f4a8
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 54d905d3643055ddd23984cb162519c7dac4f06613b63f2e0bee48e0f4b742983039f543244af8a425582b3a37fa7a65b6c163df586e802ab3ba13405e83f34e
|
7
|
+
data.tar.gz: 5d10d47653118fb716cf3af3146270e96c403eaf9041ccb1dec81dea699e481d3244eb3ef85917e4ebc6989acf3d8f13968881f382dcedaa236551c7c4ceccf5
|
data/.travis.yml
CHANGED
@@ -1,26 +1,25 @@
|
|
1
1
|
language: ruby
|
2
2
|
|
3
3
|
rvm:
|
4
|
-
- 2.
|
5
|
-
- 2.
|
6
|
-
- 2.
|
4
|
+
- 2.2.6
|
5
|
+
- 2.3.3
|
6
|
+
- 2.4.1
|
7
7
|
|
8
8
|
gemfile:
|
9
9
|
- gemfiles/rails3.2.gemfile
|
10
|
-
- gemfiles/rails4.0.gemfile
|
11
|
-
- gemfiles/rails4.1.gemfile
|
12
10
|
- gemfiles/rails4.2.gemfile
|
13
11
|
- gemfiles/rails5.0.gemfile
|
14
12
|
|
15
|
-
sudo:
|
13
|
+
sudo: required
|
16
14
|
|
17
15
|
bundler_args: --no-deployment
|
18
16
|
|
17
|
+
before_script:
|
18
|
+
- sudo cp /usr/share/doc/mysql-server-5.6/examples/my-default.cnf /usr/share/mysql/my-default.cnf
|
19
|
+
|
19
20
|
script: bundle exec rake test
|
20
21
|
|
21
22
|
matrix:
|
22
23
|
exclude:
|
23
|
-
- rvm: 2.
|
24
|
-
gemfile: gemfiles/
|
25
|
-
- rvm: 2.1.8
|
26
|
-
gemfile: gemfiles/rails5.0.gemfile
|
24
|
+
- rvm: 2.4.1
|
25
|
+
gemfile: gemfiles/rails3.2.gemfile
|
data/README.md
CHANGED
@@ -1,19 +1,20 @@
|
|
1
|
-
[](https://travis-ci.org/zendesk/ar_mysql_flexmaster)
|
2
2
|
|
3
3
|
# Flexmaster
|
4
4
|
|
5
|
-
Flexmaster is an adapter for ActiveRecord and
|
6
|
-
among a list of potential masters at runtime.
|
7
|
-
HA solution (load balancing, middleware)
|
5
|
+
Flexmaster is an adapter for ActiveRecord and MySQL that allows an application
|
6
|
+
node to find a master among a list of potential masters at runtime. It trades
|
7
|
+
some properties of a more traditional HA solution (load balancing, middleware)
|
8
|
+
for simplicity of operation.
|
8
9
|
|
9
10
|
## Configuration:
|
10
11
|
|
11
|
-
Your environment should be configured with 1 active master and N replicas.
|
12
|
-
global
|
13
|
-
depends on it).
|
12
|
+
Your environment should be configured with 1 active master and N replicas. Each
|
13
|
+
replica should have MySQL’s global `READ_ONLY` flag set to true (this is really
|
14
|
+
best practices for your replicas anyway, but Flexmaster depends on it).
|
14
15
|
|
15
|
-
database.yml should contain a list of hosts
|
16
|
-
It should look like this:
|
16
|
+
database.yml should contain a list of hosts – all of them potential masters, all
|
17
|
+
of them potential replicas. It should look like this:
|
17
18
|
|
18
19
|
```
|
19
20
|
production:
|
@@ -28,79 +29,86 @@ production_slave:
|
|
28
29
|
hosts: ["db01:3306", "db02:3306"]
|
29
30
|
```
|
30
31
|
|
31
|
-
In this example, we
|
32
|
-
|
32
|
+
In this example, we’ve configured two different connections for Rails to use.
|
33
|
+
Note that they’re identical except for the `slave: true` key in the
|
34
|
+
`production_slave` YAML block. Adding `slave: true` indicates to Flexmaster that
|
35
|
+
this connection should prefer a read-only slave wherever possible.
|
33
36
|
|
34
37
|
## How it works
|
35
38
|
|
36
39
|
### Overview
|
37
40
|
|
38
|
-
The
|
39
|
-
of the replication chain may be read-write at any given
|
40
|
-
on the
|
41
|
+
The MySQL `READ_ONLY` flag is used to indicate a current master amongst the
|
42
|
+
cluster. Only one member of the replication chain may be read-write at any given
|
43
|
+
time. The application picks in run time, based on the `READ_ONLY` flag, which
|
44
|
+
host is correct.
|
41
45
|
|
42
|
-
###
|
46
|
+
### Boot time
|
43
47
|
|
44
|
-
Your
|
45
|
-
it finds the correct host.
|
48
|
+
Your ActiveRecord application will pick a correct MySQL host for the given
|
49
|
+
configuration by probing hosts until it finds the correct host.
|
46
50
|
|
47
|
-
For master configurations (slave: true is not specified):
|
51
|
+
For master configurations (`slave: true` is not specified):
|
48
52
|
|
49
|
-
The application will probe each host in turn, and find the
|
50
|
-
that is read-write (SET GLOBAL
|
53
|
+
The application will probe each host in turn, and find the MySQL candidate among
|
54
|
+
these nodes that is read-write (`SET GLOBAL READ_ONLY=0`).
|
51
55
|
|
52
|
-
If it finds more than one node where READ_ONLY == 0
|
56
|
+
If it finds more than one node where `READ_ONLY == 0`, it will abort.
|
53
57
|
|
54
|
-
For slave configurations (slave: true specified):
|
58
|
+
For slave configurations (`slave: true` specified):
|
55
59
|
|
56
|
-
The application will choose a replica at random from amongst those where
|
57
|
-
it will fall back to the
|
60
|
+
The application will choose a replica at random from amongst those where
|
61
|
+
`READ_ONLY == 1`. If no active replicas are found, it will fall back to the
|
62
|
+
master.
|
58
63
|
|
59
|
-
###
|
64
|
+
### Run time
|
60
65
|
|
61
|
-
Before each transaction is opened on the master, the application checks the
|
62
|
-
If
|
63
|
-
|
64
|
-
|
65
|
-
|
66
|
+
Before each transaction is opened on the master, the application checks the
|
67
|
+
status of the `READ_ONLY` variable. If `READ_ONLY == 0` (our active connection
|
68
|
+
is still to the current master), it will proceed with the transaction as normal.
|
69
|
+
If `READ_ONLY == 1` (the master has been demoted), it will drop the current
|
70
|
+
connection and re-poll the cluster, sleeping for up to a default of 5 seconds
|
71
|
+
for a new master to be promoted. When it finds the new master, it will continue
|
72
|
+
playing the transaction on it.
|
66
73
|
|
67
|
-
###
|
74
|
+
### Promoting a new master
|
68
75
|
|
69
|
-
*The bin/master_cut script in this project will perform steps 3
|
76
|
+
*The `bin/master_cut` script in this project will perform steps 3–5 for you.*
|
70
77
|
|
71
78
|
The process of promoting a new master to head the cluster should be as follows:
|
72
79
|
|
73
|
-
1.
|
74
|
-
1.
|
75
|
-
chain to look like this:
|
80
|
+
1. Identify a new candidate master.
|
81
|
+
1. Ensure that all other replicas in the cluster are chained off the candidate
|
82
|
+
master; you want the chain to look like this:
|
76
83
|
|
77
84
|
```
|
78
85
|
<existing master> -> <candidate master> -> <other replicas>
|
79
|
-
-> <other replicas>
|
80
|
-
|
81
|
-
```
|
82
|
-
|
83
|
-
1. set the old master to READ_ONLY = 1
|
84
|
-
1. record the master-bin-log position of the candidate master (if you want to re-use the old master as a replica)
|
85
|
-
1. set the new master to READ_ONLY = 0
|
86
|
+
-> <other replicas>
|
86
87
|
|
87
|
-
|
88
|
-
|
88
|
+
```
|
89
|
+
1. Set the old master to `READ_ONLY = 1`.
|
90
|
+
1. Record the master-bin-log position of the candidate master (if you want to
|
91
|
+
re-use the old master as a replica).
|
92
|
+
1. Set the new master to `READ_ONLY = 0`.
|
89
93
|
|
90
|
-
The application will
|
94
|
+
The application nodes will, in time, find that the old master is inactive and
|
95
|
+
will move their connections to the new master.
|
91
96
|
|
92
|
-
|
97
|
+
The application will also eventually shift slave traffic to another node in the
|
98
|
+
cluster.
|
93
99
|
|
94
|
-
|
95
|
-
will crash. In theory there's a workaround for this problem, in pratice it's rather unwieldy due
|
96
|
-
to a lack of shared global variables in mysql.
|
100
|
+
### Caveats and gotchas
|
97
101
|
|
98
|
-
-
|
102
|
+
- Any explicit (`BEGIN` … `END`) transaction that are in-flight when the old
|
103
|
+
master goes `READ_ONLY` will crash. In theory there’s a workaround for this
|
104
|
+
problem, in pratice it’s rather unwieldy due to a lack of shared global
|
105
|
+
variables in MySQL.
|
106
|
+
- Connection variables are unsupported, due to the connection being able to go
|
107
|
+
away at any time.
|
99
108
|
|
100
109
|
## Installation
|
101
110
|
|
102
|
-
|
103
|
-
Add this line to your application's Gemfile:
|
111
|
+
Add this line to your application’s Gemfile:
|
104
112
|
|
105
113
|
gem 'ar_mysql_flexmaster'
|
106
114
|
|
@@ -119,5 +127,3 @@ Or install it yourself as:
|
|
119
127
|
3. Commit your changes (`git commit -am 'Added some feature'`)
|
120
128
|
4. Push to the branch (`git push origin my-new-feature`)
|
121
129
|
5. Create new Pull Request
|
122
|
-
|
123
|
-
|
data/Rakefile
CHANGED
data/ar_mysql_flexmaster.gemspec
CHANGED
@@ -1,27 +1,30 @@
|
|
1
1
|
# -*- encoding: utf-8 -*-
|
2
2
|
|
3
3
|
Gem::Specification.new do |gem|
|
4
|
-
gem.authors = ["Ben Osheroff"]
|
5
|
-
gem.email = ["
|
6
|
-
gem.description =
|
7
|
-
gem.summary =
|
8
|
-
gem.homepage = "http://github.com/
|
4
|
+
gem.authors = ["Ben Osheroff", "Benjamin Quorning", "Gabe Martin-Dempesy", "Michael Grosser", "Pierre Schambacher"]
|
5
|
+
gem.email = ["bquorning@zendesk.com", "gabe@zendesk.com", "mgrosser@zendesk.com", "pschambacher@zendesk.com"]
|
6
|
+
gem.description = "ar_mysql_flexmaster allows configuring N mysql servers in database.yml and auto-selects which is a master at runtime"
|
7
|
+
gem.summary = "select a master at runtime from a list"
|
8
|
+
gem.homepage = "http://github.com/zendesk/ar_mysql_flexmaster"
|
9
9
|
|
10
10
|
gem.files = `git ls-files`.split($\)
|
11
|
-
gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
|
11
|
+
gem.executables = gem.files.grep(%r{^bin/}).map { |f| File.basename(f) }
|
12
12
|
gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
|
13
13
|
gem.name = "ar_mysql_flexmaster"
|
14
14
|
gem.require_paths = ["lib"]
|
15
|
-
gem.version = "1.0.
|
15
|
+
gem.version = "1.0.3"
|
16
|
+
|
17
|
+
gem.required_ruby_version = ">= 2.2"
|
16
18
|
|
17
19
|
gem.add_runtime_dependency("mysql2")
|
18
20
|
gem.add_runtime_dependency("activerecord")
|
19
21
|
gem.add_runtime_dependency("activesupport")
|
22
|
+
gem.add_development_dependency("bundler")
|
20
23
|
gem.add_development_dependency("rake")
|
21
24
|
gem.add_development_dependency("wwtd")
|
22
25
|
gem.add_development_dependency("minitest")
|
23
26
|
gem.add_development_dependency("mocha", "~> 1.1.0")
|
24
27
|
gem.add_development_dependency("bump")
|
25
28
|
gem.add_development_dependency("pry")
|
26
|
-
gem.add_development_dependency("
|
29
|
+
gem.add_development_dependency("isolated_server")
|
27
30
|
end
|
data/bin/master_cut
CHANGED
@@ -10,8 +10,8 @@ Thread.abort_on_exception = false
|
|
10
10
|
|
11
11
|
opts = GetoptLong.new(
|
12
12
|
["--password", "-p", GetoptLong::REQUIRED_ARGUMENT],
|
13
|
-
["--rehome-master",
|
14
|
-
["--start-slave", "-s", GetoptLong::NO_ARGUMENT]
|
13
|
+
["--rehome-master", "-r", GetoptLong::NO_ARGUMENT],
|
14
|
+
["--start-slave", "-s", GetoptLong::NO_ARGUMENT]
|
15
15
|
)
|
16
16
|
|
17
17
|
opts.each do |opt, arg|
|
@@ -39,7 +39,6 @@ unless $old_master && $new_master && $username
|
|
39
39
|
usage
|
40
40
|
end
|
41
41
|
|
42
|
-
|
43
42
|
def open_cx(host)
|
44
43
|
host, port = host.split(":")
|
45
44
|
port = port.to_i if port
|
@@ -121,7 +120,7 @@ def process_kill_thread
|
|
121
120
|
end
|
122
121
|
|
123
122
|
def wait_for_slave_catchup(master, slave)
|
124
|
-
|
123
|
+
loop do
|
125
124
|
master_info = master.query("show master status").first
|
126
125
|
slave_info = slave.query("show slave status").first
|
127
126
|
break if master_info['Position'] <= slave_info['Exec_Master_Log_Pos']
|
@@ -129,11 +128,9 @@ def wait_for_slave_catchup(master, slave)
|
|
129
128
|
end
|
130
129
|
|
131
130
|
def kill_query!(cx, id)
|
132
|
-
|
133
|
-
|
134
|
-
|
135
|
-
raise e unless e.errno == 1094 # unknown thread id error
|
136
|
-
end
|
131
|
+
cx.query("kill #{id}")
|
132
|
+
rescue Mysql2::Error => e
|
133
|
+
raise e unless e.errno == 1094 # unknown thread id error
|
137
134
|
end
|
138
135
|
|
139
136
|
def swap_thread
|
@@ -1,7 +1,7 @@
|
|
1
1
|
PATH
|
2
|
-
remote:
|
2
|
+
remote: ..
|
3
3
|
specs:
|
4
|
-
ar_mysql_flexmaster (0.
|
4
|
+
ar_mysql_flexmaster (1.0.2)
|
5
5
|
activerecord
|
6
6
|
activesupport
|
7
7
|
mysql2
|
@@ -9,12 +9,12 @@ PATH
|
|
9
9
|
GEM
|
10
10
|
remote: https://rubygems.org/
|
11
11
|
specs:
|
12
|
-
actionmailer (3.2.22)
|
13
|
-
actionpack (= 3.2.22)
|
12
|
+
actionmailer (3.2.22.5)
|
13
|
+
actionpack (= 3.2.22.5)
|
14
14
|
mail (~> 2.5.4)
|
15
|
-
actionpack (3.2.22)
|
16
|
-
activemodel (= 3.2.22)
|
17
|
-
activesupport (= 3.2.22)
|
15
|
+
actionpack (3.2.22.5)
|
16
|
+
activemodel (= 3.2.22.5)
|
17
|
+
activesupport (= 3.2.22.5)
|
18
18
|
builder (~> 3.0.0)
|
19
19
|
erubis (~> 2.7.0)
|
20
20
|
journey (~> 1.0.4)
|
@@ -22,69 +22,69 @@ GEM
|
|
22
22
|
rack-cache (~> 1.2)
|
23
23
|
rack-test (~> 0.6.1)
|
24
24
|
sprockets (~> 2.2.1)
|
25
|
-
activemodel (3.2.22)
|
26
|
-
activesupport (= 3.2.22)
|
25
|
+
activemodel (3.2.22.5)
|
26
|
+
activesupport (= 3.2.22.5)
|
27
27
|
builder (~> 3.0.0)
|
28
|
-
activerecord (3.2.22)
|
29
|
-
activemodel (= 3.2.22)
|
30
|
-
activesupport (= 3.2.22)
|
28
|
+
activerecord (3.2.22.5)
|
29
|
+
activemodel (= 3.2.22.5)
|
30
|
+
activesupport (= 3.2.22.5)
|
31
31
|
arel (~> 3.0.2)
|
32
32
|
tzinfo (~> 0.3.29)
|
33
|
-
activeresource (3.2.22)
|
34
|
-
activemodel (= 3.2.22)
|
35
|
-
activesupport (= 3.2.22)
|
36
|
-
activesupport (3.2.22)
|
33
|
+
activeresource (3.2.22.5)
|
34
|
+
activemodel (= 3.2.22.5)
|
35
|
+
activesupport (= 3.2.22.5)
|
36
|
+
activesupport (3.2.22.5)
|
37
37
|
i18n (~> 0.6, >= 0.6.4)
|
38
38
|
multi_json (~> 1.0)
|
39
39
|
arel (3.0.3)
|
40
40
|
builder (3.0.4)
|
41
|
-
bump (0.5.
|
42
|
-
coderay (1.1.
|
41
|
+
bump (0.5.4)
|
42
|
+
coderay (1.1.1)
|
43
43
|
erubis (2.7.0)
|
44
44
|
hike (1.2.3)
|
45
|
-
i18n (0.
|
45
|
+
i18n (0.8.6)
|
46
|
+
isolated_server (0.4.12)
|
46
47
|
journey (1.0.4)
|
47
|
-
json (1.8.
|
48
|
-
mail (2.5.
|
48
|
+
json (1.8.6)
|
49
|
+
mail (2.5.5)
|
49
50
|
mime-types (~> 1.16)
|
50
51
|
treetop (~> 1.4.8)
|
51
52
|
metaclass (0.0.4)
|
52
53
|
method_source (0.8.2)
|
53
54
|
mime-types (1.25.1)
|
54
|
-
minitest (5.
|
55
|
+
minitest (5.10.3)
|
55
56
|
mocha (1.1.0)
|
56
57
|
metaclass (~> 0.0.1)
|
57
|
-
multi_json (1.
|
58
|
-
mysql2 (0.3.
|
59
|
-
mysql_isolated_server (0.5.3)
|
58
|
+
multi_json (1.12.1)
|
59
|
+
mysql2 (0.3.21)
|
60
60
|
polyglot (0.3.5)
|
61
|
-
pry (0.10.
|
61
|
+
pry (0.10.4)
|
62
62
|
coderay (~> 1.1.0)
|
63
63
|
method_source (~> 0.8.1)
|
64
64
|
slop (~> 3.4)
|
65
65
|
rack (1.4.7)
|
66
|
-
rack-cache (1.
|
66
|
+
rack-cache (1.7.0)
|
67
67
|
rack (>= 0.4)
|
68
68
|
rack-ssl (1.3.4)
|
69
69
|
rack
|
70
70
|
rack-test (0.6.3)
|
71
71
|
rack (>= 1.0)
|
72
|
-
rails (3.2.22)
|
73
|
-
actionmailer (= 3.2.22)
|
74
|
-
actionpack (= 3.2.22)
|
75
|
-
activerecord (= 3.2.22)
|
76
|
-
activeresource (= 3.2.22)
|
77
|
-
activesupport (= 3.2.22)
|
72
|
+
rails (3.2.22.5)
|
73
|
+
actionmailer (= 3.2.22.5)
|
74
|
+
actionpack (= 3.2.22.5)
|
75
|
+
activerecord (= 3.2.22.5)
|
76
|
+
activeresource (= 3.2.22.5)
|
77
|
+
activesupport (= 3.2.22.5)
|
78
78
|
bundler (~> 1.0)
|
79
|
-
railties (= 3.2.22)
|
80
|
-
railties (3.2.22)
|
81
|
-
actionpack (= 3.2.22)
|
82
|
-
activesupport (= 3.2.22)
|
79
|
+
railties (= 3.2.22.5)
|
80
|
+
railties (3.2.22.5)
|
81
|
+
actionpack (= 3.2.22.5)
|
82
|
+
activesupport (= 3.2.22.5)
|
83
83
|
rack-ssl (~> 1.3.2)
|
84
84
|
rake (>= 0.8.7)
|
85
85
|
rdoc (~> 3.4)
|
86
86
|
thor (>= 0.14.6, < 2.0)
|
87
|
-
rake (
|
87
|
+
rake (12.0.0)
|
88
88
|
rdoc (3.12.2)
|
89
89
|
json (~> 1.4)
|
90
90
|
slop (3.6.0)
|
@@ -93,12 +93,12 @@ GEM
|
|
93
93
|
multi_json (~> 1.0)
|
94
94
|
rack (~> 1.0)
|
95
95
|
tilt (~> 1.1, != 1.3.0)
|
96
|
-
thor (0.
|
96
|
+
thor (0.20.0)
|
97
97
|
tilt (1.4.1)
|
98
98
|
treetop (1.4.15)
|
99
99
|
polyglot
|
100
100
|
polyglot (>= 0.3.1)
|
101
|
-
tzinfo (0.3.
|
101
|
+
tzinfo (0.3.53)
|
102
102
|
wwtd (1.3.0)
|
103
103
|
|
104
104
|
PLATFORMS
|
@@ -107,14 +107,14 @@ PLATFORMS
|
|
107
107
|
DEPENDENCIES
|
108
108
|
ar_mysql_flexmaster!
|
109
109
|
bump
|
110
|
+
isolated_server
|
110
111
|
minitest
|
111
112
|
mocha (~> 1.1.0)
|
112
113
|
mysql2 (~> 0.3.0)
|
113
|
-
mysql_isolated_server (~> 0.5)
|
114
114
|
pry
|
115
115
|
rails (~> 3.2.0)
|
116
116
|
rake
|
117
117
|
wwtd
|
118
118
|
|
119
119
|
BUNDLED WITH
|
120
|
-
1.
|
120
|
+
1.15.4
|