slavery 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.gitignore +20 -0
- data/.rspec +1 -0
- data/Gemfile +3 -0
- data/LICENSE.txt +22 -0
- data/README.md +108 -0
- data/Rakefile +6 -0
- data/lib/slavery.rb +89 -0
- data/lib/slavery/railtie.rb +11 -0
- data/lib/slavery/version.rb +3 -0
- data/slavery.gemspec +25 -0
- data/spec/slavery_spec.rb +55 -0
- data/spec/spec_helper.rb +33 -0
- metadata +124 -0
data/.gitignore
ADDED
data/.rspec
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
--color
|
data/Gemfile
ADDED
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
Copyright (c) 2012 Kenn Ejima
|
|
2
|
+
|
|
3
|
+
MIT License
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
|
6
|
+
a copy of this software and associated documentation files (the
|
|
7
|
+
"Software"), to deal in the Software without restriction, including
|
|
8
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
|
9
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
|
10
|
+
permit persons to whom the Software is furnished to do so, subject to
|
|
11
|
+
the following conditions:
|
|
12
|
+
|
|
13
|
+
The above copyright notice and this permission notice shall be
|
|
14
|
+
included in all copies or substantial portions of the Software.
|
|
15
|
+
|
|
16
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
|
17
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
|
18
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
|
19
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
|
20
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
|
21
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
|
22
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
# Slavery - Simple, conservative slave read for ActiveRecord
|
|
2
|
+
|
|
3
|
+
Slavery is a simple, easy to use plugin for ActiveRecord that enables conservative slave reads, which means it doesn't automatically redirect all SELECTs to slaves. Instead, it lets you specify `Slavery.on_slave { User.count }` to send a particular query to a slave.
|
|
4
|
+
|
|
5
|
+
Probably you just start off with one single database. As your app grows, you would move to master-slave replication for redundancy. At this point, all queries still go to the master and slaves are just backups. With that configuration, it's tempting to run some long-running queries on the slave. And that's exactly what Slavery does.
|
|
6
|
+
|
|
7
|
+
* Conservative - Safe by default. Installing Slavery won't change your app's current behavior.
|
|
8
|
+
* Future proof - No dirty hacks, simply works as a proxy for `ActiveRecord::Base.connection`.
|
|
9
|
+
* Simple - Less than 100 LOC, you can read the entire source and completely stay in control.
|
|
10
|
+
|
|
11
|
+
Slavery works with ActiveRecord 3 or later.
|
|
12
|
+
|
|
13
|
+
## Install
|
|
14
|
+
|
|
15
|
+
Add this line to your application's Gemfile:
|
|
16
|
+
|
|
17
|
+
```ruby
|
|
18
|
+
gem 'slavery'
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
And create slave configs for each environment.
|
|
22
|
+
|
|
23
|
+
```yaml
|
|
24
|
+
development:
|
|
25
|
+
adapter: mysql2
|
|
26
|
+
username: root
|
|
27
|
+
database: myapp_development
|
|
28
|
+
|
|
29
|
+
development_slave:
|
|
30
|
+
adapter: mysql2
|
|
31
|
+
username: root
|
|
32
|
+
database: myapp_development
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
By convention, config keys with `[env]_slave` are automatically used for slave reads.
|
|
36
|
+
|
|
37
|
+
Notice that you just copied the settings for `development` to `development_slave`. For `development` and `test`, it's actually recommended as probably you don't want to have replicating multiple databases on your machine. Two connections to the same identical database should be fine for testing purpose.
|
|
38
|
+
|
|
39
|
+
At this point, Slavery does nothing. Run tests and confirm that anything isn't broken.
|
|
40
|
+
|
|
41
|
+
## Usage
|
|
42
|
+
|
|
43
|
+
To start using Slavery, you need to add `Slavery.on_slave` in your code. Queries in the `Slavery.on_slave` block run on the slave.
|
|
44
|
+
|
|
45
|
+
```ruby
|
|
46
|
+
Slavery.on_slave { User.count } # => runs on slave
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
You can nest `on_slave` and `on_master` interchangeably. The following code works as expected.
|
|
50
|
+
|
|
51
|
+
```ruby
|
|
52
|
+
Slavery.on_slave do
|
|
53
|
+
...
|
|
54
|
+
Slavery.on_master do
|
|
55
|
+
...
|
|
56
|
+
end
|
|
57
|
+
...
|
|
58
|
+
end
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
## Read-only user
|
|
62
|
+
|
|
63
|
+
For an extra safeguard, it is recommended to use a read-only user for slave access.
|
|
64
|
+
|
|
65
|
+
```ruby
|
|
66
|
+
development_slave:
|
|
67
|
+
adapter: mysql2
|
|
68
|
+
username: readonly
|
|
69
|
+
database: myapp_development
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
With MySQL, `GRANT SELECT` creates the user.
|
|
73
|
+
|
|
74
|
+
```SQL
|
|
75
|
+
GRANT SELECT ON *.* TO 'readonly'@'localhost';
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
With this setting, writes on slave should raises an exception.
|
|
79
|
+
|
|
80
|
+
```ruby
|
|
81
|
+
Slavery.on_slave { User.create } # => ActiveRecord::StatementInvalid: Mysql2::Error: INSERT command denied...
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
## Database failure
|
|
85
|
+
|
|
86
|
+
When one of the master or the slave goes down, you would rewrite `database.yml` to make all queries go to the surviving database, until you recover or rebuild the failed one.
|
|
87
|
+
|
|
88
|
+
In such an event, you don't want to manually remove `Slavery.on_slave` from your code. Instead, just put the following line in `config/initializers/slavery.rb`.
|
|
89
|
+
|
|
90
|
+
```ruby
|
|
91
|
+
Slavely.disabled = true
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
With this line, Slavery stops connection switching and all queries go to the master database.
|
|
95
|
+
|
|
96
|
+
## Support for non-Rails apps
|
|
97
|
+
|
|
98
|
+
If you're using ActiveRecord in a non-Rails app (e.g. Sinatra), be sure to set `Slavery.env` in the boot sequence.
|
|
99
|
+
|
|
100
|
+
```ruby
|
|
101
|
+
Slavery.env = 'development'
|
|
102
|
+
|
|
103
|
+
ActiveRecord::Base.configurations = {
|
|
104
|
+
'development' => { adapter: 'mysql2', ... },
|
|
105
|
+
'development_slave' => { adapter: 'mysql2', ... }
|
|
106
|
+
}
|
|
107
|
+
ActiveRecord::Base.establish_connection(:development)
|
|
108
|
+
```
|
data/Rakefile
ADDED
data/lib/slavery.rb
ADDED
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
require 'slavery/version'
|
|
2
|
+
require 'slavery/railtie'
|
|
3
|
+
require 'active_record'
|
|
4
|
+
require 'active_support/core_ext/module/attribute_accessors'
|
|
5
|
+
|
|
6
|
+
module Slavery
|
|
7
|
+
extend ActiveSupport::Concern
|
|
8
|
+
|
|
9
|
+
included do
|
|
10
|
+
class << self
|
|
11
|
+
alias_method_chain :connection, :slavery
|
|
12
|
+
end
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
class Error < StandardError; end
|
|
16
|
+
|
|
17
|
+
mattr_accessor :disabled
|
|
18
|
+
|
|
19
|
+
module ModuleFunctions
|
|
20
|
+
def on_slave(&block)
|
|
21
|
+
ActiveRecord::Base.on_slave(&block)
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
def on_master(&block)
|
|
25
|
+
ActiveRecord::Base.on_master(&block)
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
def env
|
|
29
|
+
self.env = defined?(Rails) ? Rails.env.to_s : 'development' unless @env
|
|
30
|
+
@env
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
def env=(string)
|
|
34
|
+
@env = ActiveSupport::StringInquirer.new(string)
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
extend ModuleFunctions
|
|
38
|
+
|
|
39
|
+
module ClassMethods
|
|
40
|
+
def on_slave(&block)
|
|
41
|
+
with_slavery true, &block
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
def on_master(&block)
|
|
45
|
+
with_slavery false, &block
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def with_slavery(new_value)
|
|
49
|
+
old_value = Thread.current[:on_slave] # Save for recursive nested calls
|
|
50
|
+
Thread.current[:on_slave] = new_value
|
|
51
|
+
yield
|
|
52
|
+
ensure
|
|
53
|
+
Thread.current[:on_slave] = old_value
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
def connection_with_slavery
|
|
57
|
+
if Thread.current[:on_slave] and slaveryable?
|
|
58
|
+
slave_connection
|
|
59
|
+
else
|
|
60
|
+
master_connection
|
|
61
|
+
end
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
def slaveryable?
|
|
65
|
+
inside_transaction = master_connection.open_transactions > 0
|
|
66
|
+
raise Error.new('on_slave cannot be used inside transaction block!') if inside_transaction
|
|
67
|
+
|
|
68
|
+
!Slavery.disabled
|
|
69
|
+
end
|
|
70
|
+
|
|
71
|
+
def master_connection
|
|
72
|
+
connection_without_slavery
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
def slave_connection
|
|
76
|
+
slave_connection_holder.connection_without_slavery
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
# Create an anonymous AR class to hold slave connection
|
|
80
|
+
def slave_connection_holder
|
|
81
|
+
@slave_connection_holder ||= Class.new(ActiveRecord::Base) {
|
|
82
|
+
self.abstract_class = true
|
|
83
|
+
establish_connection("#{Slavery.env}_slave")
|
|
84
|
+
}
|
|
85
|
+
rescue ActiveRecord::AdapterNotSpecified
|
|
86
|
+
raise Error.new("#{Slavery.env}_slave does not exist!")
|
|
87
|
+
end
|
|
88
|
+
end
|
|
89
|
+
end
|
data/slavery.gemspec
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# -*- encoding: utf-8 -*-
|
|
2
|
+
lib = File.expand_path('../lib', __FILE__)
|
|
3
|
+
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
|
4
|
+
require 'slavery/version'
|
|
5
|
+
|
|
6
|
+
Gem::Specification.new do |gem|
|
|
7
|
+
gem.name = 'slavery'
|
|
8
|
+
gem.version = Slavery::VERSION
|
|
9
|
+
gem.authors = ['Kenn Ejima']
|
|
10
|
+
gem.email = ['kenn.ejima@gmail.com']
|
|
11
|
+
gem.description = %q{Selective slave read for ActiveRecord}
|
|
12
|
+
gem.summary = %q{Selective slave read for ActiveRecord}
|
|
13
|
+
gem.homepage = 'https://github.com/kenn/slavery'
|
|
14
|
+
|
|
15
|
+
gem.files = `git ls-files`.split($/)
|
|
16
|
+
gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
|
|
17
|
+
gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
|
|
18
|
+
gem.require_paths = ['lib']
|
|
19
|
+
|
|
20
|
+
gem.add_runtime_dependency 'activerecord', '>= 3.0.0'
|
|
21
|
+
|
|
22
|
+
gem.add_development_dependency 'mysql2'
|
|
23
|
+
gem.add_development_dependency 'rspec'
|
|
24
|
+
gem.add_development_dependency 'sqlite3'
|
|
25
|
+
end
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
require 'spec_helper'
|
|
2
|
+
|
|
3
|
+
describe Slavery do
|
|
4
|
+
def on_slave?
|
|
5
|
+
Thread.current[:on_slave]
|
|
6
|
+
end
|
|
7
|
+
|
|
8
|
+
it 'sets thread local' do
|
|
9
|
+
Slavery.on_master { on_slave?.should == false }
|
|
10
|
+
Slavery.on_slave { on_slave?.should == true }
|
|
11
|
+
end
|
|
12
|
+
|
|
13
|
+
it 'returns value from block' do
|
|
14
|
+
Slavery.on_master { User.count }.should == 2
|
|
15
|
+
Slavery.on_slave { User.count }.should == 1
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
it 'handles nested calls' do
|
|
19
|
+
# Slave -> Slave
|
|
20
|
+
Slavery.on_slave do
|
|
21
|
+
on_slave?.should == true
|
|
22
|
+
|
|
23
|
+
Slavery.on_slave do
|
|
24
|
+
on_slave?.should == true
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
on_slave?.should == true
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
# Slave -> Master
|
|
31
|
+
Slavery.on_slave do
|
|
32
|
+
on_slave?.should == true
|
|
33
|
+
|
|
34
|
+
Slavery.on_master do
|
|
35
|
+
on_slave?.should == false
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
on_slave?.should == true
|
|
39
|
+
end
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
it 'disables in transaction' do
|
|
43
|
+
User.transaction do
|
|
44
|
+
expect { User.slaveryable? }.to raise_error(Slavery::Error)
|
|
45
|
+
end
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
it 'disables by configuration' do
|
|
49
|
+
Slavery.stub(:disabled).and_return(false)
|
|
50
|
+
Slavery.on_slave { User.slaveryable?.should == true }
|
|
51
|
+
|
|
52
|
+
Slavery.stub(:disabled).and_return(true)
|
|
53
|
+
Slavery.on_slave { User.slaveryable?.should == false }
|
|
54
|
+
end
|
|
55
|
+
end
|
data/spec/spec_helper.rb
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
require 'rubygems'
|
|
2
|
+
require 'bundler/setup'
|
|
3
|
+
|
|
4
|
+
require 'slavery'
|
|
5
|
+
|
|
6
|
+
# Activate Slavery
|
|
7
|
+
ActiveRecord::Base.send(:include, Slavery)
|
|
8
|
+
|
|
9
|
+
# Prepare databases
|
|
10
|
+
class User < ActiveRecord::Base
|
|
11
|
+
end
|
|
12
|
+
|
|
13
|
+
# Should be equal to Rails.env
|
|
14
|
+
Slavery.env = 'test'
|
|
15
|
+
|
|
16
|
+
ActiveRecord::Base.configurations = {
|
|
17
|
+
'test' => { adapter: 'sqlite3', database: 'test_db' },
|
|
18
|
+
'test_slave' => { adapter: 'sqlite3', database: 'test_slave_db' }
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
# Create two records on master
|
|
22
|
+
ActiveRecord::Base.establish_connection(:test)
|
|
23
|
+
ActiveRecord::Base.connection.create_table :users, force: true
|
|
24
|
+
User.create
|
|
25
|
+
User.create
|
|
26
|
+
|
|
27
|
+
# Create one record on slave, emulating replication lag
|
|
28
|
+
ActiveRecord::Base.establish_connection(:test_slave)
|
|
29
|
+
ActiveRecord::Base.connection.create_table :users, force: true
|
|
30
|
+
User.create
|
|
31
|
+
|
|
32
|
+
# Reconnect to master
|
|
33
|
+
ActiveRecord::Base.establish_connection(:test)
|
metadata
ADDED
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: slavery
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 1.0.0
|
|
5
|
+
prerelease:
|
|
6
|
+
platform: ruby
|
|
7
|
+
authors:
|
|
8
|
+
- Kenn Ejima
|
|
9
|
+
autorequire:
|
|
10
|
+
bindir: bin
|
|
11
|
+
cert_chain: []
|
|
12
|
+
date: 2012-09-20 00:00:00.000000000 Z
|
|
13
|
+
dependencies:
|
|
14
|
+
- !ruby/object:Gem::Dependency
|
|
15
|
+
name: activerecord
|
|
16
|
+
requirement: !ruby/object:Gem::Requirement
|
|
17
|
+
none: false
|
|
18
|
+
requirements:
|
|
19
|
+
- - ! '>='
|
|
20
|
+
- !ruby/object:Gem::Version
|
|
21
|
+
version: 3.0.0
|
|
22
|
+
type: :runtime
|
|
23
|
+
prerelease: false
|
|
24
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
25
|
+
none: false
|
|
26
|
+
requirements:
|
|
27
|
+
- - ! '>='
|
|
28
|
+
- !ruby/object:Gem::Version
|
|
29
|
+
version: 3.0.0
|
|
30
|
+
- !ruby/object:Gem::Dependency
|
|
31
|
+
name: mysql2
|
|
32
|
+
requirement: !ruby/object:Gem::Requirement
|
|
33
|
+
none: false
|
|
34
|
+
requirements:
|
|
35
|
+
- - ! '>='
|
|
36
|
+
- !ruby/object:Gem::Version
|
|
37
|
+
version: '0'
|
|
38
|
+
type: :development
|
|
39
|
+
prerelease: false
|
|
40
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
41
|
+
none: false
|
|
42
|
+
requirements:
|
|
43
|
+
- - ! '>='
|
|
44
|
+
- !ruby/object:Gem::Version
|
|
45
|
+
version: '0'
|
|
46
|
+
- !ruby/object:Gem::Dependency
|
|
47
|
+
name: rspec
|
|
48
|
+
requirement: !ruby/object:Gem::Requirement
|
|
49
|
+
none: false
|
|
50
|
+
requirements:
|
|
51
|
+
- - ! '>='
|
|
52
|
+
- !ruby/object:Gem::Version
|
|
53
|
+
version: '0'
|
|
54
|
+
type: :development
|
|
55
|
+
prerelease: false
|
|
56
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
57
|
+
none: false
|
|
58
|
+
requirements:
|
|
59
|
+
- - ! '>='
|
|
60
|
+
- !ruby/object:Gem::Version
|
|
61
|
+
version: '0'
|
|
62
|
+
- !ruby/object:Gem::Dependency
|
|
63
|
+
name: sqlite3
|
|
64
|
+
requirement: !ruby/object:Gem::Requirement
|
|
65
|
+
none: false
|
|
66
|
+
requirements:
|
|
67
|
+
- - ! '>='
|
|
68
|
+
- !ruby/object:Gem::Version
|
|
69
|
+
version: '0'
|
|
70
|
+
type: :development
|
|
71
|
+
prerelease: false
|
|
72
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
73
|
+
none: false
|
|
74
|
+
requirements:
|
|
75
|
+
- - ! '>='
|
|
76
|
+
- !ruby/object:Gem::Version
|
|
77
|
+
version: '0'
|
|
78
|
+
description: Selective slave read for ActiveRecord
|
|
79
|
+
email:
|
|
80
|
+
- kenn.ejima@gmail.com
|
|
81
|
+
executables: []
|
|
82
|
+
extensions: []
|
|
83
|
+
extra_rdoc_files: []
|
|
84
|
+
files:
|
|
85
|
+
- .gitignore
|
|
86
|
+
- .rspec
|
|
87
|
+
- Gemfile
|
|
88
|
+
- LICENSE.txt
|
|
89
|
+
- README.md
|
|
90
|
+
- Rakefile
|
|
91
|
+
- lib/slavery.rb
|
|
92
|
+
- lib/slavery/railtie.rb
|
|
93
|
+
- lib/slavery/version.rb
|
|
94
|
+
- slavery.gemspec
|
|
95
|
+
- spec/slavery_spec.rb
|
|
96
|
+
- spec/spec_helper.rb
|
|
97
|
+
homepage: https://github.com/kenn/slavery
|
|
98
|
+
licenses: []
|
|
99
|
+
post_install_message:
|
|
100
|
+
rdoc_options: []
|
|
101
|
+
require_paths:
|
|
102
|
+
- lib
|
|
103
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
104
|
+
none: false
|
|
105
|
+
requirements:
|
|
106
|
+
- - ! '>='
|
|
107
|
+
- !ruby/object:Gem::Version
|
|
108
|
+
version: '0'
|
|
109
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
110
|
+
none: false
|
|
111
|
+
requirements:
|
|
112
|
+
- - ! '>='
|
|
113
|
+
- !ruby/object:Gem::Version
|
|
114
|
+
version: '0'
|
|
115
|
+
requirements: []
|
|
116
|
+
rubyforge_project:
|
|
117
|
+
rubygems_version: 1.8.19
|
|
118
|
+
signing_key:
|
|
119
|
+
specification_version: 3
|
|
120
|
+
summary: Selective slave read for ActiveRecord
|
|
121
|
+
test_files:
|
|
122
|
+
- spec/slavery_spec.rb
|
|
123
|
+
- spec/spec_helper.rb
|
|
124
|
+
has_rdoc:
|