shard_handler 0.1.3 → 0.1.4
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.travis.yml +9 -1
- data/Gemfile +2 -0
- data/README.md +107 -22
- data/Rakefile +3 -7
- data/lib/shard_handler/handler.rb +0 -1
- data/lib/shard_handler/model.rb +13 -2
- data/lib/shard_handler/thread_registry.rb +1 -1
- data/lib/shard_handler/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: b73438164d74c6049b163a1f0b61dc62585a3fcb
|
4
|
+
data.tar.gz: 5dfdcf6cae3e106d57aed21e7d07a6150daed604
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 3e95086eb004cabb1bf3a8872e966b4b32102e479cfd82e62fa992674cc4a6742d8651626fd3a7898990197f8039ac2c0f4a3c6c5e40a0bb427c5b5f07ffa13f
|
7
|
+
data.tar.gz: 5f1993cd1f41e851dd3cb7f51fca8ece2398d8c5c5dcfc3eeec56e532b687921dd0d3b4bf138952c65c8c70ff4c78a3f282baa95602028113c5149cd6f694dad
|
data/.travis.yml
CHANGED
@@ -1,4 +1,12 @@
|
|
1
1
|
language: ruby
|
2
2
|
rvm:
|
3
|
-
- 2.1.
|
3
|
+
- 2.1.6
|
4
|
+
- 2.2.2
|
5
|
+
addons:
|
6
|
+
postgresql: "9.3"
|
4
7
|
before_install: gem install bundler -v 1.10.5
|
8
|
+
before_script:
|
9
|
+
- cp spec/database.yml.travis spec/database.yml
|
10
|
+
- cp spec/shards.yml.travis spec/shards.yml
|
11
|
+
script:
|
12
|
+
- bundle exec rake spec
|
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -1,8 +1,17 @@
|
|
1
1
|
# ShardHandler
|
2
2
|
|
3
|
+
[![Build Status][travis-badge]][travis-build]
|
4
|
+
[![Code Climate][cc-badge]][cc-details]
|
5
|
+
[![Test Coverage][cc-cov-badge]][cc-cov-details]
|
6
|
+
|
3
7
|
This gem is a simple sharding solution for Rails applications. It was created
|
4
|
-
to be used in multitenant applications,
|
5
|
-
databases but accessed through the same ActiveRecord model.
|
8
|
+
to be used in multitenant applications, where data is partitioned across
|
9
|
+
multiple databases but accessed through the same ActiveRecord model. Basically,
|
10
|
+
this gem is a nice connection switcher for ActiveRecord.
|
11
|
+
|
12
|
+
Keep in mind that this gem is tested only with PostgreSQL databases.
|
13
|
+
|
14
|
+
The documentation is [available on RubyDoc.info][docs].
|
6
15
|
|
7
16
|
## Installation
|
8
17
|
|
@@ -22,44 +31,120 @@ Or install it yourself as:
|
|
22
31
|
|
23
32
|
## Usage
|
24
33
|
|
25
|
-
|
26
|
-
example:
|
34
|
+
First, create an abstract model that will handle the shard connection:
|
27
35
|
|
28
36
|
```ruby
|
29
|
-
class
|
37
|
+
class Shard < ShardHandler::Model
|
38
|
+
self.abstract_class = true
|
30
39
|
end
|
31
40
|
```
|
32
41
|
|
33
|
-
|
42
|
+
Create a YAML file like this one:
|
43
|
+
|
44
|
+
```yaml
|
45
|
+
# config/shards.yml
|
46
|
+
development:
|
47
|
+
shard1:
|
48
|
+
adapter: postgresql
|
49
|
+
database: shard_db_1
|
50
|
+
username: postgres
|
51
|
+
shard2:
|
52
|
+
adapter: postgresql
|
53
|
+
database: shard_db_2
|
54
|
+
username: postgres
|
55
|
+
test:
|
56
|
+
# ...
|
57
|
+
production:
|
58
|
+
# ...
|
59
|
+
```
|
60
|
+
|
61
|
+
And configure your abstract model:
|
34
62
|
|
35
63
|
```ruby
|
36
|
-
|
37
|
-
|
64
|
+
# config/initializers/shard_handler.rb
|
65
|
+
Shard.setup(Rails.application.config_for(:shards))
|
66
|
+
```
|
67
|
+
|
68
|
+
Any model that has data shared across shards must inherit from your abstract
|
69
|
+
model:
|
38
70
|
|
39
|
-
|
71
|
+
```ruby
|
72
|
+
class Message < Shard
|
73
|
+
end
|
40
74
|
|
41
|
-
|
42
|
-
|
75
|
+
class Contact < Shard
|
76
|
+
end
|
77
|
+
```
|
78
|
+
|
79
|
+
To execute a query in a shard, you can use the `.using` method passing the
|
80
|
+
appropriate shard name:
|
81
|
+
|
82
|
+
```ruby
|
83
|
+
user = User.first
|
84
|
+
|
85
|
+
Shard.using(:shard1) do
|
86
|
+
puts user.messages.pluck(:title)
|
87
|
+
puts user.contacts.pluck(:email)
|
43
88
|
end
|
44
89
|
```
|
45
90
|
|
46
91
|
## Development
|
47
92
|
|
48
|
-
After checking out the repo
|
49
|
-
`rake test` to run the tests. You can also run `bin/console` for an interactive
|
50
|
-
prompt that will allow you to experiment.
|
93
|
+
After checking out the repo:
|
51
94
|
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
|
95
|
+
1. Install dependencies using `bin/setup`.
|
96
|
+
2. Create these 2 files: `spec/database.yml` and `spec/shards.yml` and configure
|
97
|
+
them with your PostgreSQL database. There are examples at
|
98
|
+
`spec/database.yml.example` and `spec/shards.yml.example`. **Important:**
|
99
|
+
these databases *will be destroyed and recreated* on test execution.
|
100
|
+
3. Run specs using `bundle exec rake spec` to make sure that everything is fine.
|
57
101
|
|
58
|
-
|
102
|
+
You can use `bin/console` to get an interactive prompt that will allow you to
|
103
|
+
experiment.
|
104
|
+
|
105
|
+
## Releasing a new version
|
106
|
+
|
107
|
+
If you are the maintainer of this project:
|
108
|
+
|
109
|
+
1. Update the version number in `lib/shard_handler/version.rb`.
|
110
|
+
2. Make sure that all tests are green (run `bundle exec rake spec`).
|
111
|
+
3. Execute `bundle exec rake release` to create a git tag for the version, push
|
112
|
+
git commits and tags, and publish the gem on [RubyGems.org][rubygems].
|
113
|
+
|
114
|
+
## Debugging PostgreSQL
|
115
|
+
|
116
|
+
To see what is going on in a PostgreSQL database, you can enable a more verbose
|
117
|
+
log for the database:
|
59
118
|
|
60
|
-
|
119
|
+
```txt
|
120
|
+
# %d = database name
|
121
|
+
# %l = session line number
|
122
|
+
# %x = transaction ID (0 if none)
|
123
|
+
log_line_prefix = '%d %l %x - '
|
124
|
+
log_statement = 'all'
|
125
|
+
```
|
126
|
+
|
127
|
+
After restarting your database, the log will be like this:
|
128
|
+
|
129
|
+
```txt
|
130
|
+
my_db 1 0 - LOG: statement: SELECT foo FROM bar
|
131
|
+
```
|
132
|
+
|
133
|
+
## Contributing
|
61
134
|
|
135
|
+
Bug reports and pull requests are welcome on GitHub at
|
136
|
+
https://github.com/locaweb/shard_handler.
|
62
137
|
|
63
138
|
## License
|
64
139
|
|
65
|
-
The gem is available as open source under the terms of the
|
140
|
+
The gem is available as open source under the terms of the
|
141
|
+
[MIT License](http://opensource.org/licenses/MIT).
|
142
|
+
|
143
|
+
[travis-badge]: https://travis-ci.org/locaweb/shard_handler.svg?branch=master
|
144
|
+
[travis-build]: https://travis-ci.org/locaweb/shard_handler
|
145
|
+
[cc-badge]: https://codeclimate.com/github/locaweb/shard_handler/badges/gpa.svg
|
146
|
+
[cc-details]: https://codeclimate.com/github/locaweb/shard_handler
|
147
|
+
[cc-cov-badge]: https://codeclimate.com/github/locaweb/shard_handler/badges/coverage.svg
|
148
|
+
[cc-cov-details]: https://codeclimate.com/github/locaweb/shard_handler/coverage
|
149
|
+
[docs]: http://www.rubydoc.info/gems/shard_handler
|
150
|
+
[rubygems]: https://rubygems.org/gems/shard_handler
|
data/Rakefile
CHANGED
@@ -1,10 +1,6 @@
|
|
1
1
|
require 'bundler/gem_tasks'
|
2
|
-
require '
|
2
|
+
require 'rspec/core/rake_task'
|
3
3
|
|
4
|
-
|
5
|
-
t.libs << 'test'
|
6
|
-
t.libs << 'lib'
|
7
|
-
t.test_files = FileList['test/**/*_test.rb']
|
8
|
-
end
|
4
|
+
RSpec::Core::RakeTask.new(:spec)
|
9
5
|
|
10
|
-
task default: :
|
6
|
+
task default: :spec
|
@@ -35,7 +35,6 @@ module ShardHandler
|
|
35
35
|
# @param name [Symbol, String] shard name
|
36
36
|
# @return [ActiveRecord::ConnectionAdapters::ConnectionHandler]
|
37
37
|
def connection_handler_for(name)
|
38
|
-
return if name.nil?
|
39
38
|
@cache.fetch(name.to_sym) do
|
40
39
|
fail UnknownShardError, "#{name.inspect} is not a valid shard name"
|
41
40
|
end
|
data/lib/shard_handler/model.rb
CHANGED
@@ -61,8 +61,13 @@ module ShardHandler
|
|
61
61
|
# @api private
|
62
62
|
# @return (see Handler#connection_handler_for)
|
63
63
|
def connection_handler
|
64
|
-
|
65
|
-
|
64
|
+
return super if use_master_connection?
|
65
|
+
|
66
|
+
unless defined?(@@handler)
|
67
|
+
fail(SetupError, 'the model was not setup')
|
68
|
+
end
|
69
|
+
|
70
|
+
@@handler.connection_handler_for(current_shard)
|
66
71
|
end
|
67
72
|
|
68
73
|
# This method will switch to the passed shard making all queries be
|
@@ -83,6 +88,12 @@ module ShardHandler
|
|
83
88
|
end
|
84
89
|
|
85
90
|
private :establish_connection
|
91
|
+
|
92
|
+
protected
|
93
|
+
|
94
|
+
def use_master_connection?
|
95
|
+
current_shard.nil?
|
96
|
+
end
|
86
97
|
end
|
87
98
|
end
|
88
99
|
end
|
@@ -3,7 +3,7 @@ require 'active_support/per_thread_registry'
|
|
3
3
|
module ShardHandler
|
4
4
|
# This class is used as a registry for the shard name that is being used in
|
5
5
|
# the current thread. Because ActiveRecord's connections are global in the
|
6
|
-
# thread scope, we need to make sure that the shard name is
|
6
|
+
# thread scope, we need to make sure that the shard name is changed only for
|
7
7
|
# the current thread and not on process or other threads.
|
8
8
|
#
|
9
9
|
# @see ActiveSupport::PerThreadRegistry
|