shard_handler 0.1.3 → 0.1.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.travis.yml +9 -1
- data/Gemfile +2 -0
- data/README.md +107 -22
- data/Rakefile +3 -7
- data/lib/shard_handler/handler.rb +0 -1
- data/lib/shard_handler/model.rb +13 -2
- data/lib/shard_handler/thread_registry.rb +1 -1
- data/lib/shard_handler/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: b73438164d74c6049b163a1f0b61dc62585a3fcb
|
4
|
+
data.tar.gz: 5dfdcf6cae3e106d57aed21e7d07a6150daed604
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 3e95086eb004cabb1bf3a8872e966b4b32102e479cfd82e62fa992674cc4a6742d8651626fd3a7898990197f8039ac2c0f4a3c6c5e40a0bb427c5b5f07ffa13f
|
7
|
+
data.tar.gz: 5f1993cd1f41e851dd3cb7f51fca8ece2398d8c5c5dcfc3eeec56e532b687921dd0d3b4bf138952c65c8c70ff4c78a3f282baa95602028113c5149cd6f694dad
|
data/.travis.yml
CHANGED
@@ -1,4 +1,12 @@
|
|
1
1
|
language: ruby
|
2
2
|
rvm:
|
3
|
-
- 2.1.
|
3
|
+
- 2.1.6
|
4
|
+
- 2.2.2
|
5
|
+
addons:
|
6
|
+
postgresql: "9.3"
|
4
7
|
before_install: gem install bundler -v 1.10.5
|
8
|
+
before_script:
|
9
|
+
- cp spec/database.yml.travis spec/database.yml
|
10
|
+
- cp spec/shards.yml.travis spec/shards.yml
|
11
|
+
script:
|
12
|
+
- bundle exec rake spec
|
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -1,8 +1,17 @@
|
|
1
1
|
# ShardHandler
|
2
2
|
|
3
|
+
[![Build Status][travis-badge]][travis-build]
|
4
|
+
[![Code Climate][cc-badge]][cc-details]
|
5
|
+
[![Test Coverage][cc-cov-badge]][cc-cov-details]
|
6
|
+
|
3
7
|
This gem is a simple sharding solution for Rails applications. It was created
|
4
|
-
to be used in multitenant applications,
|
5
|
-
databases but accessed through the same ActiveRecord model.
|
8
|
+
to be used in multitenant applications, where data is partitioned across
|
9
|
+
multiple databases but accessed through the same ActiveRecord model. Basically,
|
10
|
+
this gem is a nice connection switcher for ActiveRecord.
|
11
|
+
|
12
|
+
Keep in mind that this gem is tested only with PostgreSQL databases.
|
13
|
+
|
14
|
+
The documentation is [available on RubyDoc.info][docs].
|
6
15
|
|
7
16
|
## Installation
|
8
17
|
|
@@ -22,44 +31,120 @@ Or install it yourself as:
|
|
22
31
|
|
23
32
|
## Usage
|
24
33
|
|
25
|
-
|
26
|
-
example:
|
34
|
+
First, create an abstract model that will handle the shard connection:
|
27
35
|
|
28
36
|
```ruby
|
29
|
-
class
|
37
|
+
class Shard < ShardHandler::Model
|
38
|
+
self.abstract_class = true
|
30
39
|
end
|
31
40
|
```
|
32
41
|
|
33
|
-
|
42
|
+
Create a YAML file like this one:
|
43
|
+
|
44
|
+
```yaml
|
45
|
+
# config/shards.yml
|
46
|
+
development:
|
47
|
+
shard1:
|
48
|
+
adapter: postgresql
|
49
|
+
database: shard_db_1
|
50
|
+
username: postgres
|
51
|
+
shard2:
|
52
|
+
adapter: postgresql
|
53
|
+
database: shard_db_2
|
54
|
+
username: postgres
|
55
|
+
test:
|
56
|
+
# ...
|
57
|
+
production:
|
58
|
+
# ...
|
59
|
+
```
|
60
|
+
|
61
|
+
And configure your abstract model:
|
34
62
|
|
35
63
|
```ruby
|
36
|
-
|
37
|
-
|
64
|
+
# config/initializers/shard_handler.rb
|
65
|
+
Shard.setup(Rails.application.config_for(:shards))
|
66
|
+
```
|
67
|
+
|
68
|
+
Any model that has data shared across shards must inherit from your abstract
|
69
|
+
model:
|
38
70
|
|
39
|
-
|
71
|
+
```ruby
|
72
|
+
class Message < Shard
|
73
|
+
end
|
40
74
|
|
41
|
-
|
42
|
-
|
75
|
+
class Contact < Shard
|
76
|
+
end
|
77
|
+
```
|
78
|
+
|
79
|
+
To execute a query in a shard, you can use the `.using` method passing the
|
80
|
+
appropriate shard name:
|
81
|
+
|
82
|
+
```ruby
|
83
|
+
user = User.first
|
84
|
+
|
85
|
+
Shard.using(:shard1) do
|
86
|
+
puts user.messages.pluck(:title)
|
87
|
+
puts user.contacts.pluck(:email)
|
43
88
|
end
|
44
89
|
```
|
45
90
|
|
46
91
|
## Development
|
47
92
|
|
48
|
-
After checking out the repo
|
49
|
-
`rake test` to run the tests. You can also run `bin/console` for an interactive
|
50
|
-
prompt that will allow you to experiment.
|
93
|
+
After checking out the repo:
|
51
94
|
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
|
95
|
+
1. Install dependencies using `bin/setup`.
|
96
|
+
2. Create these 2 files: `spec/database.yml` and `spec/shards.yml` and configure
|
97
|
+
them with your PostgreSQL database. There are examples at
|
98
|
+
`spec/database.yml.example` and `spec/shards.yml.example`. **Important:**
|
99
|
+
these databases *will be destroyed and recreated* on test execution.
|
100
|
+
3. Run specs using `bundle exec rake spec` to make sure that everything is fine.
|
57
101
|
|
58
|
-
|
102
|
+
You can use `bin/console` to get an interactive prompt that will allow you to
|
103
|
+
experiment.
|
104
|
+
|
105
|
+
## Releasing a new version
|
106
|
+
|
107
|
+
If you are the maintainer of this project:
|
108
|
+
|
109
|
+
1. Update the version number in `lib/shard_handler/version.rb`.
|
110
|
+
2. Make sure that all tests are green (run `bundle exec rake spec`).
|
111
|
+
3. Execute `bundle exec rake release` to create a git tag for the version, push
|
112
|
+
git commits and tags, and publish the gem on [RubyGems.org][rubygems].
|
113
|
+
|
114
|
+
## Debugging PostgreSQL
|
115
|
+
|
116
|
+
To see what is going on in a PostgreSQL database, you can enable a more verbose
|
117
|
+
log for the database:
|
59
118
|
|
60
|
-
|
119
|
+
```txt
|
120
|
+
# %d = database name
|
121
|
+
# %l = session line number
|
122
|
+
# %x = transaction ID (0 if none)
|
123
|
+
log_line_prefix = '%d %l %x - '
|
124
|
+
log_statement = 'all'
|
125
|
+
```
|
126
|
+
|
127
|
+
After restarting your database, the log will be like this:
|
128
|
+
|
129
|
+
```txt
|
130
|
+
my_db 1 0 - LOG: statement: SELECT foo FROM bar
|
131
|
+
```
|
132
|
+
|
133
|
+
## Contributing
|
61
134
|
|
135
|
+
Bug reports and pull requests are welcome on GitHub at
|
136
|
+
https://github.com/locaweb/shard_handler.
|
62
137
|
|
63
138
|
## License
|
64
139
|
|
65
|
-
The gem is available as open source under the terms of the
|
140
|
+
The gem is available as open source under the terms of the
|
141
|
+
[MIT License](http://opensource.org/licenses/MIT).
|
142
|
+
|
143
|
+
[travis-badge]: https://travis-ci.org/locaweb/shard_handler.svg?branch=master
|
144
|
+
[travis-build]: https://travis-ci.org/locaweb/shard_handler
|
145
|
+
[cc-badge]: https://codeclimate.com/github/locaweb/shard_handler/badges/gpa.svg
|
146
|
+
[cc-details]: https://codeclimate.com/github/locaweb/shard_handler
|
147
|
+
[cc-cov-badge]: https://codeclimate.com/github/locaweb/shard_handler/badges/coverage.svg
|
148
|
+
[cc-cov-details]: https://codeclimate.com/github/locaweb/shard_handler/coverage
|
149
|
+
[docs]: http://www.rubydoc.info/gems/shard_handler
|
150
|
+
[rubygems]: https://rubygems.org/gems/shard_handler
|
data/Rakefile
CHANGED
@@ -1,10 +1,6 @@
|
|
1
1
|
require 'bundler/gem_tasks'
|
2
|
-
require '
|
2
|
+
require 'rspec/core/rake_task'
|
3
3
|
|
4
|
-
|
5
|
-
t.libs << 'test'
|
6
|
-
t.libs << 'lib'
|
7
|
-
t.test_files = FileList['test/**/*_test.rb']
|
8
|
-
end
|
4
|
+
RSpec::Core::RakeTask.new(:spec)
|
9
5
|
|
10
|
-
task default: :
|
6
|
+
task default: :spec
|
@@ -35,7 +35,6 @@ module ShardHandler
|
|
35
35
|
# @param name [Symbol, String] shard name
|
36
36
|
# @return [ActiveRecord::ConnectionAdapters::ConnectionHandler]
|
37
37
|
def connection_handler_for(name)
|
38
|
-
return if name.nil?
|
39
38
|
@cache.fetch(name.to_sym) do
|
40
39
|
fail UnknownShardError, "#{name.inspect} is not a valid shard name"
|
41
40
|
end
|
data/lib/shard_handler/model.rb
CHANGED
@@ -61,8 +61,13 @@ module ShardHandler
|
|
61
61
|
# @api private
|
62
62
|
# @return (see Handler#connection_handler_for)
|
63
63
|
def connection_handler
|
64
|
-
|
65
|
-
|
64
|
+
return super if use_master_connection?
|
65
|
+
|
66
|
+
unless defined?(@@handler)
|
67
|
+
fail(SetupError, 'the model was not setup')
|
68
|
+
end
|
69
|
+
|
70
|
+
@@handler.connection_handler_for(current_shard)
|
66
71
|
end
|
67
72
|
|
68
73
|
# This method will switch to the passed shard making all queries be
|
@@ -83,6 +88,12 @@ module ShardHandler
|
|
83
88
|
end
|
84
89
|
|
85
90
|
private :establish_connection
|
91
|
+
|
92
|
+
protected
|
93
|
+
|
94
|
+
def use_master_connection?
|
95
|
+
current_shard.nil?
|
96
|
+
end
|
86
97
|
end
|
87
98
|
end
|
88
99
|
end
|
@@ -3,7 +3,7 @@ require 'active_support/per_thread_registry'
|
|
3
3
|
module ShardHandler
|
4
4
|
# This class is used as a registry for the shard name that is being used in
|
5
5
|
# the current thread. Because ActiveRecord's connections are global in the
|
6
|
-
# thread scope, we need to make sure that the shard name is
|
6
|
+
# thread scope, we need to make sure that the shard name is changed only for
|
7
7
|
# the current thread and not on process or other threads.
|
8
8
|
#
|
9
9
|
# @see ActiveSupport::PerThreadRegistry
|