ar-octopus 0.8.5 → 0.8.6

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: a2e171072013526c5597b2d233d983f129a84cec
4
- data.tar.gz: 0c05c12e05f5dfb64aaf6f15e0accb59c2174a53
3
+ metadata.gz: 28a5543a3a9dc4dafa60e8d52c6b35d8064180eb
4
+ data.tar.gz: 1212d62583a96614b636058f7919c75fdaa0df14
5
5
  SHA512:
6
- metadata.gz: 19f50eb815a6273a87430354c5124096d90d561a8b27c1bfe0073a5a81f3a2242c71f099b88db0bab031018c56a8662ed587122412a0042ba543bc41c6b0d745
7
- data.tar.gz: 11828df91744a59445fe6a4add2c6081971f724ed9394b96eb12b36c6a09e69db2546d7dc1eadc56c19074ed5de1d7cd28f39a38c0bd7d3ad88030238a46a98e
6
+ metadata.gz: 3b8085b83e646f726fc68553aaec14cb0663d3bcca5e2412b8059712e815737c6e6b23a284a9944e37fccfa19a916b157c0efefc7cb0c039af37a65d09f538f6
7
+ data.tar.gz: e3bde2a286faef9f0c11531f73fe7cfaf5351a8a9cc354fc9fbf60921901eb300c82ba0ca27a11cae7d92b4f3fc06432573ebabfa42dedbca9f39391ca229aaa
@@ -5,9 +5,10 @@ before_script:
5
5
  - "bundle exec rake db:prepare"
6
6
  rvm:
7
7
  - 2.0.0
8
- - 2.1.5
9
- - 2.2.0
8
+ - 2.1.6
9
+ - 2.2.2
10
10
  gemfile:
11
+ - gemfiles/rails32.gemfile
11
12
  - gemfiles/rails4.gemfile
12
13
  - gemfiles/rails41.gemfile
13
14
  - gemfiles/rails42.gemfile
@@ -15,7 +16,3 @@ notifications:
15
16
  recipients:
16
17
  - gabriel.sobrinho@gmail.com
17
18
  - thiago.pradi@gmail.com
18
- matrix:
19
- include:
20
- - rvm: 1.9.3
21
- gemfile: gemfiles/rails32.gemfile
@@ -1,6 +1,6 @@
1
1
  # Octopus - Easy Database Sharding for ActiveRecord
2
2
 
3
- <a href='http://www.pledgie.com/campaigns/20950'><img alt='Click here to lend your support to: Octopus and make a donation at www.pledgie.com !' src='http://www.pledgie.com/campaigns/20950.png?skin_name=chrome' border='0' /></a> [![Build Status](https://travis-ci.org/tchandy/octopus.png)](https://travis-ci.org/tchandy/octopus) [![Code Climate](https://codeclimate.com/badge.png)](https://codeclimate.com/github/tchandy/octopus)
3
+ <a href='http://www.pledgie.com/campaigns/20950'><img alt='Click here to lend your support to: Octopus and make a donation at www.pledgie.com !' src='http://www.pledgie.com/campaigns/20950.png?skin_name=chrome' border='0' /></a> [![Build Status](https://travis-ci.org/thiagopradi/octopus.png)](https://travis-ci.org/thiagopradi/octopus) [![Code Climate](https://codeclimate.com/badge.png)](https://codeclimate.com/github/thiagopradi/octopus)
4
4
 
5
5
  Octopus is a better way to do Database Sharding in ActiveRecord. Sharding allows multiple databases in the same rails application. While there are several projects that implement Sharding (e.g. DbCharmer, DataFabric, MultiDb), each project has its own limitations. The main goal of octopus project is to provide a better way of doing Database Sharding.
6
6
 
@@ -15,15 +15,15 @@ Octopus supports:
15
15
  - Tools to manage database configurations. (soon)
16
16
 
17
17
  ### Replication
18
- When using replication, all writes queries will be sent to master, and read queries to slaves. More info could be found at: <a href="http://wiki.github.com/tchandy/octopus/replication"> Wiki</a>
18
+ When using replication, all writes queries will be sent to master, and read queries to slaves. More info could be found at: <a href="http://wiki.github.com/thiagopradi/octopus/replication"> Wiki</a>
19
19
 
20
20
  ### Sharding
21
- When using sharding, you need to specify which shard to send the query. Octopus supports selecting the shard inside a controller, or manually in each object. More could be found at <a href="http://wiki.github.com/tchandy/octopus/sharding"> Wiki</a>
21
+ When using sharding, you need to specify which shard to send the query. Octopus supports selecting the shard inside a controller, or manually in each object. More could be found at <a href="http://wiki.github.com/thiagopradi/octopus/sharding"> Wiki</a>
22
22
 
23
23
  ### Replication + Sharding
24
- When using replication and sharding concurrently, you must specify a shard, and can optionally specify a <a href="https://github.com/tchandy/octopus/wiki/Slave-Groups">slave group</a>.
24
+ When using replication and sharding concurrently, you must specify a shard, and can optionally specify a <a href="https://github.com/thiagopradi/octopus/wiki/Slave-Groups">slave group</a>.
25
25
  All write queries will be sent to each shard's master. If the slave group is specified read queries will be sent to slaves in it, or else to shard's master.
26
- More info could be found at <a href="https://github.com/tchandy/octopus/wiki/Slave-Groups"> Wiki</a>
26
+ More info could be found at <a href="https://github.com/thiagopradi/octopus/wiki/Slave-Groups"> Wiki</a>
27
27
 
28
28
  ## Install
29
29
 
@@ -55,7 +55,7 @@ going forward.
55
55
 
56
56
  ## How to use Octopus?
57
57
 
58
- First, you need to create a config file, shards.yml, inside your config/ directory. to see the syntax and how this file should look, please checkout <a href="http://wiki.github.com/tchandy/octopus/config-file">this page on wiki</a>.
58
+ First, you need to create a config file, shards.yml, inside your config/ directory. to see the syntax and how this file should look, please checkout <a href="http://wiki.github.com/thiagopradi/octopus/config-file">this page on wiki</a>.
59
59
 
60
60
  ### Syntax
61
61
 
@@ -73,6 +73,20 @@ Octopus.using(:slave_two) do
73
73
  end
74
74
  ```
75
75
 
76
+ If you want to use the same code for all shards or all shards in a specific group (for example in `db/seeds.rb`), you can use this syntax.
77
+
78
+ ```ruby
79
+ # This will return a list of the given block's results, per shard.
80
+ Octopus.using_all do
81
+ User.create_from_csv!
82
+ end
83
+
84
+ # This will return a list of the given block's results, per shard in history_shards group.
85
+ Octopus.using_group(:history_shards) do
86
+ HistoryCategory.create_from_csv!
87
+ end
88
+ ```
89
+
76
90
  Each model instance knows which shard it came from so this will work automatically:
77
91
 
78
92
  ```ruby
@@ -131,6 +145,19 @@ octopus:
131
145
 
132
146
  There is no need for a corresponding `default_migration_shard` - simply define that database to be your master. You might want this setting if all of your databases have identical schemas, but are not replicated.
133
147
 
148
+ You can configure a master shard for the rails application, to connect to when rails is going up. The safest would be to configure this to the shard specified in `database.yml` (some things still use it).
149
+
150
+ ```yaml
151
+ octopus:
152
+ master_shard: <%= ENV['SHARD'] || 'shard1' %>
153
+ ```
154
+
155
+ Then you can use the `SHARD` environment variable to override the `master_shard` specified in `config/shards.yml`, useful for running rake tasks.
156
+
157
+ ```bash
158
+ SHARD=shard1 rake db:setup && SHARD=shard2 rake db:setup
159
+ ```
160
+
134
161
  ### Rails Controllers
135
162
 
136
163
  If you want to send a specified action, or all actions from a controller, to a specific shard, use this syntax:
@@ -145,8 +172,8 @@ class ApplicationController < ActionController::Base
145
172
  end
146
173
  ```
147
174
 
148
- To see the complete list of features and syntax, please check out our <a href="http://wiki.github.com/tchandy/octopus/"> Wiki</a>
149
- Want to see sample rails applications using octopus features? please check it out: <a href="http://github.com/tchandy/octopus_sharding_example">Sharding Example</a> and <a href="http://github.com/tchandy/octopus_replication_example">Replication Example</a>. Also, we have an example that shows how to use Octopus without Rails: <a href="http://github.com/tchandy/octopus_sinatra"> Octopus + Sinatra Example</a>.
175
+ To see the complete list of features and syntax, please check out our <a href="http://wiki.github.com/thiagopradi/octopus/"> Wiki</a>
176
+ Want to see sample rails applications using octopus features? please check it out: <a href="http://github.com/thiagopradi/octopus_sharding_example">Sharding Example</a> and <a href="http://github.com/thiagopradi/octopus_replication_example">Replication Example</a>. Also, we have an example that shows how to use Octopus without Rails: <a href="http://github.com/thiagopradi/octopus_sinatra"> Octopus + Sinatra Example</a>.
150
177
 
151
178
  ## Mixing Octopus with the Rails multiple database model
152
179
  If you want to set a custom connection to a specific model, use the syntax `octopus_establish_connection` syntax:
@@ -174,13 +201,13 @@ CustomConnectedModel.using(:my_shard).first
174
201
  CustomConnectedModel.using(:some_other_shard).first
175
202
  ```
176
203
 
177
- This can be useful if you have a model that lives in a separate database and would like to add sharding or replication to it. For other use cases, you may be better off with <a href="https://github.com/tchandy/octopus/wiki/Slave-Groups">slave groups</a>.
204
+ This can be useful if you have a model that lives in a separate database and would like to add sharding or replication to it. For other use cases, you may be better off with <a href="https://github.com/thiagopradi/octopus/wiki/Slave-Groups">slave groups</a>.
178
205
 
179
206
  ## Contributing with Octopus
180
207
  Contributors are welcome! To run the test suite, you need mysql, postgresql and sqlite3 installed. This is what you need to setup your Octopus development environment:
181
208
 
182
209
  ```bash
183
- git clone http://github.com/tchandy/octopus.git
210
+ git clone http://github.com/thiagopradi/octopus.git
184
211
  cd octopus
185
212
  bundle install
186
213
  bundle exec rake db:prepare
@@ -200,7 +227,7 @@ cucumber
200
227
  If you are having issues running the octopus spec suite, verify your database users and passwords match those inside the config files and your permissions are correct.
201
228
 
202
229
  ## Contributors:
203
- - <a href="https://github.com/tchandy/octopus/contributors">All Contributors</a>
230
+ - <a href="https://github.com/thiagopradi/octopus/contributors">All Contributors</a>
204
231
 
205
232
  ## Mailing List:
206
233
  - <a href="http://groups.google.com/group/octopus-activerecord/">Octopus Mailing List</a>
@@ -25,7 +25,7 @@ Gem::Specification.new do |s|
25
25
  s.add_dependency 'activesupport', '>= 3.2.0'
26
26
 
27
27
  s.add_development_dependency 'appraisal', '>= 0.3.8'
28
- s.add_development_dependency 'mysql2', '> 0.3'
28
+ s.add_development_dependency 'mysql2', '~> 0.3.18'
29
29
  s.add_development_dependency 'pg', '>= 0.11.0'
30
30
  s.add_development_dependency 'rake', '>= 0.8.7'
31
31
  s.add_development_dependency 'rspec', '>= 3'
@@ -11,12 +11,12 @@ module Octopus
11
11
  end
12
12
 
13
13
  def self.rails_env
14
- @rails_env ||= self.rails? ? Rails.env.to_s : 'shards'
14
+ @rails_env ||= defined?(::Rails.env) ? Rails.env.to_s : 'shards'
15
15
  end
16
16
 
17
17
  def self.config
18
18
  @config ||= begin
19
- file_name = Octopus.directory + '/config/shards.yml'
19
+ file_name = File.join(Octopus.directory, 'config/shards.yml').to_s
20
20
 
21
21
  if File.exist?(file_name) || File.symlink?(file_name)
22
22
  config ||= HashWithIndifferentAccess.new(YAML.load(ERB.new(File.read(file_name)).result))[Octopus.env]
@@ -28,13 +28,25 @@ module Octopus
28
28
  end
29
29
  end
30
30
 
31
+ def self.load_balancer=(balancer)
32
+ @load_balancer = balancer
33
+ end
34
+
35
+ def self.load_balancer
36
+ @load_balancer ||= Octopus::LoadBalancing::RoundRobin
37
+ end
38
+
39
+ def self.master_shard
40
+ ((config && config[:master_shard]) || :master).to_sym
41
+ end
42
+
31
43
  # Public: Whether or not Octopus is configured and should hook into the
32
44
  # current environment. Checks the environments config option for the Rails
33
45
  # environment by default.
34
46
  #
35
47
  # Returns a boolean
36
48
  def self.enabled?
37
- if defined?(::Rails)
49
+ if defined?(::Rails.env)
38
50
  Octopus.environments.include?(Rails.env.to_s)
39
51
  else
40
52
  # TODO: This doens't feel right but !Octopus.config.blank? is breaking a
@@ -46,7 +58,7 @@ module Octopus
46
58
  # Returns the Rails.root_to_s when you are using rails
47
59
  # Running the current directory in a generic Ruby process
48
60
  def self.directory
49
- @directory ||= defined?(Rails) ? Rails.root.to_s : Dir.pwd
61
+ @directory ||= defined?(::Rails.root) ? Rails.root.to_s : Dir.pwd
50
62
  end
51
63
 
52
64
  # This is the default way to do Octopus Setup
@@ -90,14 +102,10 @@ module Octopus
90
102
  rails4? && ActiveRecord::VERSION::MINOR >= 1
91
103
  end
92
104
 
93
- def self.rails?
94
- defined?(Rails)
95
- end
96
-
97
105
  attr_writer :logger
98
106
 
99
107
  def self.logger
100
- if defined?(Rails)
108
+ if defined?(Rails.logger)
101
109
  @logger ||= Rails.logger
102
110
  else
103
111
  @logger ||= Logger.new($stderr)
@@ -119,15 +127,37 @@ module Octopus
119
127
  end
120
128
  end
121
129
 
130
+ def self.using_group(group, &block)
131
+ conn = ActiveRecord::Base.connection
132
+
133
+ if conn.is_a?(Octopus::Proxy)
134
+ conn.send_queries_to_group(group, &block)
135
+ else
136
+ yield
137
+ end
138
+ end
139
+
140
+ def self.using_all(&block)
141
+ conn = ActiveRecord::Base.connection
142
+
143
+ if conn.is_a?(Octopus::Proxy)
144
+ conn.send_queries_to_all_shards(&block)
145
+ else
146
+ yield
147
+ end
148
+ end
149
+
122
150
  def self.fully_replicated(&_block)
123
- old_fully_replicated = Thread.current['octopus.fully_replicated']
124
- Thread.current['octopus.fully_replicated'] = true
151
+ old_fully_replicated = Thread.current[Octopus::Proxy::FULLY_REPLICATED_KEY]
152
+ Thread.current[Octopus::Proxy::FULLY_REPLICATED_KEY] = true
125
153
  yield
126
154
  ensure
127
- Thread.current['octopus.fully_replicated'] = old_fully_replicated
155
+ Thread.current[Octopus::Proxy::FULLY_REPLICATED_KEY] = old_fully_replicated
128
156
  end
129
157
  end
130
158
 
159
+ require 'octopus/exception'
160
+
131
161
  require 'octopus/shard_tracking'
132
162
  require 'octopus/shard_tracking/attribute'
133
163
  require 'octopus/shard_tracking/dynamic'
@@ -143,7 +173,7 @@ require 'octopus/log_subscriber'
143
173
  require 'octopus/abstract_adapter'
144
174
  require 'octopus/singular_association'
145
175
 
146
- require 'octopus/railtie' if defined?(::Rails)
176
+ require 'octopus/railtie' if defined?(::Rails::Railtie)
147
177
 
148
178
  require 'octopus/proxy'
149
179
  require 'octopus/collection_proxy'
@@ -20,20 +20,16 @@ module Octopus
20
20
  end
21
21
  end
22
22
 
23
- def self.included(base)
24
- base.alias_method_chain :initialize, :octopus_shard
25
- end
26
-
27
23
  def octopus_shard
28
24
  @config[:octopus_shard]
29
25
  end
30
26
 
31
- def initialize_with_octopus_shard(*args)
32
- initialize_without_octopus_shard(*args)
27
+ def initialize(*args)
28
+ super
33
29
  @instrumenter = InstrumenterDecorator.new(self, @instrumenter)
34
30
  end
35
31
  end
36
32
  end
37
33
  end
38
34
 
39
- ActiveRecord::ConnectionAdapters::AbstractAdapter.send(:include, Octopus::AbstractAdapter::OctopusShard)
35
+ ActiveRecord::ConnectionAdapters::AbstractAdapter.send(:prepend, Octopus::AbstractAdapter::OctopusShard)
@@ -2,6 +2,7 @@ module Octopus
2
2
  module Association
3
3
  def self.included(base)
4
4
  base.send(:include, Octopus::ShardTracking::Dynamic)
5
+ base.sharded_methods :target_scope
5
6
  end
6
7
 
7
8
  def current_shard
@@ -4,37 +4,6 @@ module Octopus
4
4
  base.send(:include, InstanceMethods)
5
5
  end
6
6
 
7
- module QueryOnCurrentShard
8
- METHODS = %w(
9
- all
10
- average
11
- count
12
- empty?
13
- exists?
14
- find
15
- find_by_sql
16
- first
17
- last
18
- maximum
19
- minimum
20
- pluck
21
- scoping
22
- size
23
- sum
24
- to_a
25
- )
26
-
27
- METHODS.each do |m|
28
- define_method m.to_sym do |*args, &block|
29
- if self.respond_to?(:proxy_association) && proxy_association
30
- proxy_association.owner.run_on_shard { super(*args, &block) }
31
- else
32
- super(*args, &block)
33
- end
34
- end
35
- end
36
- end
37
-
38
7
  module InstanceMethods
39
8
  def connection_on_association=(record)
40
9
  return unless ::Octopus.enabled?
@@ -56,14 +25,7 @@ module Octopus
56
25
  end
57
26
  super
58
27
  end
59
- else
60
- def has_many(association_id, options = {}, &extension)
61
- default_octopus_opts(options)
62
- super
63
- end
64
- end
65
28
 
66
- if Octopus.rails4?
67
29
  def has_and_belongs_to_many(association_id, scope = nil, options = {}, &extension)
68
30
  if options == {} && scope.is_a?(Hash)
69
31
  default_octopus_opts(scope)
@@ -72,7 +34,13 @@ module Octopus
72
34
  end
73
35
  super
74
36
  end
37
+
75
38
  else
39
+ def has_many(association_id, options = {}, &extension)
40
+ default_octopus_opts(options)
41
+ super
42
+ end
43
+
76
44
  def has_and_belongs_to_many(association_id, options = {}, &extension)
77
45
  default_octopus_opts(options)
78
46
  super
@@ -80,23 +48,8 @@ module Octopus
80
48
  end
81
49
 
82
50
  def default_octopus_opts(options)
83
- if options[:before_add].is_a?(Array)
84
- options[:before_add] << :connection_on_association=
85
- elsif options[:before_add].is_a?(Symbol)
86
- options[:before_add] = [:connection_on_association=, options[:before_add]]
87
- else
88
- options[:before_add] = :connection_on_association=
89
- end
90
-
91
- if options[:before_remove].is_a?(Array)
92
- options[:before_remove] << :connection_on_association=
93
- elsif options[:before_remove].is_a?(Symbol)
94
- options[:before_remove] = [:connection_on_association=, options[:before_remove]]
95
- else
96
- options[:before_remove] = :connection_on_association=
97
- end
98
-
99
- options[:extend] = [Octopus::AssociationShardTracking::QueryOnCurrentShard, options[:extend]].flatten.compact
51
+ options[:before_add] = [ :connection_on_association=, options[:before_add] ].compact.flatten
52
+ options[:before_remove] = [ :connection_on_association=, options[:before_remove] ].compact.flatten
100
53
  end
101
54
  end
102
55
  end
@@ -5,6 +5,9 @@ module Octopus
5
5
  base.sharded_methods :any?, :build, :count, :create, :create!, :concat, :delete, :delete_all,
6
6
  :destroy, :destroy_all, :empty?, :find, :first, :include?, :last, :length,
7
7
  :many?, :pluck, :replace, :select, :size, :sum, :to_a, :uniq
8
+ if Octopus.rails3?
9
+ base.sharded_methods :scoped
10
+ end
8
11
  end
9
12
 
10
13
  def current_shard
@@ -0,0 +1,4 @@
1
+ module Octopus
2
+ class Exception < ::Exception
3
+ end
4
+ end
@@ -11,7 +11,7 @@ module Octopus
11
11
  end
12
12
 
13
13
  # Returns the next available slave in the pool
14
- def next
14
+ def next(options)
15
15
  @slaves_list[@slave_index = (@slave_index + 1) % @slaves_list.length]
16
16
  end
17
17
  end
@@ -51,7 +51,7 @@ module Octopus
51
51
  shards.merge(Array.wrap(shard))
52
52
  end
53
53
 
54
- shards.to_a.presence || [:master]
54
+ shards.to_a.presence || [Octopus.master_shard]
55
55
  end
56
56
  end
57
57
  end
@@ -107,7 +107,7 @@ module Octopus
107
107
 
108
108
  def up_with_octopus(migrations_paths, target_version = nil, &block)
109
109
  return up_without_octopus(migrations_paths, target_version, &block) unless connection.is_a?(Octopus::Proxy)
110
- return up_without_octopus(migrations_paths, target_version, &block) unless connection.current_shard == :master
110
+ return up_without_octopus(migrations_paths, target_version, &block) unless connection.current_shard.to_s == Octopus.master_shard.to_s
111
111
 
112
112
  connection.send_queries_to_multiple_shards(connection.shard_names) do
113
113
  up_without_octopus(migrations_paths, target_version, &block)
@@ -116,7 +116,7 @@ module Octopus
116
116
 
117
117
  def down_with_octopus(migrations_paths, target_version = nil, &block)
118
118
  return down_without_octopus(migrations_paths, target_version, &block) unless connection.is_a?(Octopus::Proxy)
119
- return down_without_octopus(migrations_paths, target_version, &block) unless connection.current_shard == :master
119
+ return down_without_octopus(migrations_paths, target_version, &block) unless connection.current_shard.to_s == Octopus.master_shard.to_s
120
120
 
121
121
  connection.send_queries_to_multiple_shards(connection.shard_names) do
122
122
  down_without_octopus(migrations_paths, target_version, &block)
@@ -21,6 +21,14 @@ module Octopus
21
21
  end
22
22
 
23
23
  def using(shard)
24
+ if block_given?
25
+ raise Octopus::Exception, <<-EOF
26
+ #{name}.using is not allowed to receive a block, it works just like a regular scope.
27
+
28
+ If you are trying to scope everything to a specific shard, use Octopus.using instead.
29
+ EOF
30
+ end
31
+
24
32
  if Octopus.enabled?
25
33
  clean_table_name
26
34
  Octopus::ScopeProxy.new(shard, self)
@@ -57,7 +65,7 @@ module Octopus
57
65
  end
58
66
 
59
67
  def equality_with_octopus(comparison_object)
60
- equality_without_octopus(comparison_object) && comparison_object.current_shard == current_shard
68
+ equality_without_octopus(comparison_object) && comparison_object.current_shard.to_s == current_shard.to_s
61
69
  end
62
70
 
63
71
  def perform_validations_with_octopus(*args)
@@ -95,11 +103,12 @@ module Octopus
95
103
  end
96
104
 
97
105
  def hijack_methods
98
- around_save :run_on_shard, :unless => -> { self.class.custom_octopus_connection }
106
+ around_save :run_on_shard, :unless => lambda { self.class.custom_octopus_connection }
99
107
  after_initialize :set_current_shard
100
108
 
109
+ class_attribute :custom_octopus_connection
110
+
101
111
  class << self
102
- attr_accessor :custom_octopus_connection
103
112
  attr_accessor :custom_octopus_table_name
104
113
 
105
114
  alias_method_chain :connection, :octopus
@@ -6,6 +6,15 @@ module Octopus
6
6
  class Proxy
7
7
  attr_accessor :config, :sharded
8
8
 
9
+ CURRENT_MODEL_KEY = 'octopus.current_model'.freeze
10
+ CURRENT_SHARD_KEY = 'octopus.current_shard'.freeze
11
+ CURRENT_GROUP_KEY = 'octopus.current_group'.freeze
12
+ CURRENT_SLAVE_GROUP_KEY = 'octopus.current_slave_group'.freeze
13
+ CURRENT_LOAD_BALANCE_OPTIONS_KEY = 'octopus.current_load_balance_options'.freeze
14
+ BLOCK_KEY = 'octopus.block'.freeze
15
+ LAST_CURRENT_SHARD_KEY = 'octopus.last_current_shard'.freeze
16
+ FULLY_REPLICATED_KEY = 'octopus.fully_replicated'.freeze
17
+
9
18
  def initialize(config = Octopus.config)
10
19
  initialize_shards(config)
11
20
  initialize_replication(config) if !config.nil? && config['replicated']
@@ -73,7 +82,7 @@ module Octopus
73
82
  end
74
83
  end
75
84
 
76
- @shards[:master] ||= ActiveRecord::Base.connection_pool_without_octopus
85
+ @shards[:master] ||= ActiveRecord::Base.connection_pool_without_octopus if Octopus.master_shard == :master
77
86
  end
78
87
 
79
88
  def initialize_replication(config)
@@ -86,29 +95,30 @@ module Octopus
86
95
 
87
96
  @slaves_list = @shards.keys.map(&:to_s).sort
88
97
  @slaves_list.delete('master')
89
- @slaves_load_balancer = Octopus::LoadBalancing::RoundRobin.new(@slaves_list)
98
+ @slaves_load_balancer = Octopus.load_balancer.new(@slaves_list)
90
99
  end
91
100
 
92
101
  def current_model
93
- Thread.current['octopus.current_model']
102
+ Thread.current[CURRENT_MODEL_KEY]
94
103
  end
95
104
 
96
105
  def current_model=(model)
97
- Thread.current['octopus.current_model'] = model.is_a?(ActiveRecord::Base) ? model.class : model
106
+ Thread.current[CURRENT_MODEL_KEY] = model.is_a?(ActiveRecord::Base) ? model.class : model
98
107
  end
99
108
 
100
109
  def current_shard
101
- Thread.current['octopus.current_shard'] ||= :master
110
+ Thread.current[CURRENT_SHARD_KEY] ||= Octopus.master_shard
102
111
  end
103
112
 
104
113
  def current_shard=(shard_symbol)
105
- self.current_slave_group = nil
106
114
  if shard_symbol.is_a?(Array)
115
+ self.current_slave_group = nil
107
116
  shard_symbol.each { |symbol| fail "Nonexistent Shard Name: #{symbol}" if @shards[symbol].nil? }
108
117
  elsif shard_symbol.is_a?(Hash)
109
118
  hash = shard_symbol
110
119
  shard_symbol = hash[:shard]
111
120
  slave_group_symbol = hash[:slave_group]
121
+ load_balance_options = hash[:load_balance_options]
112
122
 
113
123
  if shard_symbol.nil? && slave_group_symbol.nil?
114
124
  fail 'Neither shard or slave group must be specified'
@@ -123,17 +133,18 @@ module Octopus
123
133
  (@shards_slave_groups.try(:[], shard_symbol).nil? && @slave_groups[slave_group_symbol].nil?)
124
134
  fail "Nonexistent Slave Group Name: #{slave_group_symbol} in shards config: #{@shards_config.inspect}"
125
135
  end
126
- self.current_slave_group = slave_group_symbol
127
136
  end
137
+ self.current_slave_group = slave_group_symbol
138
+ self.current_load_balance_options = load_balance_options
128
139
  else
129
140
  fail "Nonexistent Shard Name: #{shard_symbol}" if @shards[shard_symbol].nil?
130
141
  end
131
142
 
132
- Thread.current['octopus.current_shard'] = shard_symbol
143
+ Thread.current[CURRENT_SHARD_KEY] = shard_symbol
133
144
  end
134
145
 
135
146
  def current_group
136
- Thread.current['octopus.current_group']
147
+ Thread.current[CURRENT_GROUP_KEY]
137
148
  end
138
149
 
139
150
  def current_group=(group_symbol)
@@ -142,35 +153,44 @@ module Octopus
142
153
  fail "Nonexistent Group Name: #{group}" unless has_group?(group)
143
154
  end
144
155
 
145
- Thread.current['octopus.current_group'] = group_symbol
156
+ Thread.current[CURRENT_GROUP_KEY] = group_symbol
146
157
  end
147
158
 
148
159
  def current_slave_group
149
- Thread.current['octopus.current_slave_group']
160
+ Thread.current[CURRENT_SLAVE_GROUP_KEY]
150
161
  end
151
162
 
152
163
  def current_slave_group=(slave_group_symbol)
153
- Thread.current['octopus.current_slave_group'] = slave_group_symbol
164
+ Thread.current[CURRENT_SLAVE_GROUP_KEY] = slave_group_symbol
165
+ Thread.current[CURRENT_LOAD_BALANCE_OPTIONS_KEY] = nil if slave_group_symbol.nil?
166
+ end
167
+
168
+ def current_load_balance_options
169
+ Thread.current[CURRENT_LOAD_BALANCE_OPTIONS_KEY]
170
+ end
171
+
172
+ def current_load_balance_options=(options)
173
+ Thread.current[CURRENT_LOAD_BALANCE_OPTIONS_KEY] = options
154
174
  end
155
175
 
156
176
  def block
157
- Thread.current['octopus.block']
177
+ Thread.current[BLOCK_KEY]
158
178
  end
159
179
 
160
180
  def block=(block)
161
- Thread.current['octopus.block'] = block
181
+ Thread.current[BLOCK_KEY] = block
162
182
  end
163
183
 
164
184
  def last_current_shard
165
- Thread.current['octopus.last_current_shard']
185
+ Thread.current[LAST_CURRENT_SHARD_KEY]
166
186
  end
167
187
 
168
188
  def last_current_shard=(last_current_shard)
169
- Thread.current['octopus.last_current_shard'] = last_current_shard
189
+ Thread.current[LAST_CURRENT_SHARD_KEY] = last_current_shard
170
190
  end
171
191
 
172
192
  def fully_replicated?
173
- @fully_replicated || Thread.current['octopus.fully_replicated']
193
+ @fully_replicated || Thread.current[FULLY_REPLICATED_KEY]
174
194
  end
175
195
 
176
196
  # Public: Whether or not a group exists with the given name converted to a
@@ -202,7 +222,7 @@ module Octopus
202
222
  # reconnect, but in Rails 3.1 the flag prevents this.
203
223
  def safe_connection(connection_pool)
204
224
  connection_pool.automatic_reconnect ||= true
205
- if !connection_pool.connected? && @shards[:master].connection.query_cache_enabled
225
+ if !connection_pool.connected? && @shards[Octopus.master_shard].connection.query_cache_enabled
206
226
  connection_pool.connection.enable_query_cache!
207
227
  end
208
228
  connection_pool.connection
@@ -229,13 +249,23 @@ module Octopus
229
249
  end
230
250
 
231
251
  def send_queries_to_multiple_shards(shards, &block)
232
- shards.each do |shard|
252
+ shards.map do |shard|
233
253
  run_queries_on_shard(shard, &block)
234
254
  end
235
255
  end
236
256
 
257
+ def send_queries_to_group(group, &block)
258
+ using_group(group) do
259
+ send_queries_to_multiple_shards(shards_for_group(group), &block)
260
+ end
261
+ end
262
+
263
+ def send_queries_to_all_shards(&block)
264
+ send_queries_to_multiple_shards(shard_names.uniq { |shard_name| @shards[shard_name] }, &block)
265
+ end
266
+
237
267
  def clean_connection_proxy
238
- self.current_shard = :master
268
+ self.current_shard = Octopus.master_shard
239
269
  self.current_model = nil
240
270
  self.current_group = nil
241
271
  self.block = nil
@@ -249,7 +279,7 @@ module Octopus
249
279
 
250
280
  def transaction(options = {}, &block)
251
281
  if !sharded && current_model_replicated?
252
- run_queries_on_shard(:master) do
282
+ run_queries_on_shard(Octopus.master_shard) do
253
283
  select_connection.transaction(options, &block)
254
284
  end
255
285
  else
@@ -411,9 +441,9 @@ module Octopus
411
441
 
412
442
  def send_queries_to_selected_slave(method, *args, &block)
413
443
  if current_model.replicated || fully_replicated?
414
- selected_slave = @slaves_load_balancer.next
444
+ selected_slave = @slaves_load_balancer.next current_load_balance_options
415
445
  else
416
- selected_slave = :master
446
+ selected_slave = Octopus.master_shard
417
447
  end
418
448
 
419
449
  send_queries_to_slave(selected_slave, method, *args, &block)
@@ -439,7 +469,7 @@ module Octopus
439
469
  # Temporarily switch `current_shard` to the next slave in a slave group and send queries to it
440
470
  # while preserving `current_shard`
441
471
  def send_queries_to_balancer(balancer, method, *args, &block)
442
- send_queries_to_slave(balancer.next, method, *args, &block)
472
+ send_queries_to_slave(balancer.next(current_load_balance_options), method, *args, &block)
443
473
  end
444
474
 
445
475
  # Temporarily switch `current_shard` to the specified slave and send queries to it
@@ -468,6 +498,9 @@ module Octopus
468
498
  # Temporarily switch `current_shard` and run the block
469
499
  def using_shard(shard, &_block)
470
500
  older_shard = current_shard
501
+ older_slave_group = current_slave_group
502
+ older_load_balance_options = current_load_balance_options
503
+
471
504
 
472
505
  begin
473
506
  unless current_model && !current_model.allowed_shard?(shard)
@@ -476,6 +509,20 @@ module Octopus
476
509
  yield
477
510
  ensure
478
511
  self.current_shard = older_shard
512
+ self.current_slave_group = older_slave_group
513
+ self.current_load_balance_options = older_load_balance_options
514
+ end
515
+ end
516
+
517
+ # Temporarily switch `current_group` and run the block
518
+ def using_group(group, &_block)
519
+ older_group = current_group
520
+
521
+ begin
522
+ self.current_group = group
523
+ yield
524
+ ensure
525
+ self.current_group = older_group
479
526
  end
480
527
  end
481
528
 
@@ -1,6 +1,4 @@
1
1
  begin
2
- require 'rails/railtie'
3
-
4
2
  module Octopus
5
3
  class Railtie < Rails::Railtie
6
4
  rake_tasks do
@@ -17,7 +17,11 @@ module Octopus
17
17
  end
18
18
 
19
19
  def method_missing(method, *args, &block)
20
- run_on_shard { @ar_relation.public_send(method, *args, &block) }
20
+ if block
21
+ @ar_relation.public_send(method, *args, &block)
22
+ else
23
+ run_on_shard { @ar_relation.public_send(method, *args) }
24
+ end
21
25
  end
22
26
 
23
27
  def ==(other)
@@ -3,11 +3,11 @@ module Octopus
3
3
  def initialize(slaves)
4
4
  slaves = HashWithIndifferentAccess.new(slaves)
5
5
  slaves_list = slaves.values
6
- @load_balancer = Octopus::LoadBalancing::RoundRobin.new(slaves_list)
6
+ @load_balancer = Octopus.load_balancer.new(slaves_list)
7
7
  end
8
8
 
9
- def next
10
- @load_balancer.next
9
+ def next(options)
10
+ @load_balancer.next options
11
11
  end
12
12
  end
13
13
  end
@@ -1,3 +1,3 @@
1
1
  module Octopus
2
- VERSION = '0.8.5'
2
+ VERSION = '0.8.6'
3
3
  end
@@ -630,6 +630,12 @@ describe Octopus::AssociationShardTracking, :shards => [:brazil, :master, :canad
630
630
  expect(@brazil_client.comments.count).to eq(2)
631
631
  end
632
632
 
633
+ it 'group + count' do
634
+ expect(@brazil_client.comments.group(:id).count.length).to eq(1)
635
+ _cmt = @brazil_client.comments.create(:name => 'Builded Comment')
636
+ expect(@brazil_client.comments.group(:id).count.length).to eq(2)
637
+ end
638
+
633
639
  it 'size' do
634
640
  expect(@brazil_client.comments.size).to eq(1)
635
641
  _cmt = @brazil_client.comments.create(:name => 'Builded Comment')
@@ -2,6 +2,10 @@ require 'spec_helper'
2
2
 
3
3
  describe Octopus::Model do
4
4
  describe '#using method' do
5
+ it 'raise when Model#using receives a block' do
6
+ expect { User.using(:master) { true } }.to raise_error(Octopus::Exception, /User\.using is not allowed to receive a block/)
7
+ end
8
+
5
9
  it 'should allow to send a block to the master shard' do
6
10
  Octopus.using(:master) do
7
11
  User.create!(:name => 'Block test')
@@ -16,6 +20,16 @@ describe Octopus::Model do
16
20
  expect(User.using('canada').find_by_name('Rafael Pilha')).not_to be_nil
17
21
  end
18
22
 
23
+ it 'should allow comparison of a string shard name with symbol shard name' do
24
+ u = User.using('canada').create!(:name => 'Rafael Pilha')
25
+ expect(u).to eq(User.using(:canada).find_by_name('Rafael Pilha'))
26
+ end
27
+
28
+ it 'should allow comparison of a symbol shard name with string shard name' do
29
+ u = User.using(:canada).create!(:name => 'Rafael Pilha')
30
+ expect(u).to eq(User.using('canada').find_by_name('Rafael Pilha'))
31
+ end
32
+
19
33
  it 'should allow to pass a string as the shard name to a block' do
20
34
  Octopus.using('canada') do
21
35
  User.create!(:name => 'Rafael Pilha')
@@ -81,6 +95,21 @@ describe Octopus::Model do
81
95
  expect(ActiveRecord::Base.connection.current_shard).to eq(:master)
82
96
  end
83
97
 
98
+ it 'should ensure that the connection will be cleaned with custom master' do
99
+ OctopusHelper.using_environment :octopus do
100
+ Octopus.config[:master_shard] = :brazil
101
+ expect(ActiveRecord::Base.connection.current_shard).to eq(:brazil)
102
+ expect do
103
+ Octopus.using(:canada) do
104
+ fail 'Some Exception'
105
+ end
106
+ end.to raise_error
107
+
108
+ expect(ActiveRecord::Base.connection.current_shard).to eq(:brazil)
109
+ Octopus.config[:master_shard] = nil
110
+ end
111
+ end
112
+
84
113
  it 'should allow creating more than one user' do
85
114
  User.using(:canada).create([{ :name => 'America User 1' }, { :name => 'America User 2' }])
86
115
  User.create!(:name => 'Thiago')
@@ -99,6 +128,15 @@ describe Octopus::Model do
99
128
  expect(User.connection.current_shard).to eq(:master)
100
129
  end
101
130
 
131
+ it 'should clean #current_shard from proxy when using execute' do
132
+ OctopusHelper.using_environment :octopus do
133
+ Octopus.config[:master_shard] = :brazil
134
+ User.using(:canada).connection.execute('select * from users limit 1;')
135
+ expect(User.connection.current_shard).to eq(:brazil)
136
+ Octopus.config[:master_shard] = nil
137
+ end
138
+ end
139
+
102
140
  it 'should allow scoping dynamically' do
103
141
  User.using(:canada).using(:master).using(:canada).create!(:name => 'oi')
104
142
  expect(User.using(:canada).using(:master).count).to eq(0)
@@ -263,6 +301,12 @@ describe Octopus::Model do
263
301
  expect(CustomConnection.connection.current_database).to eq('octopus_shard_2')
264
302
  end
265
303
 
304
+ it 'reuses parent model connection' do
305
+ klass = Class.new(CustomConnection)
306
+
307
+ expect(klass.connection).to be klass.connection
308
+ end
309
+
266
310
  it 'should not mess with custom connection table names' do
267
311
  expect(Advert.connection.current_database).to eq('octopus_shard_1')
268
312
  Advert.create!(:name => 'Teste')
@@ -211,6 +211,14 @@ describe Octopus::Proxy do
211
211
  expect(proxy.shard_name).to eq(:master)
212
212
  end
213
213
 
214
+ it 'when current_shard is empty with custom master' do
215
+ OctopusHelper.using_environment :octopus do
216
+ Octopus.config[:master_shard] = :brazil
217
+ expect(proxy.shard_name).to eq(:brazil)
218
+ Octopus.config[:master_shard] = nil
219
+ end
220
+ end
221
+
214
222
  it 'when current_shard is a single shard' do
215
223
  proxy.current_shard = :canada
216
224
  expect(proxy.shard_name).to eq(:canada)
@@ -68,14 +68,23 @@ describe Octopus::RelationProxy do
68
68
  it 'uses the correct shard' do
69
69
  expect(Item.using(:brazil).count).to eq(0)
70
70
  _clients_on_brazil = Client.using(:brazil).all
71
- Client.using(:brazil) do
71
+ Octopus.using(:brazil) do
72
72
  expect(@relation.count).to eq(1)
73
73
  end
74
74
  end
75
75
 
76
+ it 'uses the correct shard in block when method_missing is triggered on CollectionProxy objects' do
77
+ Octopus.using(:brazil) do
78
+ @client.items.each do |item|
79
+ expect(item.current_shard).to eq(:canada)
80
+ expect(ActiveRecord::Base.connection.current_shard).to eq(:brazil)
81
+ end
82
+ end
83
+ end
84
+
76
85
  it 'lazily evaluates on the correct shard' do
77
86
  expect(Item.using(:brazil).count).to eq(0)
78
- Client.using(:brazil) do
87
+ Octopus.using(:brazil) do
79
88
  expect(@relation.select(:client_id).count).to eq(1)
80
89
  end
81
90
  end
@@ -61,4 +61,31 @@ describe 'when the database is replicated and has slave groups' do
61
61
  expect(Cat.count).to eq(2)
62
62
  end
63
63
  end
64
+
65
+ it 'should keep sending to slaves in a using block' do
66
+ OctopusHelper.using_environment :replicated_slave_grouped do
67
+ Cat.create!(:name => 'Thiago1')
68
+ Cat.create!(:name => 'Thiago2')
69
+
70
+ expect(Cat.count).to eq(2)
71
+ Octopus.using(:slave_group => :slaves1) do
72
+ expect(Cat.count).to eq(0)
73
+ expect(Cat.count).to eq(0)
74
+ end
75
+ end
76
+ end
77
+
78
+ it 'should restore previous slave group after a using block' do
79
+ OctopusHelper.using_environment :replicated_slave_grouped do
80
+ Cat.create!(:name => 'Thiago1')
81
+ Cat.create!(:name => 'Thiago2')
82
+
83
+ Octopus.using(:slave_group => :slaves1) do
84
+ Octopus.using(:slave_group => :slaves2) do
85
+ expect(Cat.count).to eq(2)
86
+ end
87
+ expect(Cat.count).to eq(0)
88
+ end
89
+ end
90
+ end
64
91
  end
@@ -123,4 +123,15 @@ describe 'when the database is replicated and the entire application is replicat
123
123
  expect(Cat.connection.current_shard).to eql(:master)
124
124
  end
125
125
  end
126
+
127
+ it 'should reset current shard if slave throws an exception with custom master' do
128
+ OctopusHelper.using_environment :production_fully_replicated do
129
+ Octopus.config[:master_shard] = :slave2
130
+ Cat.create!(:name => 'Slave Cat')
131
+ expect(Cat.connection.current_shard).to eql(:slave2)
132
+ Cat.where(:rubbish => true)
133
+ expect(Cat.connection.current_shard).to eql(:slave2)
134
+ Octopus.config[:master_shard] = nil
135
+ end
136
+ end
126
137
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: ar-octopus
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.8.5
4
+ version: 0.8.6
5
5
  platform: ruby
6
6
  authors:
7
7
  - Thiago Pradi
@@ -10,7 +10,7 @@ authors:
10
10
  autorequire:
11
11
  bindir: bin
12
12
  cert_chain: []
13
- date: 2015-03-12 00:00:00.000000000 Z
13
+ date: 2016-03-05 00:00:00.000000000 Z
14
14
  dependencies:
15
15
  - !ruby/object:Gem::Dependency
16
16
  name: activerecord
@@ -58,16 +58,16 @@ dependencies:
58
58
  name: mysql2
59
59
  requirement: !ruby/object:Gem::Requirement
60
60
  requirements:
61
- - - ">"
61
+ - - "~>"
62
62
  - !ruby/object:Gem::Version
63
- version: '0.3'
63
+ version: 0.3.18
64
64
  type: :development
65
65
  prerelease: false
66
66
  version_requirements: !ruby/object:Gem::Requirement
67
67
  requirements:
68
- - - ">"
68
+ - - "~>"
69
69
  - !ruby/object:Gem::Version
70
- version: '0.3'
70
+ version: 0.3.18
71
71
  - !ruby/object:Gem::Dependency
72
72
  name: pg
73
73
  requirement: !ruby/object:Gem::Requirement
@@ -187,6 +187,7 @@ files:
187
187
  - lib/octopus/association_shard_tracking.rb
188
188
  - lib/octopus/collection_association.rb
189
189
  - lib/octopus/collection_proxy.rb
190
+ - lib/octopus/exception.rb
190
191
  - lib/octopus/has_and_belongs_to_many_association.rb
191
192
  - lib/octopus/load_balancing.rb
192
193
  - lib/octopus/load_balancing/round_robin.rb
@@ -342,7 +343,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
342
343
  version: '0'
343
344
  requirements: []
344
345
  rubyforge_project:
345
- rubygems_version: 2.4.4
346
+ rubygems_version: 2.4.3
346
347
  signing_key:
347
348
  specification_version: 4
348
349
  summary: Easy Database Sharding for ActiveRecord