rails-sharding 1.0.3 → 1.1.2

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: 87d31c97612fa757fa73b7a87ab6d9372dfa1261
4
- data.tar.gz: feb0b94762924d848198740056a29c896fc2dbd5
2
+ SHA256:
3
+ metadata.gz: 57602c5018eb3d77a8a684949ffab67a73fed32f4a606e7c918f1cef8181e3b5
4
+ data.tar.gz: bae0f706bbbc0cdc301451630cb781d6f63b39ab23a54fad8ec12d3e9178de25
5
5
  SHA512:
6
- metadata.gz: 495288c321b9aa8b2f6d77cd3b6c7028d3b1096a364346cc0473d81a8f4cb1e7a281210c6a8931187ec03a5b0ad03647999be5d101694ae548740af7f9742dce
7
- data.tar.gz: 7a7424667f0a053b83c18a22222df330dbbbb0ce8eca8cc3557493705d3b9860d309535e3ecf30de6fb89c1690bca5691b0eef9dbcc292c3ea54fd667e8b0940
6
+ metadata.gz: 134c2e5e7f8967167aef8e923c114b203b39e3ad0799de21d503346ac69b358495e52dec898934b322e2baef977c5bd16a2a989e68471cb0972c90ff3a4b07a4
7
+ data.tar.gz: 44b6700d349482e1d0fbd8d716299f821edd9fc9750bdffab746a949e0c80732e3747e8864f2594bdc79aa0abe40d73a9894c2712edacd12c33c454797be9235
@@ -1 +1 @@
1
- ruby-2.4.1
1
+ ruby-2.5.7
@@ -1,7 +1,7 @@
1
1
  sudo: false
2
2
  language: ruby
3
3
  rvm:
4
- - 2.4.1
4
+ - 2.5.7
5
5
  services:
6
6
  - mysql
7
7
  - postgresql
data/README.md CHANGED
@@ -8,28 +8,34 @@
8
8
 
9
9
  Simple and robust sharding gem for Rails, including Migrations and ActiveRecord extensions
10
10
 
11
- This gems allows you to easily create extra databases to your rails application, and freely allocate ActiveRecord instances to any of the databases. It also provides rake tasks and migrations to help you manage the schema by shard groups.
11
+ This gems allows you to easily create extra databases to your rails application, and freely allocate ActiveRecord instances to any of the databases.
12
12
 
13
- After you have setup your shards, accessing them is as simple as:
13
+ Accessing shards is as simple as:
14
14
  ```ruby
15
+ # creating a user to a specific shard
15
16
  new_user = User.using_shard(:shard_group1, :shard1).create(username: 'x')
17
+
18
+ # retrieving a user from a specific shard
16
19
  loaded_user = User.using_shard(:shard_group1, :shard1).where(username: 'x').first
17
20
  ```
18
21
 
19
- You can also use the block syntax, where all your queries inside will be directed to the correct shard:
22
+ You can also use the block syntax:
20
23
  ```ruby
21
24
  Rails::Sharding.using_shard(:shard_group1, :shard1) do
25
+ # All statements inside this block will go to the selected shard
26
+
27
+ # Do some queries
22
28
  new_user = User.create(username: 'x')
23
29
  loaded_user = User.where(username: 'x').first
24
30
  billing_infos = loaded_user.billing_infos.all
25
31
  end
26
32
  ```
27
33
 
28
- You can also pick and choose which models will be shardable, so that all the models that are not shardable will still be retrieved from the master database, even if inside a using_shard block.
34
+ You can also pick and choose which models will be shardable. Non shardable models will be retrieved from the master database, even if inside a `using_shard` block.
29
35
 
30
36
  ## Compatibility
31
37
  Gem version 1.x.x:
32
- * Rails 5.0
38
+ * Rails 5.0 and 5.1
33
39
  * Databases: MySQL, MariaDB, Postgres
34
40
 
35
41
  Gem version 0.x.x:
@@ -51,7 +57,6 @@ bundle
51
57
  ```
52
58
 
53
59
  ## Creating Shards
54
- This gem helps you create shards that are additional and completely separate from your master database. The master database is the one that is created and managed through rails, and is the default storage for all your models.
55
60
 
56
61
  To start with the rails-sharding gem, run the command
57
62
  ```
@@ -80,7 +85,9 @@ development:
80
85
  ...
81
86
  ```
82
87
 
83
- Rename it to `config/shards.yml` and change it to your database configuration. This example file defines a single shard group (named `shard_group1`) containing two shards (`shard1` and `shard2`). A shard group is simply a set of shards that should have the same schema.
88
+ Rename it to `config/shards.yml` and change it to your database configuration. This example file defines a single shard group (named `shard_group1`) containing two shards (`shard1` and `shard2`).
89
+
90
+ **A shard group is a set of shards that should have the same schema.**
84
91
 
85
92
  When you're ready to create the shards run
86
93
  ```
@@ -88,7 +95,13 @@ rake shards:create
88
95
  ```
89
96
 
90
97
  ## Migrating Shards
91
- Go to the directory `db/shards_migrations/shard_group1` and add all migrations that you want to run on the shards of `shard_group1`. By design, all shards in a same group should always have the same schema. For example, add the following migration to your `db/shards_migrations/shard_group1`:
98
+ Go to the directory `db/shards_migrations/shard_group1` and add all migrations that you want to run on the shards of `shard_group1`. By design, all shards in a same group should always have the same schema.
99
+
100
+
101
+ As of now, there is no generator for migrations. You can use the regular rails generator and move the migrations to the `shards_migration` folder.
102
+
103
+
104
+ For example, add the following migration to your `db/shards_migrations/shard_group1`:
92
105
  ```ruby
93
106
  # 20160808000000_create_users.rb
94
107
  class CreateClients < ActiveRecord::Migration[5.0]
@@ -175,6 +188,9 @@ Rails::Sharding.setup do |config|
175
188
  end
176
189
  ```
177
190
 
191
+ ## Wiki
192
+ Want to know more? How to integrate with RSpec, Capistrano, etc? Take a look at our [wiki](https://github.com/hsgubert/rails-sharding/wiki).
193
+
178
194
  ## Development and Contributing
179
195
 
180
196
  After checking out the repo:
@@ -26,13 +26,30 @@ module Rails::Sharding
26
26
 
27
27
  resolver = ActiveRecord::ConnectionAdapters::ConnectionSpecification::Resolver.new(shard_group_configurations)
28
28
  begin
29
+ connection_spec = resolver.spec(shard_name.to_sym)
30
+
31
+ # since Rails 5.1 connection_spec already comes with :name set to whatever
32
+ # key you used to retrieve it from the resolver, in this case the shard_name.
33
+ # We don't want that, so we overwrite it with our connection name formed
34
+ # by shard_group:shard_name
29
35
  connection_name = connection_name(shard_group, shard_name)
30
- connection_spec = resolver.spec(shard_name.to_sym, connection_name)
36
+ connection_spec.instance_variable_set(:@name, connection_name)
31
37
  rescue ActiveRecord::AdapterNotSpecified
32
- raise Errors::ConfigNotFoundError, "Cannot find configuration for shard '#{shard_group}:#{shard_name}' in environment '#{environment}' in #{Config.shards_config_file}"
38
+ raise Errors::ConfigNotFoundError, "Cannot find configuration for shard '#{shard_group}:#{shard_name}' in environment '#{environment}' in #{Config.shards_config_file}, or it does not specify :adapter"
33
39
  end
34
40
 
35
- connection_handler.establish_connection(connection_spec)
41
+ # Since Rails 5.1 we cannot use connection_handler.establish_connection anymore,
42
+ # because it does more than establishing_connection. It does a second
43
+ # connection specification lookup on Base.configurations (where our spec
44
+ # is not, because we define it in a shards.yml instead of the regular
45
+ # database.yml) and it notifies subscribers of a event that does not concern
46
+ # us. Because of this we directly create the connection_pool and inject to
47
+ # the handler.
48
+ # Note: we should consider writting our connection handler from scratch, since
49
+ # it has become simpler than reusing the one from rails. And it will not break
50
+ # as long as the ConnectionPool interface is stable.
51
+ connection_handler.remove_connection(connection_spec.name)
52
+ connection_handler.send(:owner_to_pool)[connection_spec.name] = ActiveRecord::ConnectionAdapters::ConnectionPool.new(connection_spec)
36
53
  end
37
54
 
38
55
  def self.connection_pool(shard_group, shard_name)
@@ -61,7 +78,7 @@ module Rails::Sharding
61
78
  end
62
79
 
63
80
  def self.with_connection(shard_group, shard_name, &block)
64
- connection_pool(shard_group, shard_name).with_connection do |connection|
81
+ connection_pool(shard_group, shard_name).with_connection do |connection|
65
82
  if connection && Config.add_shard_tag_to_query_logs
66
83
  connection_name = connection_name(shard_group, shard_name)
67
84
  add_shard_tag_to_connection_log(connection, connection_name)
@@ -91,6 +108,8 @@ module Rails::Sharding
91
108
  end
92
109
 
93
110
  # Adds a shard tag to the log of all queries executed through this connection
111
+ # Obs: connection inherits from ActiveRecord::ConnectionAdapters::AbstractAdapter
112
+ # but its class depends on the database adapter used.
94
113
  def self.add_shard_tag_to_connection_log(connection, shard_tag)
95
114
  # avoids modifing connection twice
96
115
  if connection.respond_to? :shard_tag
@@ -107,9 +126,9 @@ module Rails::Sharding
107
126
 
108
127
  # defines a new #log that adds a tag to the log
109
128
  class << connection
110
- def log(sql, name="SQL", binds=[], statement_name=nil, &block)
129
+ def log(sql, name="SQL", binds=[], type_casted_binds=[], statement_name=nil, &block)
111
130
  name = (name.to_s + " (#{shard_tag})").strip
112
- self.original_log(sql, name, binds, statement_name, &block)
131
+ original_log(sql, name, binds, type_casted_binds, statement_name, &block)
113
132
  end
114
133
  end
115
134
 
@@ -1,5 +1,5 @@
1
1
  module Rails
2
2
  module Sharding
3
- VERSION = "1.0.3"
3
+ VERSION = "1.1.2"
4
4
  end
5
5
  end
@@ -274,22 +274,31 @@ shards_namespace = namespace :shards do
274
274
 
275
275
  desc "Empty the test shards (drops all tables) (options: SHARD_GROUP=x, SHARD=x)"
276
276
  task :purge => [:_make_activerecord_base_shardable] do
277
- Rails::Sharding.for_each_shard(ENV["SHARD_GROUP"], ENV["SHARD"]) do |shard_group, shard, configuration|
278
- puts "== Purging test shard #{shard_group}:#{shard}"
279
- begin
280
- # establishes connection with test shard, saving if it was connected before (rails 4.2 doesn't do this, but should)
281
- should_reconnect = Rails::Sharding::ConnectionHandler.connection_pool(shard_group, shard).active_connection?
282
- Rails::Sharding::ConnectionHandler.establish_connection(shard_group, shard, 'test')
277
+ begin
278
+ # saves the current RAILS_ENV (we must change it so the environment is set correcly on the metadata table)
279
+ initial_rails_env = Rails.env
280
+ Rails.env = 'test'
283
281
 
284
- Rails::Sharding.using_shard(shard_group, shard) do
285
- ActiveRecord::Tasks::DatabaseTasks.purge(configuration)
286
- end
287
- ensure
288
- if should_reconnect
289
- # reestablishes connection for RAILS_ENV environment (whatever that is)
290
- Rails::Sharding::ConnectionHandler.establish_connection(shard_group, shard)
282
+ Rails::Sharding.for_each_shard(ENV["SHARD_GROUP"], ENV["SHARD"]) do |shard_group, shard, configuration|
283
+ puts "== Purging test shard #{shard_group}:#{shard}"
284
+ begin
285
+ # establishes connection with test shard, saving if it was connected before (rails 4.2 doesn't do this, but should)
286
+ should_reconnect = Rails::Sharding::ConnectionHandler.connection_pool(shard_group, shard).active_connection?
287
+ Rails::Sharding::ConnectionHandler.establish_connection(shard_group, shard, 'test')
288
+
289
+ Rails::Sharding.using_shard(shard_group, shard) do
290
+ ActiveRecord::Tasks::DatabaseTasks.purge(configuration)
291
+ end
292
+ ensure
293
+ if should_reconnect
294
+ # reestablishes connection for RAILS_ENV environment (whatever that is)
295
+ Rails::Sharding::ConnectionHandler.establish_connection(shard_group, shard)
296
+ end
291
297
  end
292
298
  end
299
+ ensure
300
+ # restores rails env
301
+ Rails.env = initial_rails_env
293
302
  end
294
303
  end
295
304
  end
@@ -23,14 +23,14 @@ Gem::Specification.new do |spec|
23
23
  spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
24
24
  spec.require_paths = ["lib"]
25
25
 
26
- spec.add_runtime_dependency 'rails', '5.0.2'
26
+ spec.add_runtime_dependency 'rails', '~> 5.1.0'
27
27
 
28
- spec.add_development_dependency "bundler", "~> 1.15"
29
- spec.add_development_dependency "rake", "~> 12.0"
28
+ spec.add_development_dependency "bundler", "~> 1.17"
29
+ spec.add_development_dependency "rake", "~> 13.0"
30
30
  spec.add_development_dependency "rspec", "~> 3.0"
31
- spec.add_development_dependency "byebug", '~> 9'
31
+ spec.add_development_dependency "byebug", '~> 11'
32
32
  spec.add_development_dependency "mysql2", '~> 0'
33
- spec.add_development_dependency "pg" # postgres driver
33
+ spec.add_development_dependency "pg", '~> 0' # postgres driver
34
34
  spec.add_development_dependency "codeclimate-test-reporter", '~> 1'
35
- spec.add_development_dependency "simplecov"
35
+ spec.add_development_dependency "simplecov", '~> 0'
36
36
  end
metadata CHANGED
@@ -1,57 +1,57 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: rails-sharding
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.3
4
+ version: 1.1.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Henrique Gubert
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2017-09-13 00:00:00.000000000 Z
11
+ date: 2019-11-08 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rails
15
15
  requirement: !ruby/object:Gem::Requirement
16
16
  requirements:
17
- - - '='
17
+ - - "~>"
18
18
  - !ruby/object:Gem::Version
19
- version: 5.0.2
19
+ version: 5.1.0
20
20
  type: :runtime
21
21
  prerelease: false
22
22
  version_requirements: !ruby/object:Gem::Requirement
23
23
  requirements:
24
- - - '='
24
+ - - "~>"
25
25
  - !ruby/object:Gem::Version
26
- version: 5.0.2
26
+ version: 5.1.0
27
27
  - !ruby/object:Gem::Dependency
28
28
  name: bundler
29
29
  requirement: !ruby/object:Gem::Requirement
30
30
  requirements:
31
31
  - - "~>"
32
32
  - !ruby/object:Gem::Version
33
- version: '1.15'
33
+ version: '1.17'
34
34
  type: :development
35
35
  prerelease: false
36
36
  version_requirements: !ruby/object:Gem::Requirement
37
37
  requirements:
38
38
  - - "~>"
39
39
  - !ruby/object:Gem::Version
40
- version: '1.15'
40
+ version: '1.17'
41
41
  - !ruby/object:Gem::Dependency
42
42
  name: rake
43
43
  requirement: !ruby/object:Gem::Requirement
44
44
  requirements:
45
45
  - - "~>"
46
46
  - !ruby/object:Gem::Version
47
- version: '12.0'
47
+ version: '13.0'
48
48
  type: :development
49
49
  prerelease: false
50
50
  version_requirements: !ruby/object:Gem::Requirement
51
51
  requirements:
52
52
  - - "~>"
53
53
  - !ruby/object:Gem::Version
54
- version: '12.0'
54
+ version: '13.0'
55
55
  - !ruby/object:Gem::Dependency
56
56
  name: rspec
57
57
  requirement: !ruby/object:Gem::Requirement
@@ -72,14 +72,14 @@ dependencies:
72
72
  requirements:
73
73
  - - "~>"
74
74
  - !ruby/object:Gem::Version
75
- version: '9'
75
+ version: '11'
76
76
  type: :development
77
77
  prerelease: false
78
78
  version_requirements: !ruby/object:Gem::Requirement
79
79
  requirements:
80
80
  - - "~>"
81
81
  - !ruby/object:Gem::Version
82
- version: '9'
82
+ version: '11'
83
83
  - !ruby/object:Gem::Dependency
84
84
  name: mysql2
85
85
  requirement: !ruby/object:Gem::Requirement
@@ -98,14 +98,14 @@ dependencies:
98
98
  name: pg
99
99
  requirement: !ruby/object:Gem::Requirement
100
100
  requirements:
101
- - - ">="
101
+ - - "~>"
102
102
  - !ruby/object:Gem::Version
103
103
  version: '0'
104
104
  type: :development
105
105
  prerelease: false
106
106
  version_requirements: !ruby/object:Gem::Requirement
107
107
  requirements:
108
- - - ">="
108
+ - - "~>"
109
109
  - !ruby/object:Gem::Version
110
110
  version: '0'
111
111
  - !ruby/object:Gem::Dependency
@@ -126,14 +126,14 @@ dependencies:
126
126
  name: simplecov
127
127
  requirement: !ruby/object:Gem::Requirement
128
128
  requirements:
129
- - - ">="
129
+ - - "~>"
130
130
  - !ruby/object:Gem::Version
131
131
  version: '0'
132
132
  type: :development
133
133
  prerelease: false
134
134
  version_requirements: !ruby/object:Gem::Requirement
135
135
  requirements:
136
- - - ">="
136
+ - - "~>"
137
137
  - !ruby/object:Gem::Version
138
138
  version: '0'
139
139
  description: |-
@@ -193,8 +193,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
193
193
  - !ruby/object:Gem::Version
194
194
  version: '0'
195
195
  requirements: []
196
- rubyforge_project:
197
- rubygems_version: 2.6.11
196
+ rubygems_version: 3.0.6
198
197
  signing_key:
199
198
  specification_version: 4
200
199
  summary: Simple and robust sharding for Rails, including Migrations and ActiveRecord