rails-sharding 1.1.0 → 1.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 9f4f44b9e47e25d00cce1cd88897d85750be2465
-  data.tar.gz: c84a17b9c9a59fd471e627c0784450cba5930451
+SHA256:
+  metadata.gz: b567029d770be424e7c1141bd5f483fd5fee49a630ee31248d66af9d57976ab9
+  data.tar.gz: 0aa3dd59efebe3aba5949784fcd342d3595dbe719057db9a68341480d3dae9bd
 SHA512:
-  metadata.gz: 113d4ba656d6ff072c81a116141cc4b7057d914fa64f9ccf00f64e4a0a08b062e4fde0b388216143ef5c5ba378c564b90fb2615b7d765683901d9d335b7ca28a
-  data.tar.gz: edda93d0db35871a3dd1b2cc0a95838112dfa928da60e925ae98f7d4467e298112907a6d3b800fc7267a9d0d41d8ee5a3d68d5433f629748e6dbad1b7dacf049
+  metadata.gz: 88089653b9b9c99acb012872175d426d5b60aad54b7e65a2d98afb6b128c7b7a1e904cdc62d06db1d68bd4c3d2589d1ed5cc7e6537d9ced97bc2f9e0044ba35b
+  data.tar.gz: 338d7f533b28c46625a0102cc861932a8819bde1084a02b37cf8a7ae1593b948bd35aefc6b0c5355967399453a70232f28bb465aaefdb86aaf2fe965c1d3b4c1

data/.ruby-version

@@ -1 +1 @@
-ruby-2.4.1
+ruby-2.5.7

data/.travis.yml

@@ -1,7 +1,7 @@
 sudo: false
 language: ruby
 rvm:
-  - 2.4.1
+  - 2.5.7
 services:
   - mysql
   - postgresql

data/README.md

@@ -8,28 +8,34 @@
 
 Simple and robust sharding gem for Rails, including Migrations and ActiveRecord extensions
 
-This gems allows you to easily create extra databases to your rails application, and freely allocate ActiveRecord instances to any of the databases. It also provides rake tasks and migrations to help you manage the schema by shard groups.
+This gems allows you to easily create extra databases to your rails application, and freely allocate ActiveRecord instances to any of the databases.
 
-After you have setup your shards, accessing them is as simple as:
+Accessing shards is as simple as:
 ```ruby
+  # creating a user to a specific shard
   new_user = User.using_shard(:shard_group1, :shard1).create(username: 'x')
+
+  # retrieving a user from a specific shard
   loaded_user = User.using_shard(:shard_group1, :shard1).where(username: 'x').first
 ```
 
-You can also use the block syntax, where all your queries inside will be directed to the correct shard:
+You can also use the block syntax:
 ```ruby
   Rails::Sharding.using_shard(:shard_group1, :shard1) do
+    # All statements inside this block will go to the selected shard
+
+    # Do some queries
     new_user = User.create(username: 'x')
     loaded_user = User.where(username: 'x').first
     billing_infos = loaded_user.billing_infos.all
   end
 ```
 
-You can also pick and choose which models will be shardable, so that all the models that are not shardable will still be retrieved from the master database, even if inside a using_shard block.
+You can also pick and choose which models will be shardable. Non shardable models will be retrieved from the master database, even if inside a `using_shard` block.
 
 ## Compatibility
 Gem version 1.x.x:
-* Rails 5.0
+* Rails 5.0, 5.1 and 5.2
 * Databases: MySQL, MariaDB, Postgres
 
 Gem version 0.x.x:
@@ -51,7 +57,6 @@ bundle
 ```
 
 ## Creating Shards
-This gem helps you create shards that are additional and completely separate from your master database. The master database is the one that is created and managed through rails, and is the default storage for all your models.
 
 To start with the rails-sharding gem, run the command
 ```
@@ -80,7 +85,9 @@ development:
 ...
 ```
 
-Rename it to `config/shards.yml` and change it to your database configuration. This example file defines a single shard group (named `shard_group1`) containing two shards (`shard1` and `shard2`). A shard group is simply a set of shards that should have the same schema.
+Rename it to `config/shards.yml` and change it to your database configuration. This example file defines a single shard group (named `shard_group1`) containing two shards (`shard1` and `shard2`).
+
+**A shard group is a set of shards that should have the same schema.**
 
 When you're ready to create the shards run
 ```
@@ -88,7 +95,13 @@ rake shards:create
 ```
 
 ## Migrating Shards
-Go to the directory `db/shards_migrations/shard_group1` and add all migrations that you want to run on the shards of `shard_group1`. By design, all shards in a same group should always have the same schema. For example, add the following migration to your `db/shards_migrations/shard_group1`:
+Go to the directory `db/shards_migrations/shard_group1` and add all migrations that you want to run on the shards of `shard_group1`. By design, all shards in a same group should always have the same schema.
+
+
+As of now, there is no generator for migrations. You can use the regular rails generator and move the migrations to the `shards_migration` folder.
+
+
+For example, add the following migration to your `db/shards_migrations/shard_group1`:
 ```ruby
 # 20160808000000_create_users.rb
 class CreateClients < ActiveRecord::Migration[5.0]
@@ -175,15 +188,18 @@ Rails::Sharding.setup do |config|
 end
 ```
 
+## Wiki
+Want to know more? How to integrate with RSpec, Capistrano, etc? Take a look at our [wiki](https://github.com/hsgubert/rails-sharding/wiki).
+
 ## Development and Contributing
 
 After checking out the repo:
 
-1. Run `bundle` to install gems 
+1. Run `bundle` to install gems
 
 1. Create your `spec/fixtures/shards.yml` based on the example on this same folder (you need MySQL and Postgres)
 
-1. Run `rake db:test:prepare` to create the test shards. 
+1. Run `rake db:test:prepare` to create the test shards.
 
 1. Run `rspec` to run the tests.
 

data/lib/rails/sharding/active_record_extensions.rb

@@ -78,7 +78,7 @@ module Rails::Sharding
       alias_method :eql?, :==
     end
 
-    # Fixes case when behavior when ScopeProxy is passed to case
+    # Fixes case-when behavior when ScopeProxy is passed to case
     # (otherwise classes don't match)
     module CaseFixer
       def ===(other)

data/lib/rails/sharding/connection_handler.rb

@@ -78,7 +78,7 @@ module Rails::Sharding
     end
 
     def self.with_connection(shard_group, shard_name, &block)
-    	connection_pool(shard_group, shard_name).with_connection do |connection|
+      connection_pool(shard_group, shard_name).with_connection do |connection|
         if connection && Config.add_shard_tag_to_query_logs
           connection_name = connection_name(shard_group, shard_name)
           add_shard_tag_to_connection_log(connection, connection_name)

data/lib/rails/sharding/core.rb

@@ -13,10 +13,11 @@ module Rails::Sharding
       ShardThreadRegistry.push_current_shard(shard_group, shard_name)
       yield
     ensure
+      was_connected_to_master = ShardThreadRegistry.connecting_to_master?
       shard_group, shard_name, connection_used = ShardThreadRegistry.pop_current_shard
 
-      # shows warning to user
-      if Config.no_connection_retrieved_warning && !connection_used
+      # shows warning to user (except when connected to master database)
+      if Config.no_connection_retrieved_warning && !connection_used && !was_connected_to_master
         puts "Warning: no connection to shard '#{shard_group}:#{shard_name}' was retrieved inside the using_shard block. Make sure you don't forget to include Rails::Sharding::ShardableModel to the models you want to be sharded. Disable this warning with Rails::Sharding::Config.no_connection_retrieved_warning = false."
       end
 
@@ -32,7 +33,7 @@ module Rails::Sharding
       environment_config = @@db_configs[environment]
       return environment_config if environment_config
 
-      raise Errors::ConfigNotFoundError, 'Found no shard configurations for enviroment "' + environment + '" in ' + Config.shards_config_file.to_s + ' file was not found'
+      raise Errors::ConfigNotFoundError, 'Found no shard configurations for environment "' + environment + '" in ' + Config.shards_config_file.to_s + ' file was not found'
     rescue Errno::ENOENT
       raise Errors::ConfigNotFoundError, Config.shards_config_file.to_s + ' file was not found'
     end
@@ -56,11 +57,11 @@ module Rails::Sharding
     # yields a block for each shard in each shard group, with its configurations
     # shard_group_filter: if passed yields only shards of this group
     # shard_name_filter: if passed yields only shards with this name
-    def self.for_each_shard(shard_group_filter=nil, shard_name_filter=nil)
+    def self.for_each_shard(environment:Rails.env, shard_group_filter:nil, shard_name_filter:nil)
       shard_group_filter.to_s if shard_group_filter
       shard_name_filter.to_s if shard_name_filter
 
-      configurations.each do |shard_group, shards_configurations|
+      configurations(environment).each do |shard_group, shards_configurations|
         next if shard_group_filter && shard_group_filter != shard_group.to_s
 
         shards_configurations.each do |shard, configuration|

data/lib/rails/sharding/shardable_model.rb

@@ -52,7 +52,7 @@ module Rails::Sharding
         end
       end
 
-      # @overrides ActiveRecord::ConnectionHandling#sharded_connected?
+      # @overrides ActiveRecord::ConnectionHandling#connected?
       def sharded_connected?
         if ShardThreadRegistry.connecting_to_master?
           return original_connected?
@@ -107,6 +107,15 @@ module Rails::Sharding
           return ConnectionHandler.connection_handler.clear_all_connections!
         end
       end
+
+      # @overrides ActiveRecord::ConnectionHandling#flush_idle_connections!
+      def sharded_flush_idle_connections!
+        if ShardThreadRegistry.connecting_to_master?
+          return original_flush_idle_connections!
+        else
+          return ConnectionHandler.connection_handler.flush_idle_connections!
+        end
+      end
     end
 
   end

data/lib/rails/sharding/version.rb

@@ -1,5 +1,5 @@
 module Rails
   module Sharding
-    VERSION = "1.1.0"
+    VERSION = "1.2.1"
   end
 end

data/lib/tasks/rails-sharding.rake

@@ -27,25 +27,32 @@ shards_namespace = namespace :shards do
 
   desc "Creates database shards (options: RAILS_ENV=x SHARD_GROUP=x SHARD=x)"
   task create: [:environment] do
-    Rails::Sharding.for_each_shard(ENV["SHARD_GROUP"], ENV["SHARD"]) do |shard_group, shard, configuration|
-      puts "== Creating shard #{shard_group}:#{shard}"
-      ActiveRecord::Tasks::DatabaseTasks.create(configuration)
+    # creates DB for both development and test envs, when in development
+    each_current_environment do |environment|
+      Rails::Sharding.for_each_shard(environment: environment, shard_group_filter: ENV["SHARD_GROUP"], shard_name_filter: ENV["SHARD"]) do |shard_group, shard, configuration|
+        puts "== Creating shard #{shard_group}:#{shard}"
+
+        ActiveRecord::Tasks::DatabaseTasks.create(configuration)
+      end
     end
   end
 
   desc "Drops database shards (options: RAILS_ENV=x SHARD_GROUP=x SHARD=x)"
   task drop: [:environment, :check_protected_environments] do
-    Rails::Sharding.for_each_shard(ENV["SHARD_GROUP"], ENV["SHARD"]) do |shard_group, shard, configuration|
-      puts "== Dropping shard #{shard_group}:#{shard}"
+    # drops DB for both development and test envs, when in development
+    each_current_environment do |environment|
+      Rails::Sharding.for_each_shard(shard_group_filter: ENV["SHARD_GROUP"], shard_name_filter: ENV["SHARD"]) do |shard_group, shard, configuration|
+        puts "== Dropping shard #{shard_group}:#{shard}"
 
-      # closes connections with shard before dropping (postgres requires this, mysql does not but there is no harm)
-      Rails::Sharding::ConnectionHandler.remove_connection(shard_group, shard)
+        # closes connections with shard before dropping (postgres requires this, mysql does not but there is no harm)
+        Rails::Sharding::ConnectionHandler.remove_connection(shard_group, shard)
 
-      ActiveRecord::Tasks::DatabaseTasks.drop(configuration)
+        ActiveRecord::Tasks::DatabaseTasks.drop(configuration)
 
-      # reestablishes connection (because we removed before). You can do this even if the database does not exist yet,
-      # you just cannot retrieve the connection yet.
-      Rails::Sharding::ConnectionHandler.establish_connection(shard_group, shard)
+        # reestablishes connection (because we removed before). You can do this even if the database does not exist yet,
+        # you just cannot retrieve the connection yet.
+        Rails::Sharding::ConnectionHandler.establish_connection(shard_group, shard)
+      end
     end
   end
 
@@ -91,7 +98,7 @@ shards_namespace = namespace :shards do
     task dump: [:_make_activerecord_base_shardable] do
       require "active_record/schema_dumper"
 
-      Rails::Sharding.for_each_shard(ENV["SHARD_GROUP"], ENV["SHARD"]) do |shard_group, shard, _configuration|
+      Rails::Sharding.for_each_shard(shard_group_filter: ENV["SHARD_GROUP"], shard_name_filter: ENV["SHARD"]) do |shard_group, shard, _configuration|
         puts "== Dumping schema of #{shard_group}:#{shard}"
 
         schema_filename = shard_schema_path(shard_group, shard)
@@ -216,7 +223,7 @@ shards_namespace = namespace :shards do
 
   desc "Retrieves the current schema version number"
   task version: [:_make_activerecord_base_shardable] do
-    Rails::Sharding.for_each_shard(ENV["SHARD_GROUP"], ENV["SHARD"]) do |shard_group, shard, _configuration|
+    Rails::Sharding.for_each_shard(shard_group_filter: ENV["SHARD_GROUP"], shard_name_filter: ENV["SHARD"]) do |shard_group, shard, _configuration|
       Rails::Sharding.using_shard(shard_group, shard) do
         puts "Shard #{shard_group}:#{shard} version: #{ActiveRecord::Migrator.current_version}"
       end
@@ -273,8 +280,8 @@ shards_namespace = namespace :shards do
     end
 
     desc "Empty the test shards (drops all tables) (options: SHARD_GROUP=x, SHARD=x)"
-    task :purge => [:_make_activerecord_base_shardable] do
-      Rails::Sharding.for_each_shard(ENV["SHARD_GROUP"], ENV["SHARD"]) do |shard_group, shard, configuration|
+    task purge: [:_make_activerecord_base_shardable] do
+      Rails::Sharding.for_each_shard(environment: 'test', shard_group_filter: ENV["SHARD_GROUP"], shard_name_filter: ENV["SHARD"]) do |shard_group, shard, configuration|
         puts "== Purging test shard #{shard_group}:#{shard}"
         begin
           # establishes connection with test shard, saving if it was connected before (rails 4.2 doesn't do this, but should)
@@ -316,4 +323,13 @@ shards_namespace = namespace :shards do
     raise error_message unless version
     version
   end
+
+  def each_current_environment
+    environments = [Rails.env]
+    environments << "test" if environments == ["development"]
+
+    environments.each do |env|
+      yield env
+    end
+  end
 end

data/rails-sharding.gemspec

@@ -23,12 +23,12 @@ Gem::Specification.new do |spec|
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ["lib"]
 
-  spec.add_runtime_dependency 'rails', '~> 5.1.0'
+  spec.add_runtime_dependency 'rails', '~> 5.2.0'
 
-  spec.add_development_dependency "bundler", "~> 1.15"
-  spec.add_development_dependency "rake", "~> 12.0"
+  spec.add_development_dependency "bundler", "~> 1.17"
+  spec.add_development_dependency "rake", "~> 13.0"
   spec.add_development_dependency "rspec", "~> 3.0"
-  spec.add_development_dependency "byebug", '~> 9'
+  spec.add_development_dependency "byebug", '~> 11'
   spec.add_development_dependency "mysql2", '~> 0'
   spec.add_development_dependency "pg", '~> 0' # postgres driver
   spec.add_development_dependency "codeclimate-test-reporter", '~> 1'

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: rails-sharding
 version: !ruby/object:Gem::Version
-  version: 1.1.0
+  version: 1.2.1
 platform: ruby
 authors:
 - Henrique Gubert
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2017-09-13 00:00:00.000000000 Z
+date: 2020-06-10 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
@@ -16,42 +16,42 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 5.1.0
+        version: 5.2.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 5.1.0
+        version: 5.2.0
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.15'
+        version: '1.17'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.15'
+        version: '1.17'
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '12.0'
+        version: '13.0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '12.0'
+        version: '13.0'
 - !ruby/object:Gem::Dependency
   name: rspec
   requirement: !ruby/object:Gem::Requirement
@@ -72,14 +72,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '9'
+        version: '11'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '9'
+        version: '11'
 - !ruby/object:Gem::Dependency
   name: mysql2
   requirement: !ruby/object:Gem::Requirement
@@ -193,8 +193,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project: 
-rubygems_version: 2.6.11
+rubygems_version: 3.0.6
 signing_key: 
 specification_version: 4
 summary: Simple and robust sharding for Rails, including Migrations and ActiveRecord