ar-octopus 0.8.5 → 0.10.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: a2e171072013526c5597b2d233d983f129a84cec
-  data.tar.gz: 0c05c12e05f5dfb64aaf6f15e0accb59c2174a53
+SHA256:
+  metadata.gz: be74de9c9b6dfacee338dfa5d500da6ce758f501e3be7fa0f154afda07f31f40
+  data.tar.gz: 1892942a155383583a054e4d61cc588ebca2b33749ca191762d550bf9bf13022
 SHA512:
-  metadata.gz: 19f50eb815a6273a87430354c5124096d90d561a8b27c1bfe0073a5a81f3a2242c71f099b88db0bab031018c56a8662ed587122412a0042ba543bc41c6b0d745
-  data.tar.gz: 11828df91744a59445fe6a4add2c6081971f724ed9394b96eb12b36c6a09e69db2546d7dc1eadc56c19074ed5de1d7cd28f39a38c0bd7d3ad88030238a46a98e
+  metadata.gz: c0b21eb7e0fa5397b77c5df6c0bdce5fced41b7284ee7fc4869539ec6219bfcbe2f02a7e85239979b7534a298e2b7ad34e6833fdeb0fd81c67c536b17ea73ae9
+  data.tar.gz: 34bd15fbfb892fca1329f69abf85ba4421d563278d846a67014b710b473919d50cfc3425ed76de1580484a4f10554d4464a490e5ac49b3339553b810acdacfd5

data/.gitignore

@@ -2,6 +2,7 @@
 *.sqlite3
 .bundle
 .rvmrc
+.ruby-version
 Gemfile.lock
 gemfiles/*.lock
 pkg/*

data/.travis.yml

@@ -4,18 +4,15 @@ env:
 before_script:
   - "bundle exec rake db:prepare"
 rvm:
-  - 2.0.0
-  - 2.1.5
-  - 2.2.0
+  - 2.2.7
+  - 2.3.4
+  - 2.4.1
 gemfile:
-  - gemfiles/rails4.gemfile
-  - gemfiles/rails41.gemfile
   - gemfiles/rails42.gemfile
+  - gemfiles/rails5.gemfile
+  - gemfiles/rails51.gemfile
+  - gemfiles/rails52.gemfile
 notifications:
   recipients:
     - gabriel.sobrinho@gmail.com
     - thiago.pradi@gmail.com
-matrix:
-  include:
-    - rvm: 1.9.3
-      gemfile: gemfiles/rails32.gemfile

data/Appraisals

@@ -1,16 +1,16 @@
-appraise "rails32" do
-  gem "activerecord", "~> 3.2.0"
+appraise "rails42" do
+  gem "activerecord", "~> 4.2.0"
 end
 
-appraise "rails4" do
-  gem "activerecord", "~> 4.0.0"
+appraise "rails5" do
+  gem "activerecord", "~> 5.0.0"
 end
 
-appraise "rails41" do
-  gem "activerecord", "~> 4.1.0"
+appraise "rails51" do
+  gem "activerecord", "~> 5.1.0"
 end
 
-appraise "rails42" do
-  gem "activerecord", "~> 4.2.0"
+appraise "rails52" do
+  gem "activerecord", "~> 5.2.0"
 end
 # vim: ft=ruby

data/README.mkdn

@@ -1,11 +1,16 @@
+# Important Notice:
+
+Octopus will enter into maintainance mode once Rails 6 is released - since Rails 6 will include all the features for Database Sharding / Replication into Rails Core (PR: https://github.com/rails/rails/pull/34052).
+Once the first version of Rails 6 beta is released, there will be a migration guide to help users migrate from Octopus to Rails 6.
+
 # Octopus  - Easy Database Sharding for ActiveRecord
 
-<a href='http://www.pledgie.com/campaigns/20950'><img alt='Click here to lend your support to: Octopus and make a donation at www.pledgie.com !' src='http://www.pledgie.com/campaigns/20950.png?skin_name=chrome' border='0' /></a> [![Build Status](https://travis-ci.org/tchandy/octopus.png)](https://travis-ci.org/tchandy/octopus) [![Code Climate](https://codeclimate.com/badge.png)](https://codeclimate.com/github/tchandy/octopus)
+[![Build Status](https://travis-ci.org/thiagopradi/octopus.svg)](https://travis-ci.org/thiagopradi/octopus) [![Code Climate](https://codeclimate.com/badge.png)](https://codeclimate.com/github/thiagopradi/octopus)
 
 Octopus is a better way to do Database Sharding in ActiveRecord. Sharding allows multiple databases in the same rails application. While there are several projects that implement Sharding (e.g. DbCharmer, DataFabric, MultiDb), each project has its own limitations. The main goal of octopus project is to provide a better way of doing Database Sharding.
 
 ## Feature list:
-The api is designed to be simple as possible. Octopus focuses on the end user, giving the power of multiple databases but with reliable code and flexibility. Octopus is compatible with Rails 3.2 and Rails 4.
+The api is designed to be simple as possible. Octopus focuses on the end user, giving the power of multiple databases but with reliable code and flexibility. Octopus is compatible with Rails 4 and Rails 5.
 
 Octopus supports:
 
@@ -15,15 +20,15 @@ Octopus supports:
 - Tools to manage database configurations. (soon)
 
 ### Replication
-When using replication, all writes queries will be sent to master, and read queries to slaves. More info could be found at: <a href="http://wiki.github.com/tchandy/octopus/replication"> Wiki</a>
+When using replication, all writes queries will be sent to master, and read queries to slaves. More info could be found at: <a href="http://wiki.github.com/thiagopradi/octopus/replication"> Wiki</a>
 
 ### Sharding
- When using sharding, you need to specify which shard to send the query. Octopus supports selecting the shard inside a controller, or manually in each object. More could be found at <a href="http://wiki.github.com/tchandy/octopus/sharding"> Wiki</a>
+ When using sharding, you need to specify which shard to send the query. Octopus supports selecting the shard inside a controller, or manually in each object. More could be found at <a href="http://wiki.github.com/thiagopradi/octopus/sharding"> Wiki</a>
 
 ### Replication + Sharding
- When using replication and sharding concurrently, you must specify a shard, and can optionally specify a <a href="https://github.com/tchandy/octopus/wiki/Slave-Groups">slave group</a>.
+ When using replication and sharding concurrently, you must specify a shard, and can optionally specify a <a href="https://github.com/thiagopradi/octopus/wiki/Slave-Groups">slave group</a>.
  All write queries will be sent to each shard's master. If the slave group is specified read queries will be sent to slaves in it, or else to shard's master.
- More info could be found at <a href="https://github.com/tchandy/octopus/wiki/Slave-Groups"> Wiki</a>
+ More info could be found at <a href="https://github.com/thiagopradi/octopus/wiki/Slave-Groups"> Wiki</a>
 
 ## Install
 
@@ -55,21 +60,35 @@ going forward.
 
 ## How to use Octopus?
 
-First, you need to create a config file, shards.yml, inside your config/ directory. to see the syntax and how this file should look, please checkout <a href="http://wiki.github.com/tchandy/octopus/config-file">this page on wiki</a>.
+First, you need to create a config file, shards.yml, inside your config/ directory. to see the syntax and how this file should look, please checkout <a href="http://wiki.github.com/thiagopradi/octopus/config-file">this page on wiki</a>.
 
 ### Syntax
 
 Octopus adds a method to each AR Class and object: the using method is used to select the shard like this:
 
 ```ruby
-User.where(:name => "Thiago").limit(3).using(:slave_one)
+User.where(:name => "Boba").limit(3).using(:read_replica_one)
 ```
 
 Octopus also supports queries within a block. When you pass a block to the using method, all queries inside the block will be sent to the specified shard.
 
 ```ruby
-Octopus.using(:slave_two) do
-  User.create(:name => "Mike")
+Octopus.using(:read_replica_two) do
+  User.create(:name => "Thiago")
+end
+```
+
+If you want to use the same code for all shards or all shards in a specific group (for example in `db/seeds.rb`), you can use this syntax.
+
+```ruby
+# This will return a list of the given block's results, per shard.
+Octopus.using_all do
+  User.create_from_csv!
+end
+
+# This will return a list of the given block's results, per shard in history_shards group.
+Octopus.using_group(:history_shards) do
+  HistoryCategory.create_from_csv!
 end
 ```
 
@@ -131,6 +150,19 @@ octopus:
 
 There is no need for a corresponding `default_migration_shard` - simply define that database to be your master. You might want this setting if all of your databases have identical schemas, but are not replicated.
 
+You can configure a master shard for the rails application, to connect to when rails is going up. The safest would be to configure this to the shard specified in `database.yml` (some things still use it).
+
+```yaml
+octopus:
+  master_shard: <%= ENV['SHARD'] || 'shard1' %>
+```
+
+Then you can use the `SHARD` environment variable to override the `master_shard` specified in `config/shards.yml`, useful for running rake tasks.
+
+```bash
+SHARD=shard1 rake db:setup && SHARD=shard2 rake db:setup
+```
+
 ### Rails Controllers
 
 If you want to send a specified action, or all actions from a controller, to a specific shard,  use this syntax:
@@ -145,8 +177,8 @@ class ApplicationController < ActionController::Base
 end
 ```
 
- To see the complete list of features and syntax, please check out our <a href="http://wiki.github.com/tchandy/octopus/"> Wiki</a>
-Want to see sample rails applications using octopus features? please check it out: <a href="http://github.com/tchandy/octopus_sharding_example">Sharding Example</a> and <a href="http://github.com/tchandy/octopus_replication_example">Replication Example</a>. Also, we have an example that shows how to use Octopus without Rails: <a href="http://github.com/tchandy/octopus_sinatra"> Octopus + Sinatra Example</a>.
+ To see the complete list of features and syntax, please check out our <a href="http://wiki.github.com/thiagopradi/octopus/"> Wiki</a>
+Want to see sample rails applications using octopus features? please check it out: <a href="http://github.com/thiagopradi/octopus_sharding_example">Sharding Example</a> and <a href="http://github.com/thiagopradi/octopus_replication_example">Replication Example</a>. Also, we have an example that shows how to use Octopus without Rails: <a href="http://github.com/thiagopradi/octopus_sinatra"> Octopus + Sinatra Example</a>.
 
 ## Mixing Octopus with the Rails multiple database model
 If you want to set a custom connection to a specific model, use the syntax `octopus_establish_connection` syntax:
@@ -174,13 +206,13 @@ CustomConnectedModel.using(:my_shard).first
 CustomConnectedModel.using(:some_other_shard).first
 ```
 
-This can be useful if you have a model that lives in a separate database and would like to add sharding or replication to it. For other use cases, you may be better off with <a href="https://github.com/tchandy/octopus/wiki/Slave-Groups">slave groups</a>.
+This can be useful if you have a model that lives in a separate database and would like to add sharding or replication to it. For other use cases, you may be better off with <a href="https://github.com/thiagopradi/octopus/wiki/Slave-Groups">slave groups</a>.
 
 ## Contributing with Octopus
 Contributors are welcome! To run the test suite, you need mysql, postgresql and sqlite3 installed. This is what you need to setup your Octopus development environment:
 
 ```bash
-git clone http://github.com/tchandy/octopus.git
+git clone http://github.com/thiagopradi/octopus.git
 cd octopus
 bundle install
 bundle exec rake db:prepare
@@ -200,7 +232,7 @@ cucumber
 If you are having issues running the octopus spec suite, verify your database users and passwords match those inside the config files and your permissions are correct.
 
 ## Contributors:
-- <a href="https://github.com/tchandy/octopus/contributors">All Contributors</a>
+- <a href="https://github.com/thiagopradi/octopus/contributors">All Contributors</a>
 
 ## Mailing List:
 - <a href="http://groups.google.com/group/octopus-activerecord/">Octopus Mailing List</a>

data/Rakefile

@@ -65,6 +65,7 @@ namespace :db do
       class BlankModel < ActiveRecord::Base; end
 
       BlankModel.using(shard_symbol).connection.initialize_schema_migrations_table
+      BlankModel.using(shard_symbol).connection.initialize_metadata_table if Octopus.atleast_rails50? 
 
       BlankModel.using(shard_symbol).connection.create_table(:users) do |u|
         u.string :name

data/ar-octopus.gemspec

@@ -21,22 +21,19 @@ Gem::Specification.new do |s|
                            'Octopus now stores schema version information in each shard and migrations will not ' \
                            'work properly unless this task is invoked.'
 
-  s.add_dependency 'activerecord', '>= 3.2.0'
-  s.add_dependency 'activesupport', '>= 3.2.0'
+  s.required_ruby_version = '>= 2.2.0'
+
+  s.add_dependency 'activerecord', '>= 4.2.0'
+  s.add_dependency 'activesupport', '>= 4.2.0'
 
   s.add_development_dependency 'appraisal', '>= 0.3.8'
-  s.add_development_dependency 'mysql2', '> 0.3'
-  s.add_development_dependency 'pg', '>= 0.11.0'
-  s.add_development_dependency 'rake', '>= 0.8.7'
+  s.add_development_dependency 'mysql2', '>= 0.3.18', "< 0.5"
+  s.add_development_dependency 'pg', '~> 0.18'
+  s.add_development_dependency 'rake'
   s.add_development_dependency 'rspec', '>= 3'
   s.add_development_dependency 'rubocop'
-  s.add_development_dependency 'sqlite3', '>= 1.3.4'
-
-  if RUBY_VERSION < '2.0.0'
-    s.add_development_dependency 'pry-debugger'
-  else
-    s.add_development_dependency 'pry-byebug'
-  end
+  s.add_development_dependency 'sqlite3', '~> 1.3.6'
+  s.add_development_dependency 'pry-byebug'
 
   s.license = 'MIT'
 end

data/gemfiles/rails42.gemfile

@@ -3,5 +3,5 @@
 source "https://rubygems.org"
 
 gem "activerecord", "~> 4.2.0"
-
-gemspec :path => "../"
+gem "mysql2", "0.4.10"
+gemspec path: "../"

data/gemfiles/{rails32.gemfile → rails5.gemfile}

@@ -2,6 +2,6 @@
 
 source "https://rubygems.org"
 
-gem "activerecord", "~> 3.2.0"
+gem "activerecord", "~> 5.0.0"
 
-gemspec :path => "../"
+gemspec path: "../"

data/gemfiles/{rails4.gemfile → rails51.gemfile}

@@ -2,6 +2,6 @@
 
 source "https://rubygems.org"
 
-gem "activerecord", "~> 4.0.0"
+gem "activerecord", "~> 5.1.0"
 
-gemspec :path => "../"
+gemspec path: "../"

data/gemfiles/{rails41.gemfile → rails52.gemfile}

@@ -2,6 +2,6 @@
 
 source "https://rubygems.org"
 
-gem "activerecord", "~> 4.1.0"
+gem "activerecord", "~> 5.2.0"
 
-gemspec :path => "../"
+gemspec path: "../"

data/lib/octopus/abstract_adapter.rb

@@ -2,9 +2,7 @@
 module Octopus
   module AbstractAdapter
     module OctopusShard
-      parent = Octopus.rails3? ? ActiveSupport::BasicObject : ActiveSupport::ProxyObject
-
-      class InstrumenterDecorator < parent
+      class InstrumenterDecorator < ActiveSupport::ProxyObject
         def initialize(adapter, instrumenter)
           @adapter = adapter
           @instrumenter = instrumenter
@@ -20,20 +18,16 @@ module Octopus
         end
       end
 
-      def self.included(base)
-        base.alias_method_chain :initialize, :octopus_shard
-      end
-
       def octopus_shard
         @config[:octopus_shard]
       end
 
-      def initialize_with_octopus_shard(*args)
-        initialize_without_octopus_shard(*args)
+      def initialize(*args)
+        super
         @instrumenter = InstrumenterDecorator.new(self, @instrumenter)
       end
     end
   end
 end
 
-ActiveRecord::ConnectionAdapters::AbstractAdapter.send(:include, Octopus::AbstractAdapter::OctopusShard)
+ActiveRecord::ConnectionAdapters::AbstractAdapter.send(:prepend, Octopus::AbstractAdapter::OctopusShard)

data/lib/octopus/association.rb

@@ -2,6 +2,7 @@ module Octopus
   module Association
     def self.included(base)
       base.send(:include, Octopus::ShardTracking::Dynamic)
+      base.sharded_methods :target_scope
     end
 
     def current_shard

data/lib/octopus/association_shard_tracking.rb

@@ -1,102 +1,72 @@
 module Octopus
   module AssociationShardTracking
-    def self.extended(base)
-      base.send(:include, InstanceMethods)
-    end
+    class MismatchedShards < StandardError
+      attr_reader :record, :current_shard
 
-    module QueryOnCurrentShard
-      METHODS = %w(
-        all
-        average
-        count
-        empty?
-        exists?
-        find
-        find_by_sql
-        first
-        last
-        maximum
-        minimum
-        pluck
-        scoping
-        size
-        sum
-        to_a
-      )
+      def initialize(record, current_shard)
+        @record = record
+        @current_shard = current_shard
+      end
 
-      METHODS.each do |m|
-        define_method m.to_sym do |*args, &block|
-          if self.respond_to?(:proxy_association) && proxy_association
-            proxy_association.owner.run_on_shard { super(*args, &block) }
-          else
-            super(*args, &block)
-          end
-        end
+      def message
+        [
+          "Association Error: Records are from different shards",
+          "Record: #{record.inspect}",
+          "Current Shard: #{current_shard.inspect}",
+          "Current Record Shard: #{record.current_shard.inspect}",
+        ].join(" ")
       end
     end
 
+    def self.extended(base)
+      base.send(:include, InstanceMethods)
+    end
+
     module InstanceMethods
       def connection_on_association=(record)
         return unless ::Octopus.enabled?
         return if !self.class.connection.respond_to?(:current_shard) || !self.respond_to?(:current_shard)
-        if !record.current_shard.nil? && !current_shard.nil? && record.current_shard != current_shard
-          fail 'Association Error: Records are from different shards'
+
+        if !record.current_shard.nil? && !current_shard.nil? && record.current_shard.to_s != current_shard.to_s
+          raise MismatchedShards.new(record, current_shard)
         end
 
         record.current_shard = self.class.connection.current_shard = current_shard if should_set_current_shard?
       end
     end
 
-    if Octopus.rails4?
-      def has_many(association_id, scope = nil, options = {}, &extension)
-        if options == {} && scope.is_a?(Hash)
-          default_octopus_opts(scope)
-        else
-          default_octopus_opts(options)
-        end
-        super
-      end
-    else
-      def has_many(association_id, options = {}, &extension)
+    def has_many(name, scope = nil, **options, &extension)
+      if options == {} && scope.is_a?(Hash)
+        default_octopus_opts(scope)
+      else
         default_octopus_opts(options)
-        super
       end
+      super
     end
 
-    if Octopus.rails4?
-      def has_and_belongs_to_many(association_id, scope = nil, options = {}, &extension)
-        if options == {} && scope.is_a?(Hash)
-          default_octopus_opts(scope)
-        else
-          default_octopus_opts(options)
-        end
-        super
-      end
-    else
-      def has_and_belongs_to_many(association_id, options = {}, &extension)
-        default_octopus_opts(options)
-        super
-      end
+    def has_and_belongs_to_many(association_id, scope = nil, options = {}, &extension)
+      assign_octopus_opts(scope, options)
+      super
     end
 
     def default_octopus_opts(options)
-      if options[:before_add].is_a?(Array)
-        options[:before_add] << :connection_on_association=
-      elsif options[:before_add].is_a?(Symbol)
-        options[:before_add] = [:connection_on_association=, options[:before_add]]
-      else
-        options[:before_add] = :connection_on_association=
-      end
+      options[:before_add] = [ :connection_on_association=, options[:before_add] ].compact.flatten
+      options[:before_remove] = [ :connection_on_association=, options[:before_remove] ].compact.flatten
+    end
 
-      if options[:before_remove].is_a?(Array)
-        options[:before_remove] << :connection_on_association=
-      elsif options[:before_remove].is_a?(Symbol)
-        options[:before_remove] = [:connection_on_association=, options[:before_remove]]
+    def assign_octopus_opts(scope, options)
+      if options == {} && scope.is_a?(Hash)
+        default_octopus_opts(scope)
       else
-        options[:before_remove] = :connection_on_association=
+        default_octopus_opts(options)
       end
+    end
 
-      options[:extend] = [Octopus::AssociationShardTracking::QueryOnCurrentShard, options[:extend]].flatten.compact
+    if Octopus.atleast_rails51?
+      def has_and_belongs_to_many(association_id, scope = nil, **options, &extension)
+        assign_octopus_opts(scope, options)
+        super
+      end
     end
   end
 end

data/lib/octopus/collection_association.rb

@@ -1,9 +1,15 @@
 module Octopus
   module CollectionAssociation
     def self.included(base)
-      base.sharded_methods :reader, :writer, :ids_reader, :ids_writer, :create, :create!,
-                           :build, :any?, :count, :empty?, :first, :include?, :last, :length,
-                           :load_target, :many?, :reload, :size, :select, :uniq
+      if Octopus.rails51? || Octopus.rails52?
+        base.sharded_methods :reader, :writer, :ids_reader, :ids_writer, :create, :create!,
+                             :build, :include?,
+                             :load_target, :reload, :size, :select
+      else
+        base.sharded_methods :reader, :writer, :ids_reader, :ids_writer, :create, :create!,
+                             :build, :any?, :count, :empty?, :first, :include?, :last, :length,
+                             :load_target, :many?, :reload, :size, :select, :uniq
+      end
     end
   end
 end

data/lib/octopus/exception.rb

@@ -0,0 +1,4 @@
+module Octopus
+  class Exception < ::Exception
+  end
+end

data/lib/octopus/finder_methods.rb

@@ -0,0 +1,8 @@
+# find_nth / find_nth! must be public here to allow Octopus to call
+# them on the scope proxy object
+if Octopus.rails42? || Octopus.rails50?
+  module ActiveRecord::FinderMethods
+    public :find_nth
+    public :find_nth!
+  end
+end

data/lib/octopus/load_balancing/round_robin.rb

@@ -6,12 +6,13 @@ module Octopus
   module LoadBalancing
     class RoundRobin
       def initialize(slaves_list)
+        raise Octopus::Exception.new("No slaves available") if slaves_list.empty?
         @slaves_list = slaves_list
         @slave_index = 0
       end
 
       # Returns the next available slave in the pool
-      def next
+      def next(options)
         @slaves_list[@slave_index = (@slave_index + 1) % @slaves_list.length]
       end
     end

data/lib/octopus/log_subscriber.rb

@@ -3,8 +3,12 @@ module Octopus
   module LogSubscriber
     def self.included(base)
       base.send(:attr_accessor, :octopus_shard)
-      base.alias_method_chain :sql, :octopus_shard
-      base.alias_method_chain :debug, :octopus_shard
+
+      base.send :alias_method, :sql_without_octopus_shard, :sql
+      base.send :alias_method, :sql, :sql_with_octopus_shard
+
+      base.send :alias_method, :debug_without_octopus_shard, :debug
+      base.send :alias_method, :debug, :debug_with_octopus_shard
     end
 
     def sql_with_octopus_shard(event)