seamless_database_pool 1.0.12 → 1.0.13

data/README.rdoc

@@ -1,81 +1,83 @@
-Seamless Database Pool provides a simple way in which to add support for a master/slave database cluster to ActiveRecord to allow massive scalability and automatic failover. The guiding design principle behind this code is to make it absolutely trivial to add to an existing, complex application. That way when you have a big, nasty application which needs to scale the database you won't have to stop all feature development just to refactor your database connection code. Let's face it, when the database is having scaling problems, you are in for a world of hurt and the faster you can fix the problem the better.
-
-This code is available as both a Rails plugin and a gem so it will work with any ActiveRecord application.
-
-= Database Clusters
-
-In a master/slave cluster you have one master database server which uses replication to feed all changes to one or more slave databases which are set up to only handle reads. Since most applications put most of the load on the server with reads, this setup can scale out an application quite well. You'll need to work with your database of choice to get replication set up. This plugin has an connection adapter which will handle proxying database requests to the right server.
-
-= Simple Integration
-
-You can convert a standard Rails application (i.e. one that follows the scaffold conventions) to use a database cluster with three simple steps:
-
-1. Set up the database cluster (OK maybe this one isn't simple)
-2. Update database.yml settings to point to the servers in the cluster
-3. Add this code to ApplicationController:
-
-  include SeamlessDatabasePool::ControllerFilter
-  use_database_pool :all => :persistent, [:create, :update, :destroy] => :master
-
-If needed you can control how the connection pool is utilized by wrapping your code in some simple blocks.
- 
-= Failover
-
-One of the other main advantages of using any sort of cluster is that one node can fail without bringing down your application. This plugin automatically handles failing over dead database connections in the read pool. That is if it tries to use a read connection and it is found to be inactive, the connector will try to reconnect. If that fails, it will try another connection in the read pool. After thirty seconds it will try to reconnect the dead connection again. One limitation on failover is that a server with an unreachable IP can't failover on startup. If you have a server completely die and it can't be restarted, you should update the pool configuration immediately to remove that entry.
-
-= Configuration
-
-== The pool configuration
-
-The cluster connections are configured in database.yml using the seamless_database_pool adapter. Any properties you configure for the connection will be inherited by all connections in the pool. In this way, you can configure ports, usernames, etc. once instead of for each connection. One exception is that you can set the pool_adapter property which each connection will inherit as the adapter property. Each connection in the pool uses all the same configuration properties as normal for the adapters.
-
-== The read pool
-
-The read pool is specified with a read_pool property in the pool connection definition in database.yml. This should be specified as an array of hashes where each hash is the configuration for each read connection you'd like to use (see below for an example). As noted above, the configuration for the entire pool will be merged in with the options for each connection.
-
-Each connection can be assigned an additional option of pool_weight. This value should be number which indicates the relative weight that the connection should be given in the pool. If no value is specified, it will default to one. Setting the value to zero will keep the connection out of the pool.
-
-If possible, you should set the permissions on the database user for the read connections to one that only has select permission. This can be especially useful in development and testing to ensure that the read connection never have writes sent to them.
-
-== The master connection
-
-The master connection is specified with a master_connection property in the pool connection definition in database.yml (see below for an example). The master connection will be used for all non-select statements against the database (i.e. insert, update, delete, etc.). It will also be used for all statements inside a transaction or any reload commands.
-
-By default, the master connection will be included in the read pool. If you would like to dedicate this connection only for write operations, you should set the pool weight to zero. Do not duplicate the master connection in the read pool as this will result in the additional overhead of two connections to the database.
-
-== Example configuration
-
-  development:
-    adapter: seamless_database_pool
-    database: mydb_development
-    username: read_user
-    password: abc123
-    pool_adapter: mysql
-    port: 3306
-    master:
-      host: master-db.example.com
-      port: 6000
-      username: master_user
-      password: 567pass
-    read_pool:
-      - host: read-db-1.example.com
-        pool_weight: 2
-      - host: read-db-2.example.com
-
-In this configuration, the master connection will be a mysql connection to master-db.example.com:6000 using the username master_user and the password 567pass.
-
-The read pool will use three mysql connections to master-db, read-db-1, and read-db-2. The master connection will use a different port, username, password for the connection. The read connections will use the same values. Further, the connection read-db-1 will get half the traffic as the other two connections, so presumably it's on a more powerful box.
-
-You must use compatible database adapters for both the master and the read connections. For example, you cannot use an Oracle server as your master and PostgreSQL servers as you read slaves.
-
-= Using the read pool
-
-By default, the master connection will be used for everything. This is not terribly useful, so you should really specify a method of using the read pool for the actions that need it. Read connections will only be used for select statements against the database.
-
-This is done with static methods on SeamlessDatabasePool.
-
-= Controller Filters
-
-To ease integration into a Ruby on Rails application, several controller filters are provided to invoke the above connection methods in a block. These are not implemented as standard controller filters so that the connection methods can be in effect for other filters.
-
+Seamless Database Pool provides a simple way in which to add support for a master/slave database cluster to ActiveRecord to allow massive scalability and automatic failover. The guiding design principle behind this code is to make it absolutely trivial to add to an existing, complex application. That way when you have a big, nasty application which needs to scale the database you won't have to stop all feature development just to refactor your database connection code. Let's face it, when the database is having scaling problems, you are in for a world of hurt and the faster you can fix the problem the better.
+
+This code is available as both a Rails plugin and a gem so it will work with any ActiveRecord application.
+
+= Database Clusters
+
+In a master/slave cluster you have one master database server which uses replication to feed all changes to one or more slave databases which are set up to only handle reads. Since most applications put most of the load on the server with reads, this setup can scale out an application quite well. You'll need to work with your database of choice to get replication set up. This plugin has an connection adapter which will handle proxying database requests to the right server.
+
+= Simple Integration
+
+You can convert a standard Rails application (i.e. one that follows the scaffold conventions) to use a database cluster with three simple steps:
+
+1. Set up the database cluster (OK maybe this one isn't simple)
+2. Update database.yml settings to point to the servers in the cluster
+3. Add this code to ApplicationController:
+
+  include SeamlessDatabasePool::ControllerFilter
+  use_database_pool :all => :persistent, [:create, :update, :destroy] => :master
+
+If needed you can control how the connection pool is utilized by wrapping your code in some simple blocks.
+ 
+= Failover
+
+One of the other main advantages of using any sort of cluster is that one node can fail without bringing down your application. This plugin automatically handles failing over dead database connections in the read pool. That is if it tries to use a read connection and it is found to be inactive, the connector will try to reconnect. If that fails, it will try another connection in the read pool. After thirty seconds it will try to reconnect the dead connection again.
+
+One limitation on failover is when database servers are down when the pool is being initialized during startup. In this case, the connections cannot be initialized and are not added to the pool. If this happens, you will need to restart your processes once the database servers are back online.
+
+= Configuration
+
+== The pool configuration
+
+The cluster connections are configured in database.yml using the seamless_database_pool adapter. Any properties you configure for the connection will be inherited by all connections in the pool. In this way, you can configure ports, usernames, etc. once instead of for each connection. One exception is that you can set the pool_adapter property which each connection will inherit as the adapter property. Each connection in the pool uses all the same configuration properties as normal for the adapters.
+
+== The read pool
+
+The read pool is specified with a read_pool property in the pool connection definition in database.yml. This should be specified as an array of hashes where each hash is the configuration for each read connection you'd like to use (see below for an example). As noted above, the configuration for the entire pool will be merged in with the options for each connection.
+
+Each connection can be assigned an additional option of pool_weight. This value should be number which indicates the relative weight that the connection should be given in the pool. If no value is specified, it will default to one. Setting the value to zero will keep the connection out of the pool.
+
+If possible, you should set the permissions on the database user for the read connections to one that only has select permission. This can be especially useful in development and testing to ensure that the read connection never have writes sent to them.
+
+== The master connection
+
+The master connection is specified with a master_connection property in the pool connection definition in database.yml (see below for an example). The master connection will be used for all non-select statements against the database (i.e. insert, update, delete, etc.). It will also be used for all statements inside a transaction or any reload commands.
+
+By default, the master connection will be included in the read pool. If you would like to dedicate this connection only for write operations, you should set the pool weight to zero. Do not duplicate the master connection in the read pool as this will result in the additional overhead of two connections to the database.
+
+== Example configuration
+
+  development:
+    adapter: seamless_database_pool
+    database: mydb_development
+    username: read_user
+    password: abc123
+    pool_adapter: mysql2
+    port: 3306
+    master:
+      host: master-db.example.com
+      port: 6000
+      username: master_user
+      password: 567pass
+    read_pool:
+      - host: read-db-1.example.com
+        pool_weight: 2
+      - host: read-db-2.example.com
+
+In this configuration, the master connection will be a mysql connection to master-db.example.com:6000 using the username master_user and the password 567pass.
+
+The read pool will use three mysql connections to master-db, read-db-1, and read-db-2. The master connection will use a different port, username, password for the connection. The read connections will use the same values. Further, the connection read-db-1 will get half the traffic as the other two connections, so presumably it's on a more powerful box.
+
+You must use compatible database adapters for both the master and the read connections. For example, you cannot use an Oracle server as your master and PostgreSQL servers as you read slaves.
+
+= Using the read pool
+
+By default, the master connection will be used for everything. This is not terribly useful, so you should really specify a method of using the read pool for the actions that need it. Read connections will only be used for select statements against the database.
+
+This is done with static methods on SeamlessDatabasePool.
+
+= Controller Filters
+
+To ease integration into a Ruby on Rails application, several controller filters are provided to invoke the above connection methods in a block. These are not implemented as standard controller filters so that the connection methods can be in effect for other filters.
+
 See SeamlessDatabasePool::ControllerFilter for more details.

data/Rakefile

@@ -65,7 +65,7 @@ begin
   Jeweler::Tasks.new do |gem|
     gem.name = "seamless_database_pool"
     gem.summary = "Add support for master/slave database clusters in ActiveRecord to improve performance."
-    gem.email = "brian@embellishedvisions.com"
+    gem.email = "bbdurand@gmail.com"
     gem.homepage = "http://github.com/bdurand/seamless_database_pool"
     gem.authors = ["Brian Durand"]
     gem.files = FileList["lib/**/*", "spec/**/*", "README.rdoc", "Rakefile", "MIT-LICENSE"].to_a

data/lib/active_record/connection_adapters/seamless_database_pool_adapter.rb

@@ -38,14 +38,8 @@ module ActiveRecord
           end
         end if config[:read_pool]
 
-        @seamless_database_pool_classes ||= {}
-        klass = @seamless_database_pool_classes[master_connection.class]
-        unless klass
-          klass = ActiveRecord::ConnectionAdapters::SeamlessDatabasePoolAdapter.adapter_class(master_connection)
-          @seamless_database_pool_classes[master_connection.class] = klass
-        end
-
-        return klass.new(nil, logger, master_connection, read_connections, pool_weights)
+        klass = ::ActiveRecord::ConnectionAdapters::SeamlessDatabasePoolAdapter.adapter_class(master_connection)
+        klass.new(nil, logger, master_connection, read_connections, pool_weights)
       end
 
       def establish_adapter(adapter)
@@ -95,6 +89,9 @@ module ActiveRecord
       class << self
         # Create an anonymous class that extends this one and proxies methods to the pool connections.
         def adapter_class(master_connection)
+          adapter_class_name = master_connection.adapter_name.classify
+          return const_get(adapter_class_name) if const_defined?(adapter_class_name, false)
+          
           # Define methods to proxy to the appropriate pool
           read_only_methods = [:select_one, :select_all, :select_value, :select_values, :select, :select_rows, :execute, :tables, :columns]
           master_methods = []
@@ -134,7 +131,9 @@ module ActiveRecord
             EOS
           end
           klass.send :protected, :select
-        
+
+          const_set(adapter_class_name, klass)
+          
           return klass
         end
       
@@ -149,11 +148,11 @@ module ActiveRecord
       end
       
       def initialize(connection, logger, master_connection, read_connections, pool_weights)
-        super(connection, logger)
-        
         @master_connection = master_connection
         @read_connections = read_connections.dup.freeze
         
+        super(connection, logger)
+        
         @weighted_read_connections = []
         pool_weights.each_pair do |conn, weight|
           weight.times{@weighted_read_connections << conn}
@@ -214,6 +213,23 @@ module ActiveRecord
         do_to_connections {|conn| total += conn.reset_runtime}
         total
       end
+
+      def lease
+        synchronize do
+          unless @in_use
+            @in_use   = true
+            @last_use = Time.now
+          end
+        end
+      end
+
+      def expire
+        @in_use = false
+      end
+
+      def close
+        pool.checkin self
+      end
       
       # Get a random read connection from the pool. If the connection is not active, it will attempt to reconnect
       # to the database. If that fails, it will be removed from the pool for one minute.

data/lib/seamless_database_pool.rb

@@ -2,6 +2,8 @@ require File.join(File.dirname(__FILE__), 'seamless_database_pool', 'connect_tim
 require File.join(File.dirname(__FILE__), 'seamless_database_pool', 'connection_statistics.rb')
 require File.join(File.dirname(__FILE__), 'seamless_database_pool', 'controller_filter.rb')
 require File.join(File.dirname(__FILE__), 'active_record', 'connection_adapters', 'seamless_database_pool_adapter.rb')
+require File.join(File.dirname(__FILE__), 'seamless_database_pool', 'railtie.rb') if defined?(Rails::Railtie)
+
 $LOAD_PATH << File.dirname(__FILE__) unless $LOAD_PATH.include?(File.dirname(__FILE__))
 
 # This module allows setting the read pool connection type. Generally you will use one of
@@ -126,6 +128,22 @@ module SeamlessDatabasePool
       class_name = ADAPTER_TO_CLASS_NAME_MAP[name] || name.camelize
       "ActiveRecord::ConnectionAdapters::#{class_name}Adapter".constantize
     end
+    
+    # Pull out the master configuration for compatibility with such things as the Rails' rake db:*
+    # tasks which only support known adapters.
+    def master_database_configuration(database_configs)
+      configs = {}
+      database_configs.each do |key, values|
+        if values['adapter'] == 'seamless_database_pool'
+          values['adapter'] = values.delete('pool_adapter')
+          values = values.merge(values['master']) if values['master'].is_a?(Hash)
+          values.delete('pool_weight')
+          values.delete('master')
+          values.delete('read_pool')
+        end
+        configs[key] = values
+      end
+      configs
+    end
   end
-  
 end

data/lib/seamless_database_pool/railtie.rb

@@ -0,0 +1,13 @@
+module SeamlessDatabasePool
+  class Railtie < ::Rails::Railtie
+    rake_tasks do
+      namespace :db do
+        task :load_config do
+          # Override seamless_database_pool configuration so db:* rake tasks work as expected.
+          original_config = Rails.application.config.database_configuration
+          ActiveRecord::Base.configurations = SeamlessDatabasePool.master_database_configuration(original_config)
+        end
+      end
+    end
+  end
+end

data/spec/connection_adapters_spec.rb

@@ -14,12 +14,12 @@ describe "Test connection adapters" do
   
         before(:all) do
           ActiveRecord::Base.establish_connection('adapter' => "sqlite3", 'database' => ":memory:")
-          model.create_tables
+          SeamlessDatabasePool::TestModel.db_model(adapter).create_tables
         end
   
         after(:all) do
-          model.drop_tables
-          model.cleanup_database!
+          SeamlessDatabasePool::TestModel.db_model(adapter).drop_tables
+          SeamlessDatabasePool::TestModel.db_model(adapter).cleanup_database!
         end
   
         before(:each) do

data/spec/database.yml

@@ -30,8 +30,8 @@ mysql:
   username: root
   password:
   master:
-    adapter: mysql
+    adapter: mysql2
     pool_weight: 0
   read_pool:
     - adapter: read_only
-      real_adapter: mysql
+      real_adapter: mysql2

data/spec/seamless_database_pool_spec.rb

@@ -131,4 +131,38 @@ describe "SeamlessDatabasePool" do
     SeamlessDatabasePool.read_only_connection_type(nil).should == nil
   end
   
+  it "should pull out the master configurations for compatibility with rake db:* tasks" do
+    config = {
+      'development' => {
+        'adapter' => 'seamless_database_pool',
+        'pool_adapter' => 'mysql2',
+        'database' => 'development',
+        'username' => 'root',
+        'master' => {
+          'host' => 'localhost',
+          'pool_weight' => 2
+        },
+        'read_pool' => {
+          'host' => 'slavehost',
+          'pool_weight' => 5
+        }
+      },
+      'test' => {
+        'adapter' => 'mysql2',
+        'database' => 'test'
+      }
+    }
+    SeamlessDatabasePool.master_database_configuration(config).should == {
+      'development' => {
+        'adapter' => 'mysql2',
+        'database' => 'development',
+        'username' => 'root',
+        'host' => 'localhost'
+      },
+      'test' => {
+        'adapter' => 'mysql2',
+        'database' => 'test'
+      }
+    }
+  end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: seamless_database_pool
 version: !ruby/object:Gem::Version
-  version: 1.0.12
+  version: 1.0.13
   prerelease: 
 platform: ruby
 authors:
@@ -9,11 +9,11 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-11-08 00:00:00.000000000 Z
+date: 2013-06-11 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
-  requirement: &70158253976860 !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -21,10 +21,15 @@ dependencies:
         version: 2.2.2
   type: :runtime
   prerelease: false
-  version_requirements: *70158253976860
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 2.2.2
 - !ruby/object:Gem::Dependency
   name: rspec
-  requirement: &70158253974440 !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -32,10 +37,15 @@ dependencies:
         version: '2.0'
   type: :development
   prerelease: false
-  version_requirements: *70158253974440
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '2.0'
 - !ruby/object:Gem::Dependency
   name: jeweler
-  requirement: &70158253971880 !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -43,10 +53,15 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *70158253971880
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: sqlite3
-  requirement: &70158253970680 !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -54,10 +69,15 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *70158253970680
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: mysql
-  requirement: &70158253960160 !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -65,10 +85,15 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *70158253960160
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: pg
-  requirement: &70158253958100 !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -76,9 +101,14 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *70158253958100
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 description: 
-email: brian@embellishedvisions.com
+email: bbdurand@gmail.com
 executables: []
 extensions: []
 extra_rdoc_files:
@@ -94,6 +124,7 @@ files:
 - lib/seamless_database_pool/connect_timeout.rb
 - lib/seamless_database_pool/connection_statistics.rb
 - lib/seamless_database_pool/controller_filter.rb
+- lib/seamless_database_pool/railtie.rb
 - spec/connection_adapters_spec.rb
 - spec/connection_statistics_spec.rb
 - spec/controller_filter_spec.rb
@@ -123,7 +154,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 1.8.11
+rubygems_version: 1.8.25
 signing_key: 
 specification_version: 3
 summary: Add support for master/slave database clusters in ActiveRecord to improve