active_record_host_pool 1.2.4 → 2.0.0.pre.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3f3e4946feb2a146c7dd715f51c583dc6bc9723fe08491023ec85adac537a22b
-  data.tar.gz: 708604983ef9815557c23c13ce88fb40588f4eb62ce4cf4e5363bd2a2f2eec2a
+  metadata.gz: c4c99be0433d92bea15d4c49896f356536455e944e7b11e2ae7f7ea53b3e7d1a
+  data.tar.gz: 96a0b0d168ea82987e4e128263351c23dd80c390abed82d187e73393f0003b80
 SHA512:
-  metadata.gz: d67cdfa238ffb5a68f333a86eda8855e25b3e763f1bc1b5e8abb11766d48e524de7d48745229f62f241b17cd79c699e40ab8561995875771133ae7b28cfca3d4
-  data.tar.gz: 2dac7894db1c4b33c66924323578464929aeb370d6a79dfeab9c8ecb597bcb5299ab5de1294c41d7a68e949637f897928e54e8e3c4b8090300576a4e8976001e
+  metadata.gz: 2fadaf3620e2adad2931483ad3d6f1d379e3ab252872245c7cf1e3fe3133bfc2dda16ecb8f83632d4898fd850621d064393a84078edc0e2c62338cb67aecd080
+  data.tar.gz: b28ac1e4864dc7c0d9ed8924132af7394b9144ec5a77a55e1d99ce2a3863209f8cb1967732dc628675bbc580ef252189c49891f473fd56ccbff00aca42464c50

data/Changelog.md

@@ -0,0 +1,112 @@
+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and as of v1.0.0 this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [2.0.0.pre.2]
+
+### Added
+- Add support for Rails 7.1.
+- `Trilogy` is now a supported MySQL database adapter. ActiveRecordHostPool no longer requires `mysql2`, nor does it explicitly require `activerecord-trilogy-adapter`. Applications using ARHP will now need to explicitly require one of these adapters in its gemfile. When using `activerecord-trilogy-adapter` also ensure that the `trilogy` gem is locked to `v2.5.0+`.
+
+### Removed
+- Remove `mysql2` as a direct dependency, test Rails 7.0 with `mysql2` and `activerecord-trilogy-adapter`.
+- Remove support for Rails 5.1, 5.2, and 6.0.
+
+## [1.2.5] - 2023-07-14
+### Added
+- Start testing with Ruby 3.2.
+
+### Removed
+- Drop Ruby 2.6.
+
+### Fixed
+- Use a mutex inside `PoolProxy#disconnect!`. This might fix some `ActiveRecord::ConnectionNotEstablished` issues when a multi-threaded application is under heavy load. (Only applies when using Rails 6.1 or newer).
+
+## [1.2.4] - 2023-03-20
+### Fixed
+- Fixed the warning when using `ruby2_keywords` on `execute_with_switching`.
+- Simplified the `clear_query_caches_for_current_thread` patch.
+
+## [1.2.3] - 2023-01-19
+### Fixed
+- Fix the patch for `ActiveRecord::Base.clear_query_caches_for_current_thread` to work correctly right after the creation of a new connection pool. (https://github.com/zendesk/active_record_host_pool/pull/105)
+
+## [1.2.2] - 2023-01-18
+### Added
+- Add a new `ActiveRecordHostPool::PoolProxy#_unproxied_connection` method which gives access to the underlying, "real", shared connection without going through the connection proxy, which would call `#_host_pool_current_database=` on the underlying connection. (https://github.com/zendesk/active_record_host_pool/pull/104)
+
+### Fixed
+- Fix the patch for `ActiveRecord::Base.clear_on_handler` to work correctly right after the creation of a new connection pool. (https://github.com/zendesk/active_record_host_pool/pull/104)
+
+## [1.2.1] - 2022-12-23
+### Fixed
+- Fix forwarding of kwargs when calling `#execute` in Rails 7. (https://github.com/zendesk/active_record_host_pool/pull/101)
+
+## [1.2.0] - 2022-10-13
+### Added
+- Support for Rails 7.0 with [legacy_connection_handling=false and legacy_connection_handling=true](https://github.com/zendesk/active_record_host_pool/pull/95)
+- Start testing with Ruby 3.0 & 3.1
+
+## [1.1.1] - 2022-08-26
+### Fixed
+- Ensure that recently added files "lib/active_record_host_pool/pool_proxy_6_1.rb" and "lib/active_record_host_pool/pool_proxy_legacy.rb" are built into the shipped gem. (https://github.com/zendesk/active_record_host_pool/pull/92)
+
+## [1.1.0] - 2022-08-26
+### Added
+- Support for Rails 6.1 with [legacy_connection_handling=false](https://github.com/zendesk/active_record_host_pool/pull/90) and [legacy_connection_handling=true](https://github.com/zendesk/active_record_host_pool/pull/88)
+
+### Removed
+- Removed compatibility with Rails 4.2. (https://github.com/zendesk/active_record_host_pool/pull/71)
+- Removed compatibility with Ruby 2.5 and lower. (https://github.com/zendesk/active_record_host_pool/pull/80)
+
+## [1.0.3] - 2021-02-09
+### Fixed
+- Add missing file to the released gem. (https://github.com/zendesk/active_record_host_pool/pull/68)
+
+## [1.0.2] - 2021-02-09
+### Fixed
+- Fix unintended connection switching while clearing query cache in Rails 6.0. (https://github.com/zendesk/active_record_host_pool/pull/61)
+
+## [1.0.1] - 2020-03-30
+### Fixed
+- Fix connection leakage when calling `release_connection` on pre-Rails 5 applications. (https://github.com/zendesk/active_record_host_pool/pull/58)
+
+## [1.0.0] - 2020-02-25
+### Added
+- Support for Rails 6.0.x. (https://github.com/zendesk/active_record_host_pool/pull/53)
+
+### Changed
+- This gem now adheres to semantic versioning.
+
+## [0.13.0] - 2019-08-26
+### Added
+- Support for Rails 5.2.3. (https://github.com/zendesk/active_record_host_pool/pull/48)
+
+### Removed
+- Removed testing with EOL Ruby 2.3 (https://github.com/zendesk/active_record_host_pool/pull/49)
+
+## [0.12.0] - 2019-08-21
+### Added
+- Start testing with Ruby 2.5
+- Update Gem ownership (https://github.com/zendesk/active_record_host_pool/pull/38)
+
+### Removed
+- Removed compatibility with Rails 3.2 and lower.
+- Removed compatibility with Rails 5.0.
+- Stop testing with Ruby 2.2.
+
+## [0.11.0] - 2018-04-24
+### Added
+- Compatibility with Rails 5.1 (https://github.com/zendesk/active_record_host_pool/pull/31)
+- Compatibility with Rails 5.2 (https://github.com/zendesk/active_record_host_pool/pull/32), (https://github.com/zendesk/active_record_host_pool/pull/34)
+
+### Removed
+- Removed support for the mysql gem, and only support mysql2 (https://github.com/zendesk/active_record_host_pool/pull/35)
+
+## <= [0.10.1]
+
+Unwritten

data/Readme.md

@@ -43,7 +43,7 @@ test_pool_2 => 127.0.0.1/3306//root/false
 
 ## Support
 
-For now, the only backend known to work is MySQL, with the mysql2 gem.
+For now, the only backend known to work is MySQL, with the mysql2 or activerecord-trilogy-adapter gem. When using the activerecord-trilogy-adapter ensure that the transitive dependency Trilogy is v2.5.0+.
 Postgres, from an informal reading of the docs, will never support the concept of one server connection sharing multiple dbs.
 
 ## Installation
@@ -62,11 +62,11 @@ You need a local user called 'john-doe'.
 
 With mysql running locally, run
 
-    BUNDLE_GEMFILE=gemfiles/rails5.2.gemfile bundle exec rake test
+    BUNDLE_GEMFILE=gemfiles/rails6.1.gemfile bundle exec rake test
 
  Or
 
-    BUNDLE_GEMFILE=gemfiles/rails5.2.gemfile ruby test/test_arhp.rb --seed 19911 --verbose
+    BUNDLE_GEMFILE=gemfiles/rails6.1.gemfile ruby test/test_arhp.rb --seed 19911 --verbose
 
 ## Copyright
 

data/lib/active_record_host_pool/clear_query_cache_patch.rb

@@ -17,30 +17,29 @@
 # This is a private Rails API and may change in future releases as they're
 # actively working on sharding in Rails 6 and above.
 module ActiveRecordHostPool
-  # For Rails 6.1 & 7.0.
-  module ClearOnHandlerPatch
-    def clear_on_handler(handler)
-      handler.all_connection_pools.each do |pool|
+  # For Rails 7.1.
+  module ClearQueryCachePatch
+    def clear_query_caches_for_current_thread
+      connection_handler.each_connection_pool do |pool|
         pool._unproxied_connection.clear_query_cache if pool.active_connection?
       end
     end
   end
 
-  # For Rails 6.0.
-  module ClearQueryCachePatch
-    def clear_query_caches_for_current_thread
-      ActiveRecord::Base.connection_handlers.each_value do |handler|
-        handler.connection_pool_list.each do |pool|
-          pool._unproxied_connection.clear_query_cache if pool.active_connection?
-        end
+  # For Rails 6.1 & 7.0.
+  module ClearOnHandlerPatch
+    def clear_on_handler(handler)
+      handler.all_connection_pools.each do |pool|
+        pool._unproxied_connection.clear_query_cache if pool.active_connection?
       end
     end
   end
 end
 
 case "#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}"
-when '6.1', '7.0'
-  ActiveRecord::Base.singleton_class.prepend(ActiveRecordHostPool::ClearOnHandlerPatch)
-when '6.0'
+when "7.1"
+  # Fix https://github.com/rails/rails/commit/401e2f24161ed6047ae33c322aaf6584b7728ab9
   ActiveRecord::Base.singleton_class.prepend(ActiveRecordHostPool::ClearQueryCachePatch)
+when "6.1", "7.0"
+  ActiveRecord::Base.singleton_class.prepend(ActiveRecordHostPool::ClearOnHandlerPatch)
 end

data/lib/active_record_host_pool/connection_adapter_mixin.rb

@@ -1,6 +1,20 @@
 # frozen_string_literal: true
 
-require "active_record/connection_adapters/mysql2_adapter"
+case ActiveRecordHostPool.loaded_db_adapter
+when :mysql2
+  require "active_record/connection_adapters/mysql2_adapter"
+when :trilogy
+  case "#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}"
+  when "6.1", "7.0"
+    require "trilogy_adapter/connection"
+    require "trilogy_adapter/errors"
+    ActiveRecord::Base.extend(TrilogyAdapter::Connection)
+  when "7.1"
+    require "active_record/connection_adapters/trilogy_adapter"
+  else
+    raise "Unsupported version of Rails (v#{ActiveRecord::VERSION::STRING})"
+  end
+end
 
 module ActiveRecordHostPool
   module DatabaseSwitch
@@ -8,8 +22,16 @@ module ActiveRecordHostPool
       base.class_eval do
         attr_reader(:_host_pool_current_database)
 
-        alias_method :execute_without_switching, :execute
-        alias_method :execute, :execute_with_switching
+        # Patch `raw_execute` instead of `execute` since this commit:
+        # https://github.com/rails/rails/commit/f69bbcbc0752ca5d5af327d55922614a26f5c7e9
+        case "#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}"
+        when "7.1"
+          alias_method :raw_execute_without_switching, :raw_execute
+          alias_method :raw_execute, :raw_execute_with_switching
+        else
+          alias_method :execute_without_switching, :execute
+          alias_method :execute, :execute_with_switching
+        end
 
         alias_method :drop_database_without_no_switching, :drop_database
         alias_method :drop_database, :drop_database_with_no_switching
@@ -34,13 +56,22 @@ module ActiveRecordHostPool
 
     def self.ruby2_keywords(*); end unless respond_to?(:ruby2_keywords, true)
     # This one really does need ruby2_keywords; in Rails 6.0 the method does not take
-    # any keyword arguments, but in Rails 7.0 it does. So, we don't know whether or not
+    # any keyword arguments, but in Rails 7.0+ it does. So, we don't know whether or not
     # what we're delegating to takes kwargs, so ruby2_keywords is needed.
-    ruby2_keywords def execute_with_switching(*args)
-      if _host_pool_current_database && !_no_switch
-        _switch_connection
+    if ActiveRecord.version >= Gem::Version.new("7.1")
+      ruby2_keywords def raw_execute_with_switching(*args)
+        if _host_pool_current_database && !_no_switch
+          _switch_connection
+        end
+        raw_execute_without_switching(*args)
+      end
+    else
+      ruby2_keywords def execute_with_switching(*args)
+        if _host_pool_current_database && !_no_switch
+          _switch_connection
+        end
+        execute_without_switching(*args)
       end
-      execute_without_switching(*args)
     end
 
     def drop_database_with_no_switching(*args)
@@ -69,12 +100,12 @@ module ActiveRecordHostPool
 
     def _switch_connection
       if _host_pool_current_database &&
-         (
-           (_host_pool_current_database != @_cached_current_database) ||
-           @connection.object_id != @_cached_connection_object_id
-         )
+          (
+            (_host_pool_current_database != @_cached_current_database) ||
+            @connection.object_id != @_cached_connection_object_id
+          )
         log("select_db #{_host_pool_current_database}", "SQL") do
-          clear_cache! if respond_to?(:clear_cache!)
+          clear_cache!
           raw_connection.select_db(_host_pool_current_database)
         end
         @_cached_current_database = _host_pool_current_database
@@ -97,36 +128,11 @@ module ActiveRecordHostPool
   end
 end
 
-module ActiveRecord
-  module ConnectionAdapters
-    class ConnectionHandler
-      case "#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}"
-      when '6.1', '7.0'
-
-        :noop
-
-      when '5.1', '5.2', '6.0'
-
-        def establish_connection(spec)
-          resolver = ConnectionAdapters::ConnectionSpecification::Resolver.new(Base.configurations)
-          spec = resolver.spec(spec)
-
-          owner_to_pool[spec.name] = ActiveRecordHostPool::PoolProxy.new(spec)
-        end
-
-      else
-
-        raise "Unsupported version of Rails (v#{ActiveRecord::VERSION::STRING})"
-      end
-    end
-  end
+case ActiveRecordHostPool.loaded_db_adapter
+when :mysql2
+  ActiveRecord::ConnectionAdapters::Mysql2Adapter.include(ActiveRecordHostPool::DatabaseSwitch)
+when :trilogy
+  ActiveRecord::ConnectionAdapters::TrilogyAdapter.include(ActiveRecordHostPool::DatabaseSwitch)
 end
 
-ActiveRecord::ConnectionAdapters::Mysql2Adapter.include(ActiveRecordHostPool::DatabaseSwitch)
-
-# In Rails 6.1 Connection Pools are no longer instantiated in #establish_connection but in a
-# new pool method.
-case "#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}"
-when '6.1', '7.0'
-  ActiveRecord::ConnectionAdapters::PoolConfig.prepend(ActiveRecordHostPool::PoolConfigPatch)
-end
+ActiveRecord::ConnectionAdapters::PoolConfig.prepend(ActiveRecordHostPool::PoolConfigPatch)

data/lib/active_record_host_pool/connection_proxy.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
-require 'delegate'
+require "delegate"
 
 # the ConnectionProxy sits between user-code and a real connection and says "I expect to be on this database"
 # for each call to the connection.  upon executing a statement, the connection will switch to that database.
@@ -50,11 +50,13 @@ module ActiveRecordHostPool
         __getobj__.send(symbol, *args, &blk)
       end
     end
+    ruby2_keywords :send if respond_to?(:ruby2_keywords, true)
 
     private
 
     def select(*args)
       @cx.__send__(:select, *args)
     end
+    ruby2_keywords :select if respond_to?(:ruby2_keywords, true)
   end
 end

data/lib/active_record_host_pool/pool_proxy.rb

@@ -1,8 +1,185 @@
 # frozen_string_literal: true
 
-case "#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}"
-when '6.1', '7.0'
-  require 'active_record_host_pool/pool_proxy_6_1'
-else
-  require 'active_record_host_pool/pool_proxy_legacy'
+require "delegate"
+require "active_record"
+require "active_record_host_pool/connection_adapter_mixin"
+require "mutex_m"
+
+# this module sits in between ConnectionHandler and a bunch of different ConnectionPools (one per host).
+# when a connection is requested, it goes like:
+# ActiveRecordClass -> ConnectionHandler#connection
+# ConnectionHandler#connection -> (find or create PoolProxy)
+# PoolProxy -> shared list of Pools
+# Pool actually gives back a connection, then PoolProxy turns this
+# into a ConnectionProxy that can inform (on execute) which db we should be on.
+
+module ActiveRecordHostPool
+  # Sits between ConnectionHandler and a bunch of different ConnectionPools (one per host).
+  class PoolProxy < Delegator
+    include Mutex_m
+
+    case ActiveRecordHostPool.loaded_db_adapter
+    when :mysql2
+      RESCUABLE_DB_ERROR = Mysql2::Error
+    when :trilogy
+      RESCUABLE_DB_ERROR = Trilogy::ProtocolError
+    end
+
+    def initialize(pool_config)
+      super(pool_config)
+      @pool_config = pool_config
+      @config = pool_config.db_config.configuration_hash
+    end
+
+    def __getobj__
+      _connection_pool
+    end
+
+    def __setobj__(pool_config)
+      @pool_config = pool_config
+      @config = pool_config.db_config.configuration_hash
+      @_pool_key = nil
+    end
+
+    attr_reader :pool_config
+
+    def connection(*args)
+      real_connection = _unproxied_connection(*args)
+      _connection_proxy_for(real_connection, @config[:database])
+    rescue RESCUABLE_DB_ERROR, ActiveRecord::NoDatabaseError, ActiveRecord::StatementInvalid
+      _connection_pools.delete(_pool_key)
+      Kernel.raise
+    end
+
+    def _unproxied_connection(*args)
+      _connection_pool.connection(*args)
+    end
+
+    # by the time we are patched into ActiveRecord, the current thread has already established
+    # a connection.  thus we need to patch both connection and checkout/checkin
+    def checkout(*args, &block)
+      cx = _connection_pool.checkout(*args, &block)
+      _connection_proxy_for(cx, @config[:database])
+    end
+
+    def checkin(cx)
+      cx = cx.unproxied
+      _connection_pool.checkin(cx)
+    end
+
+    def with_connection
+      cx = checkout
+      yield cx
+    ensure
+      checkin cx
+    end
+
+    def disconnect!
+      p = _connection_pool(false)
+      return unless p
+
+      synchronize do
+        p.disconnect!
+        p.automatic_reconnect = true
+        _clear_connection_proxy_cache
+      end
+    end
+
+    def automatic_reconnect=(value)
+      p = _connection_pool(false)
+      return unless p
+
+      p.automatic_reconnect = value
+    end
+
+    def clear_reloadable_connections!
+      _connection_pool.clear_reloadable_connections!
+      _clear_connection_proxy_cache
+    end
+
+    def release_connection(*args)
+      p = _connection_pool(false)
+      return unless p
+
+      p.release_connection(*args)
+    end
+
+    def flush!
+      p = _connection_pool(false)
+      return unless p
+
+      p.flush!
+    end
+
+    def discard!
+      p = _connection_pool(false)
+      return unless p
+
+      p.discard!
+
+      # All connections in the pool (even if they're currently
+      # leased!) have just been discarded, along with the pool itself.
+      # Any further interaction with the pool (except #pool_config and #schema_cache)
+      # is undefined.
+      # Remove the connection for the given key so a new one can be created in its place
+      _connection_pools.delete(_pool_key)
+    end
+
+    private
+
+    def _connection_pools
+      @@_connection_pools ||= {}
+    end
+
+    def _pool_key
+      @_pool_key ||= "#{@config[:host]}/#{@config[:port]}/#{@config[:socket]}/" \
+                     "#{@config[:username]}/#{replica_configuration? && "replica"}"
+    end
+
+    def _connection_pool(auto_create = true)
+      pool = _connection_pools[_pool_key]
+      if pool.nil? && auto_create
+        pool = _connection_pools[_pool_key] = ActiveRecord::ConnectionAdapters::ConnectionPool.new(@pool_config)
+      end
+      pool
+    end
+
+    # Work around https://github.com/rails/rails/pull/48061/commits/63c0d6b31bcd0fc33745ec6fd278b2d1aab9be54
+    # standard:disable Lint/DuplicateMethods
+    case "#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}"
+    when "6.1", "7.0"
+      def _connection_proxy_for(connection, database)
+        @connection_proxy_cache ||= {}
+        key = [connection, database]
+
+        @connection_proxy_cache[key] ||= begin
+          cx = ActiveRecordHostPool::ConnectionProxy.new(connection, database)
+          cx.execute("SELECT 1")
+
+          cx
+        end
+      end
+    else
+      def _connection_proxy_for(connection, database)
+        @connection_proxy_cache ||= {}
+        key = [connection, database]
+
+        @connection_proxy_cache[key] ||= begin
+          cx = ActiveRecordHostPool::ConnectionProxy.new(connection, database)
+          cx.raw_execute("SELECT 1", "ARHP SWITCH DB")
+
+          cx
+        end
+      end
+    end
+    # standard:enable Lint/DuplicateMethods
+
+    def _clear_connection_proxy_cache
+      @connection_proxy_cache = {}
+    end
+
+    def replica_configuration?
+      @config[:replica] || @config[:slave]
+    end
+  end
 end

data/lib/active_record_host_pool/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module ActiveRecordHostPool
-  VERSION = "1.2.4"
+  VERSION = "2.0.0.pre.2"
 end

data/lib/active_record_host_pool.rb

@@ -1,14 +1,26 @@
 # frozen_string_literal: true
 
-require 'active_record'
-require 'active_record/base'
-require 'active_record/connection_adapters/abstract_adapter'
-
-require 'active_record_host_pool/clear_query_cache_patch'
-require 'active_record_host_pool/connection_proxy'
-require 'active_record_host_pool/pool_proxy'
-require 'active_record_host_pool/connection_adapter_mixin'
-require 'active_record_host_pool/version'
+require "active_record"
+require "active_record/base"
+require "active_record/connection_adapters/abstract_adapter"
 
 module ActiveRecordHostPool
+  class << self
+    attr_accessor :loaded_db_adapter
+  end
+end
+
+if Gem.loaded_specs.include?("mysql2")
+  require "mysql2"
+  ActiveRecordHostPool.loaded_db_adapter = :mysql2
+elsif Gem.loaded_specs.include?("trilogy")
+  require "trilogy"
+  require "activerecord-trilogy-adapter" if ActiveRecord.version < Gem::Version.new("7.1")
+  ActiveRecordHostPool.loaded_db_adapter = :trilogy
 end
+
+require "active_record_host_pool/clear_query_cache_patch"
+require "active_record_host_pool/connection_proxy"
+require "active_record_host_pool/pool_proxy"
+require "active_record_host_pool/connection_adapter_mixin"
+require "active_record_host_pool/version"