db-charmer 1.6.19 → 1.7.0.pre1

data/CHANGES

@@ -1,3 +1,15 @@
+1.7.0 (not released yet):
+
+Beta feature: Added force_slave_reads functionality. Now we could have models with slaves
+that are not used by default, but could be turned on globally (per-controller or per-action).
+
+Heavily reorganized the source code to match Rails code structure (class names, etc). This should
+make it much easier for other contributors to work with the code.
+Added smarter environment detection (using Rails.env, RAILS_ENV or RACK_ENV).
+
+Bugfixes: Fix for N+1 queries when accessing shard_info for db_block_group_map sharding method.
+
+----------------------------------------------------------------------------------------
 1.6.17-19 (2011-04-25):
 
 Bugfixes: Do not touch database for sharded models until we really need to. Before 1.6.17
@@ -13,10 +25,10 @@ being used with db-charmer gem.
 ----------------------------------------------------------------------------------------
 1.6.13 (2010-08-17):
 
-Starting with this version we use Rails.env instead of RAILS_ENV to auto-detect rails 
+Starting with this version we use Rails.env instead of RAILS_ENV to auto-detect rails
 environment. If you use DbCharmer in non-rails project, please set DbCharmer.env manually.
 
-Bugfixes: Thanks to Eric Lindvall we now allow connection names that have symbols ruby 
+Bugfixes: Thanks to Eric Lindvall we now allow connection names that have symbols ruby
 wouldn't like for class names.
 
 ----------------------------------------------------------------------------------------

data/README.rdoc

@@ -28,7 +28,7 @@ To install db-charmer as a Rails plugin use this:
   script/plugin install git://github.com/kovyrin/db-charmer.git
 
 _Notice_: If you use db-charmer in a non-rails project, you may need to set <tt>DbCharmer.env</tt> to a correct value
-before using any of its connection management methods. Correct value here is a valid <tt>database.yml</tt> 
+before using any of its connection management methods. Correct value here is a valid <tt>database.yml</tt>
 first-level section name.
 
 
@@ -52,6 +52,21 @@ Sample code:
   Foo.switch_connection_to(Baz.connection)
   Foo.switch_connection_to(nil)
 
+Sample <tt>database.yml</tt> configuration:
+
+  production:
+    blah:
+      adapter: mysql
+      username: blah
+      host: blah.local
+      database: blah
+
+    foo:
+      adapter: mysql
+      username: foo
+      host: foo.local
+      database: foo
+
 The +switch_connection_to+ method has an optional second parameter +should_exist+ which is true
 by default. This parameter is used when the method is called with a string or a symbol connection
 name and there is no such connection configuration in the database.yml file. If this parameter
@@ -243,6 +258,65 @@ This behaviour is controlled by the <tt>DbCharmer.connections_should_exist</tt>
 configuration attribute which could be set from a rails initializer.
 
 
+=== Forced Slave Reads
+
+In some cases we could have models that are too important to be used in default "send all
+reads to the slave" mode, but we still would like to be able to switch them to this mode
+sometimes. For example, you could have +User+ model, which you would like to keep from
+lagging with your slaves because users do not like to see outdated information about their
+accounts. But in some cases (like logged-out profile page views, etc) it would be perfectly
+reasonable to switch all reads to the slave.
+
+For this use-case starting with +DbCharmer+ release 1.7.0 we have a feature called forced
+slave reads. It consists of a few separate small features that together make it really
+powerful:
+
+1) <tt>:force_slave_reads => false</tt> option for +ActiveRecord+'s <tt>db_magic</tt> method.
+This option could be used to disable automated slave reads on your models so that you could
+call <tt>on_slave</tt> or use other methods to enable slave reads when you need it. Example:
+
+  class User < ActiveRecord::Base
+    db_magic :slave => slave01, :forced_slave_reads => false
+  end
+
+2) <tt>force_slave_reads</tt> +ActionController+ class method. This method could be used to
+enable per-controller (when called with no arguments), or per-action (<tt>:only</tt> and
+<tt>:except</tt> params) forced reads from slaves. This is really useful for actions in
+which you know you could tolerate some slave lag so all your models with slaves defined will
+send their reads to slaves. Example:
+
+  class ProfilesController < Application
+    force_slave_reads :except => [ :login, :logout ]
+    ...
+  end
+
+3) <tt>force_slave_reads!</tt> +ActionController+ instance method, that could be used within
+your actions or in controller filters to temporarily switch your models to forced slave reads
+mode. This could be useful for cases when the same actions could be called by logged-in and
+anonymous users. Then you could authorize users in <tt>before_filter</tt> and call
+<tt>force_slave_reads!</tt> method for anonymous page views.
+
+  class ProfilesController < Application
+    before_filter do
+      force_slave_reads! unless current_user
+    end
+    ...
+  end
+
+4) <tt>DbCharmer.force_slave_reads</tt> method that could be used with a block of ruby code
+and would enable forced slave reads mode until the end of the block execution. This is really
+powerful feature allowing high granularity in your control of forced slave reads mode. Example:
+
+  DbCharmer.force_slave_reads do
+    ...
+    total_users = User.count
+    ...
+  end
+
+Notice: At this point the feature considered beta and should be used with caution. It is fully covered
+with tests, but there still could be unexpected issues when used in real-world applications.
+
+
 === Associations Connection Management
 
 ActiveRecord models can have an associations with each other and since every model has its
@@ -492,12 +566,23 @@ most of the parts of plugin's code.
 
 == What Ruby and Rails implementations does it work for?
 
-We have a continuous integration setups for this plugin on MRI 1.8.6 with Rails 2.2 and 2.3.
-We use the plugin in production on Scribd.com with MRI (rubyee) 1.8.6 and Rails 2.2.
+We have a continuous integration setups for this plugin on MRI 1.8.7 with Rails 2.2 and 2.3.
+We use the plugin in production on Scribd.com with MRI (rubyee) 1.8.7 and Rails 2.2, Rails 2.3,
+Sinatra and plain Rack applications.
 
 
 == Who are the authors?
 
 This plugin has been created in Scribd.com for our internal use and then the sources were opened for
-other people to use. All the code in this package has been developed by Alexey Kovyrin for Scribd.com
+other people to use. Most of the code in this package has been developed by Alexey Kovyrin for Scribd.com
 and is released under the MIT license. For more details, see the LICENSE file.
+
+Other contributors who have helped with the development of this library are (alphabetically ordered):
+* Allen Madsen
+* Andrew Geweke
+* Ashley Martens
+* Dmytro Shteflyuk
+* Eric Lindvall
+* Gregory Man
+* Michael Birk
+* Tyler McMullen

data/Rakefile

@@ -1,5 +1,7 @@
 begin
   require 'jeweler'
+  require './lib/db_charmer/version.rb'
+
   Jeweler::Tasks.new do |gemspec|
     gemspec.name = 'db-charmer'
     gemspec.summary = 'ActiveRecord Connections Magic'
@@ -8,6 +10,8 @@ begin
     gemspec.homepage = 'http://github.com/kovyrin/db-charmer'
     gemspec.authors = ['Alexey Kovyrin']
 
+    gemspec.version = DbCharmer::Version::STRING
+
     gemspec.add_dependency('rails', '~> 2.2')
     gemspec.add_dependency('blankslate', '>= 0')
   end

data/db-charmer.gemspec

@@ -5,11 +5,11 @@
 
 Gem::Specification.new do |s|
   s.name = %q{db-charmer}
-  s.version = "1.6.19"
+  s.version = "1.7.0.pre1"
 
-  s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
+  s.required_rubygems_version = Gem::Requirement.new("> 1.3.1") if s.respond_to? :required_rubygems_version=
   s.authors = ["Alexey Kovyrin"]
-  s.date = %q{2011-04-25}
+  s.date = %q{2011-05-16}
   s.description = %q{ActiveRecord Connections Magic (slaves, multiple connections, etc)}
   s.email = %q{alexey@kovyrin.net}
   s.extra_rdoc_files = [
@@ -23,22 +23,24 @@ Gem::Specification.new do |s|
      "Makefile",
      "README.rdoc",
      "Rakefile",
-     "VERSION",
      "db-charmer.gemspec",
      "init.rb",
      "lib/db_charmer.rb",
-     "lib/db_charmer/abstract_adapter_extensions.rb",
-     "lib/db_charmer/active_record_extensions.rb",
-     "lib/db_charmer/association_preload.rb",
+     "lib/db_charmer/abstract_adapter/log_formatting.rb",
+     "lib/db_charmer/action_controller/force_slave_reads.rb",
+     "lib/db_charmer/active_record/association_preload.rb",
+     "lib/db_charmer/active_record/class_attributes.rb",
+     "lib/db_charmer/active_record/connection_switching.rb",
+     "lib/db_charmer/active_record/db_magic.rb",
+     "lib/db_charmer/active_record/finder_overrides.rb",
+     "lib/db_charmer/active_record/migration/multi_db_migrations.rb",
+     "lib/db_charmer/active_record/multi_db_proxy.rb",
+     "lib/db_charmer/active_record/named_scope/scope_proxy.rb",
+     "lib/db_charmer/active_record/sharding.rb",
      "lib/db_charmer/connection_factory.rb",
      "lib/db_charmer/connection_proxy.rb",
-     "lib/db_charmer/connection_switch.rb",
      "lib/db_charmer/core_extensions.rb",
-     "lib/db_charmer/db_magic.rb",
-     "lib/db_charmer/finder_overrides.rb",
-     "lib/db_charmer/multi_db_migrations.rb",
-     "lib/db_charmer/multi_db_proxy.rb",
-     "lib/db_charmer/scope_proxy.rb",
+     "lib/db_charmer/force_slave_reads.rb",
      "lib/db_charmer/sharding.rb",
      "lib/db_charmer/sharding/connection.rb",
      "lib/db_charmer/sharding/method/db_block_group_map.rb",
@@ -46,16 +48,16 @@ Gem::Specification.new do |s|
      "lib/db_charmer/sharding/method/hash_map.rb",
      "lib/db_charmer/sharding/method/range.rb",
      "lib/db_charmer/sharding/stub_connection.rb",
+     "lib/db_charmer/version.rb",
      "lib/tasks/databases.rake"
   ]
   s.homepage = %q{http://github.com/kovyrin/db-charmer}
   s.rdoc_options = ["--charset=UTF-8"]
   s.require_paths = ["lib"]
-  s.rubygems_version = %q{1.3.7}
+  s.rubygems_version = %q{1.6.2}
   s.summary = %q{ActiveRecord Connections Magic}
 
   if s.respond_to? :specification_version then
-    current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
     s.specification_version = 3
 
     if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then

data/lib/db_charmer.rb

@@ -2,7 +2,16 @@ module DbCharmer
   @@connections_should_exist = true
   mattr_accessor :connections_should_exist
 
-  @@env = defined?(Rails) ? Rails.env : 'development'
+  # Try to detect current environment or use development by default
+  if defined?(Rails)
+    @@env = Rails.env
+  elsif ENV['RAILS_ENV']
+    @@env = ENV['RAILS_ENV']
+  elsif ENV['RACK_ENV']
+    @@env = ENV['RACK_ENV']
+  else
+    @@env = 'development'
+  end
   mattr_accessor :env
 
   def self.connections_should_exist?
@@ -15,16 +24,16 @@ module DbCharmer
   end
 
   def self.with_remapped_databases(mappings, &proc)
-    old_mappings = ActiveRecord::Base.db_charmer_database_remappings
+    old_mappings = ::ActiveRecord::Base.db_charmer_database_remappings
     begin
-      ActiveRecord::Base.db_charmer_database_remappings = mappings
+      ::ActiveRecord::Base.db_charmer_database_remappings = mappings
       if mappings[:master] || mappings['master']
         with_all_hijacked(&proc)
       else
         proc.call
       end
     ensure
-      ActiveRecord::Base.db_charmer_database_remappings = old_mappings
+      ::ActiveRecord::Base.db_charmer_database_remappings = old_mappings
     end
   end
 
@@ -39,7 +48,7 @@ private
     old_hijack_new_classes = @@hijack_new_classes
     begin
       @@hijack_new_classes = true
-      ActiveRecord::Base.send(:subclasses).each do |subclass|
+      ::ActiveRecord::Base.send(:subclasses).each do |subclass|
         subclass.hijack_connection!
       end
       yield
@@ -49,45 +58,32 @@ private
   end
 end
 
-# These methods are added to all objects so we could call proxy? on anything
-# and figure if an object is a proxy w/o hitting method_missing or respond_to?
-class Object
-  def self.proxy?
-    false
-  end
-
-  def proxy?
-    false
-  end
-end
-
 # We need blankslate for all the proxies we have
 require 'blankslate'
 
-require 'db_charmer/active_record_extensions'
-require 'db_charmer/abstract_adapter_extensions'
+# Add useful methods to global object
 require 'db_charmer/core_extensions'
 
 require 'db_charmer/connection_factory'
 require 'db_charmer/connection_proxy'
-require 'db_charmer/connection_switch'
-require 'db_charmer/scope_proxy'
-require 'db_charmer/multi_db_proxy'
+require 'db_charmer/force_slave_reads'
 
-# Enable misc AR extensions
-ActiveRecord::Base.extend(DbCharmer::ActiveRecordExtensions::ClassMethods)
-ActiveRecord::ConnectionAdapters::AbstractAdapter.send(:include, DbCharmer::AbstractAdapterExtensions::InstanceMethods)
+# Add our custom class-level attributes to AR models
+ActiveRecord::Base.extend(DbCharmer::ActiveRecord::ClassAttributes)
 
 # Enable connections switching in AR
-ActiveRecord::Base.extend(DbCharmer::ConnectionSwitch::ClassMethods)
+ActiveRecord::Base.extend(DbCharmer::ActiveRecord::ConnectionSwitching)
+
+# Enable misc AR extensions
+ActiveRecord::ConnectionAdapters::AbstractAdapter.send(:include, DbCharmer::AbstractAdapter::LogFormatting)
 
 # Enable connection proxy in AR
-ActiveRecord::Base.extend(DbCharmer::MultiDbProxy::ClassMethods)
-ActiveRecord::Base.extend(DbCharmer::MultiDbProxy::MasterSlaveClassMethods)
-ActiveRecord::Base.send(:include, DbCharmer::MultiDbProxy::InstanceMethods)
+ActiveRecord::Base.extend(DbCharmer::ActiveRecord::MultiDbProxy::ClassMethods)
+ActiveRecord::Base.extend(DbCharmer::ActiveRecord::MultiDbProxy::MasterSlaveClassMethods)
+ActiveRecord::Base.send(:include, DbCharmer::ActiveRecord::MultiDbProxy::InstanceMethods)
 
 # Enable connection proxy for scopes
-ActiveRecord::NamedScope::Scope.send(:include, DbCharmer::ScopeProxy::InstanceMethods)
+ActiveRecord::NamedScope::Scope.send(:include, DbCharmer::ActiveRecord::NamedScope::ScopeProxy)
 
 # Enable connection proxy for associations
 # WARNING: Inject methods to association class right here (they proxy include calls somewhere else, so include does not work)
@@ -114,24 +110,14 @@ module ActiveRecord
   end
 end
 
-require 'db_charmer/db_magic'
-require 'db_charmer/finder_overrides'
-require 'db_charmer/association_preload'
-require 'db_charmer/multi_db_migrations'
-require 'db_charmer/multi_db_proxy'
-
-require 'db_charmer/sharding'
-require 'db_charmer/sharding/connection'
-require 'db_charmer/sharding/stub_connection'
-
 # Enable multi-db migrations
-ActiveRecord::Migration.extend(DbCharmer::MultiDbMigrations)
+ActiveRecord::Migration.extend(DbCharmer::ActiveRecord::Migration::MultiDbMigrations)
 
 # Enable the magic
-ActiveRecord::Base.extend(DbCharmer::DbMagic::ClassMethods)
+ActiveRecord::Base.extend(DbCharmer::ActiveRecord::DbMagic)
 
 # Setup association preload magic
-ActiveRecord::Base.extend(DbCharmer::AssociationPreload::ClassMethods)
+ActiveRecord::Base.extend(DbCharmer::ActiveRecord::AssociationPreload)
 
 # Open up really useful API method
 ActiveRecord::AssociationPreload::ClassMethods.send(:public, :preload_associations)
@@ -147,3 +133,8 @@ class ActiveRecord::Base
     alias_method_chain :inherited, :hijacking
   end
 end
+
+#-----------------------------------------------------------------------------------------------------------------------
+# Extend ActionController to support forcing slave reads
+ActionController::Base.extend(DbCharmer::ActionController::ForceSlaveReads::ClassMethods)
+ActionController::Base.send(:include, DbCharmer::ActionController::ForceSlaveReads::InstanceMethods)

data/lib/db_charmer/{abstract_adapter_extensions.rb → abstract_adapter/log_formatting.rb}

@@ -1,6 +1,7 @@
 module DbCharmer
-  module AbstractAdapterExtensions
-    module InstanceMethods
+  module AbstractAdapter
+    module LogFormatting
+
       def self.included(base)
         base.alias_method_chain :format_log_entry, :connection_name
       end
@@ -12,9 +13,10 @@ module DbCharmer
 
       def format_log_entry_with_connection_name(message, dump = nil)
         msg = connection_name ? "[#{connection_name}] " : ''
-        msg = "  \e[0;34;1m#{msg}\e[0m" if connection_name && ActiveRecord::Base.colorize_logging
+        msg = "  \e[0;34;1m#{msg}\e[0m" if connection_name && ::ActiveRecord::Base.colorize_logging
         msg << format_log_entry_without_connection_name(message, dump)
       end
+
     end
   end
 end

data/lib/db_charmer/action_controller/force_slave_reads.rb

@@ -0,0 +1,65 @@
+module DbCharmer
+  module ActionController
+    module ForceSlaveReads
+
+      module ClassMethods
+        @@db_charmer_force_slave_reads_actions = {}
+        def force_slave_reads(params)
+          @@db_charmer_force_slave_reads_actions[self.name] = {
+            :except => params[:except] ? [*params[:except]].map(&:to_s) : [],
+            :only => params[:only] ? [*params[:only]].map(&:to_s) : []
+          }
+        end
+
+        def force_slave_reads_options
+          @@db_charmer_force_slave_reads_actions[self.name]
+        end
+
+        def force_slave_reads_action?(name = nil)
+          name = name.to_s
+
+          options = force_slave_reads_options
+          # If no options were defined for this controller, all actions are not forced to use slaves
+          return false unless options
+
+          # Actions where force_slave_reads mode was turned off
+          return false if options[:except].include?(name)
+
+          # Only for these actions force_slave_reads was turned on
+          return options[:only].include?(name) if options[:only].any?
+
+          # If :except is not empty, we're done with the checks and rest of the actions are should force slave reads
+          # Otherwise, all the actions are not in force_slave_reads mode
+          options[:except].any?
+        end
+      end
+
+      module InstanceMethods
+        def self.included(base)
+          base.alias_method_chain :perform_action, :forced_slave_reads
+        end
+
+        def force_slave_reads!
+          @db_charmer_force_slave_reads = true
+        end
+
+        def dont_force_slave_reads!
+          @db_charmer_force_slave_reads = false
+        end
+
+        def force_slave_reads?
+          @db_charmer_force_slave_reads || self.class.force_slave_reads_action?(params[:action])
+        end
+
+      protected
+
+        def perform_action_with_forced_slave_reads
+          DbCharmer.with_controller(self) do
+            perform_action_without_forced_slave_reads
+          end
+        end
+      end
+
+    end
+  end
+end