activerecord-dbslayer-adapter 0.2.0

data/History.txt

@@ -0,0 +1,10 @@
+== 0.0.2 2008-05-01
+
+* 2 major enhancments:
+  * tests
+  * it actually works!
+
+== 0.0.1 2008-04-15
+
+* 1 major enhancement:
+  * Initial release

data/License.txt

@@ -0,0 +1,20 @@
+Copyright (c) 2008 FIXME full name
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Manifest.txt

@@ -0,0 +1,30 @@
+History.txt
+License.txt
+Manifest.txt
+README.txt
+Rakefile
+config/hoe.rb
+config/requirements.rb
+lib/active_record/connection_adapters/dbslayer_adapter.rb
+lib/active_record/connection_adapters/dbslayer_connection.rb
+lib/activerecord-dbslayer-adapter.rb
+log/debug.log
+script/console
+script/destroy
+script/generate
+script/txt2html
+setup.rb
+tasks/deployment.rake
+tasks/environment.rake
+tasks/website.rake
+test/helper.rb
+test/localtest.rb
+test/test_dbslayer_adapter.rb
+test/test_dbslayer_column.rb
+test/test_dbslayer_connection.rb
+test/test_dbslayer_results.rb
+website/index.html
+website/index.txt
+website/javascripts/rounded_corners_lite.inc.js
+website/stylesheets/screen.css
+website/template.html.erb

data/README.txt

@@ -0,0 +1,80 @@
+= activerecord_dbslayer_adapter
+
+Jacob Harris
+jharris@nytimes.com
+
+== DESCRIPTION:
+
+An ActiveRecord adapter for using the DBSlayer proxy/pooling layer. This allows you to proxy and pool connections to the MySQL backend without worrying about scaling up the front instances more than MySQL is configured to handle (the dreaded Too Many Connections error). Of course, you can also reconfigure master/slave instances on the fly in DBSlayer without having to reconfigure or restart your Rails applications. 
+
+This adapter is really just a judicious subclassing of functionality in the MySQL adapter, so the documented methods might seem very sparse (I only have to override what's changed). Mainly, I swapped it out to execute queries as JSON-over-HTTP calls to DBSlayer rather than using the MySQL connection library. 
+
+== MAJOR CAVEATS:
+
+DBSlayer is a stateless pooling layer. This allows it to easily scale (why do you think HTTP is stateless?), but there are certain database techniques that will have to be abandoned or modified if you use it. The main problem is that each MySQL statement may execute on a different connection (and sometimes on different servers if you've set up transparent slave pooling), so you can not assume the context of one statement is available to the next.
+
+This becomes a problem with transactions. Although DBSlayer can support transactions if you use semicolons to include them all as one statement like
+  BEGIN TRANSACTION; more SQL; COMMIT TRANSACTION
+There is no readily apparent way to do that via ActiveRecord's block construction (all the adapter has are begin_transaction, commit_transaction methods). So for the moment, transactions do not throw errors but they don't actually use SQL transactions either. Sorry about that. In addition, disabling referential integrity for a connection is not allowed.
+
+The biggest problem is that Rails sets a connection variable for all MySQL connections in order to fix an error selecting null IDs (http://dev.rubyonrails.org/ticket/6778). Since DBSlayer is stateless, this fix doesn't work, but there unfortunately also doesn't seem to be a database or server setting you can alter instead. As a result, the following types of queries WILL return unexpected results when using the DBSlayer adapter:
+
+	Restaurant.find(:all, :conditions => 'id IS NULL')
+	
+The problem only occurs when searching for null autoincrement primary key columns (not other columns). With the regular MySQL adapter, this would return records with an id value (normally errors). With the DBSlayer adapter, it returns the last inserted item into the table (this is the default MySQL behavior). Luckily, this is not a common idiom in Rails, but you should be aware if you attempt to use it for finding errors in your tables.
+
+== FEATURES/PROBLEMS:
+
+* More tests
+* Better documentation
+
+== SYNOPSIS:
+
+In your databases.yml
+
+  production:
+    adapter: dbslayer
+    host: localhost
+    port: 9090
+
+All of the other normal database setting like the MySQL server, username, password, etc. are specified in the dbslayer daemon's configuration. 
+
+== REQUIREMENTS:
+
+* the JSON gem
+* A running DBSlayer instance
+
+== INSTALL:
+
+* gem install activerecord-dbslayer-adapter
+* in your rails project, add the following line at the end of config/environment.rb
+  Rails::Initializer.run do |config|
+    ...
+  
+    require 'activerecord-dbslayer-adapter'
+ end
+
+== LICENSE:
+
+(The MIT License)
+
+Copyright (c) 2008
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Rakefile

@@ -0,0 +1,4 @@
+require 'config/requirements'
+require 'config/hoe' # setup Hoe + all gem configuration
+
+Dir['tasks/**/*.rake'].each { |rake| load rake }

data/config/hoe.rb

@@ -0,0 +1,70 @@
+require 'activerecord-dbslayer-adapter'
+
+AUTHOR = 'Jacob Harris'  # can also be an array of Authors
+EMAIL = "jharris@nytimes.com"
+DESCRIPTION = "An ActiveRecord adapter to DBSlayer"
+GEM_NAME = 'activerecord-dbslayer-adapter' # what ppl will type to install your gem
+RUBYFORGE_PROJECT = 'ar-dbslayer' # The unix name for your project
+HOMEPATH = "http://#{RUBYFORGE_PROJECT}.rubyforge.org"
+DOWNLOAD_PATH = "http://rubyforge.org/projects/#{RUBYFORGE_PROJECT}"
+
+@config_file = "~/.rubyforge/user-config.yml"
+@config = nil
+RUBYFORGE_USERNAME = "unknown"
+def rubyforge_username
+  unless @config
+    begin
+      @config = YAML.load(File.read(File.expand_path(@config_file)))
+    rescue
+      puts <<-EOS
+ERROR: No rubyforge config file found: #{@config_file}
+Run 'rubyforge setup' to prepare your env for access to Rubyforge
+ - See http://newgem.rubyforge.org/rubyforge.html for more details
+      EOS
+      exit
+    end
+  end
+  RUBYFORGE_USERNAME.replace @config["username"]
+end
+
+
+REV = nil 
+# UNCOMMENT IF REQUIRED: 
+# REV = YAML.load(`svn info`)['Revision']
+VERS = ActiveRecord::ConnectionAdapters::DbslayerAdapter::VERSION + (REV ? ".#{REV}" : "")
+RDOC_OPTS = ['--quiet', '--title', 'activerecord-dbslayer-adapter documentation',
+    "--opname", "index.html",
+    "--line-numbers", 
+    "--main", "README",
+    "--inline-source"]
+
+class Hoe
+  def extra_deps 
+    @extra_deps.reject! { |x| Array(x).first == 'hoe' } 
+    @extra_deps
+  end 
+end
+
+# Generate all the Rake tasks
+# Run 'rake -T' to see list of generated tasks (from gem root directory)
+$hoe = Hoe.new(GEM_NAME, VERS) do |p|
+  p.developer(AUTHOR, EMAIL)
+  p.description = DESCRIPTION
+  p.summary = DESCRIPTION
+  p.url = HOMEPATH
+  p.rubyforge_name = RUBYFORGE_PROJECT if RUBYFORGE_PROJECT
+  p.test_globs = ["test/**/test_*.rb"]
+  p.clean_globs |= ['**/.*.sw?', '*.gem', '.config', '**/.DS_Store']  #An array of file patterns to delete on clean.
+  
+  # == Optional
+  p.changes = p.paragraphs_of("History.txt", 0..1).join("\n\n")
+  #p.extra_deps = []     # An array of rubygem dependencies [name, version], e.g. [ ['active_support', '>= 1.3.1'] ]
+  
+  #p.spec_extras = {}    # A hash of extra values to set in the gemspec.
+  
+end
+
+CHANGES = $hoe.paragraphs_of('History.txt', 0..1).join("\\n\\n")
+PATH    = (RUBYFORGE_PROJECT == GEM_NAME) ? RUBYFORGE_PROJECT : "#{RUBYFORGE_PROJECT}/#{GEM_NAME}"
+$hoe.remote_rdoc_dir = File.join(PATH.gsub(/^#{RUBYFORGE_PROJECT}\/?/,''), 'rdoc')
+$hoe.rsync_args = '-av --delete --ignore-errors'

data/config/requirements.rb

@@ -0,0 +1,15 @@
+require 'fileutils'
+include FileUtils
+
+require 'rubygems'
+%w[rake hoe newgem rubigen mocha active_support active_record].each do |req_gem|
+  begin
+    require req_gem
+  rescue LoadError
+    puts "This Rakefile requires the '#{req_gem}' RubyGem."
+    puts "Installation: gem install #{req_gem} -y"
+    exit
+  end
+end
+
+$:.unshift(File.join(File.dirname(__FILE__), %w[.. lib]))

data/lib/active_record/connection_adapters/dbslayer_adapter.rb

@@ -0,0 +1,356 @@
+require 'active_record/connection_adapters/abstract_adapter'
+require 'active_record/connection_adapters/dbslayer_connection'
+require 'active_record/connection_adapters/mysql_adapter'
+
+module ActiveRecord
+  class Base
+    # Establishes a connection to the database that's used by all Active Record objects.
+    def self.dbslayer_connection(config) # :nodoc:
+      config = config.symbolize_keys
+      host   = config[:host]
+      port   = config[:port]
+      
+      connection = ConnectionAdapters::DbslayerConnection.new(host, port)
+      ConnectionAdapters::DbslayerAdapter.new(connection, logger, [host, port], config)
+    end
+  end
+
+  module ConnectionAdapters
+    ##
+    # This is just a basic inheritance of MysqlColumn
+    class DbslayerColumn < MysqlColumn #:nodoc:
+      private
+        def simplified_type(field_type) #:nodoc:
+          return :boolean if DbslayerAdapter.emulate_booleans && field_type.downcase.index("tinyint(1)")
+          return :string  if field_type =~ /enum/i
+          super
+        end
+    end
+
+    # The DbslayerAdapter is an adapter to use Rails with the DBSlayer
+    #
+    # Options:
+    #
+    # * <tt>:host</tt> -- Defaults to localhost
+    # * <tt>:port</tt> -- Defaults to 3306
+    #
+    # Like the MySQL adapter: by default, the MysqlAdapter will consider all columns of type tinyint(1)
+    # as boolean. If you wish to disable this emulation (which was the default
+    # behavior in versions 0.13.1 and earlier) you can add the following line
+    # to your environment.rb file:
+    #
+    #   ActiveRecord::ConnectionAdapters::DbslayerAdapter.emulate_booleans = false
+    #
+    # MAJOR WARNING: The MySQL adapter in Rails sets the 
+    class DbslayerAdapter < MysqlAdapter
+      def initialize(connection, logger, connection_options, config)
+        super(connection, logger, connection_options, config)
+        ActiveRecord::Base.allow_concurrency = true
+      end
+
+      def adapter_name #:nodoc:
+        'DBSlayer (MySQL)'
+      end
+
+      # def supports_migrations? #:nodoc:
+      #   true
+      # end
+      # 
+      # def native_database_types #:nodoc:
+      #   {
+      #     :primary_key => "int(11) DEFAULT NULL auto_increment PRIMARY KEY",
+      #     :string      => { :name => "varchar", :limit => 255 },
+      #     :text        => { :name => "text" },
+      #     :integer     => { :name => "int", :limit => 11 },
+      #     :float       => { :name => "float" },
+      #     :decimal     => { :name => "decimal" },
+      #     :datetime    => { :name => "datetime" },
+      #     :timestamp   => { :name => "datetime" },
+      #     :time        => { :name => "time" },
+      #     :date        => { :name => "date" },
+      #     :binary      => { :name => "blob" },
+      #     :boolean     => { :name => "tinyint", :limit => 1 }
+      #   }
+      # end
+      # 
+      # 
+      # # QUOTING ==================================================
+      # 
+      # def quote(value, column = nil)
+      #   if value.kind_of?(String) && column && column.type == :binary && column.class.respond_to?(:string_to_binary)
+      #     s = column.class.string_to_binary(value).unpack("H*")[0]
+      #     "x'#{s}'"
+      #   elsif value.kind_of?(BigDecimal)
+      #     "'#{value.to_s("F")}'"
+      #   else
+      #     super
+      #   end
+      # end
+      # 
+      # def quote_column_name(name) #:nodoc:
+      #   "`#{name}`"
+      # end
+      # 
+      # def quote_table_name(name) #:nodoc:
+      #   quote_column_name(name).gsub('.', '`.`')
+      # end
+      # 
+      # def quote_string(string) #:nodoc:
+      #   @connection.quote(string)
+      # end
+      # 
+      # def quoted_true
+      #   "1"
+      # end
+      # 
+      # def quoted_false
+      #   "0"
+      # end
+
+      # REFERENTIAL INTEGRITY ====================================
+
+      def disable_referential_integrity(&block) #:nodoc:
+        #FIXME: I CAN'T LET YOU DO THIS
+        # old = select_value("SELECT @@FOREIGN_KEY_CHECKS")
+        # 
+        # begin
+        #   update("SET FOREIGN_KEY_CHECKS = 0")
+        #   yield
+        # ensure
+        #   update("SET FOREIGN_KEY_CHECKS = #{old}")
+        # end
+      end
+
+      # CONNECTION MANAGEMENT ====================================
+
+      def active?
+        stats = @connection.mysql_stats
+        !stats.nil? && !stats.empty?
+      rescue
+        false
+      end
+
+      def reconnect!
+        # DO NOTHING, we connect on the request
+      end
+
+      def disconnect!
+        # DO NOTHING, we connect on the request
+      end
+
+
+      # DATABASE STATEMENTS ======================================
+
+      def select_rows(sql, name = nil)
+        result = execute(sql, name)
+        result.rows
+      end
+
+      def execute(sql, name = nil) #:nodoc:
+        log(sql, name) { 
+          @connection.execute(sql)
+        }
+      end
+
+      # def insert_sql(sql, name = nil, pk = nil, id_value = nil, sequence_name = nil) #:nodoc:
+      #   super sql, name
+      #   id_value || @connection.insert_id
+      # end
+      # 
+      # def update_sql(sql, name = nil) #:nodoc:
+      #   super
+      #   @connection.affected_rows
+      # end
+
+      def begin_db_transaction #:nodoc:
+        # FIXME: raise exception?
+      #   execute "BEGIN"
+      # rescue Exception
+        # Transactions aren't supported
+      end
+
+      def commit_db_transaction #:nodoc:
+        # FIXME: raise exception?
+
+      #   execute "COMMIT"
+      # rescue Exception
+        # Transactions aren't supported
+      end
+
+      def rollback_db_transaction #:nodoc:
+        # FIXME: raise exception?
+
+      #   execute "ROLLBACK"
+      # rescue Exception
+        # Transactions aren't supported
+      end
+
+      # SCHEMA STATEMENTS ========================================
+
+      # def structure_dump #:nodoc:
+      #   if supports_views?
+      #     sql = "SHOW FULL TABLES WHERE Table_type = 'BASE TABLE'"
+      #   else
+      #     sql = "SHOW TABLES"
+      #   end
+      # 
+      #   select_all(sql).inject("") do |structure, table|
+      #     table.delete('Table_type')
+      #     structure += select_one("SHOW CREATE TABLE #{quote_table_name(table.to_a.first.last)}")["Create Table"] + ";\n\n"
+      #   end
+      # end
+      # 
+      # def recreate_database(name) #:nodoc:
+      #   drop_database(name)
+      #   create_database(name)
+      # end
+
+      # Create a new MySQL database with optional :charset and :collation.
+      # Charset defaults to utf8.
+      #
+      # Example:
+      #   create_database 'charset_test', :charset => 'latin1', :collation => 'latin1_bin'
+      #   create_database 'matt_development'
+      #   create_database 'matt_development', :charset => :big5
+      # def create_database(name, options = {})
+      #   if options[:collation]
+      #     execute "CREATE DATABASE `#{name}` DEFAULT CHARACTER SET `#{options[:charset] || 'utf8'}` COLLATE `#{options[:collation]}`"
+      #   else
+      #     execute "CREATE DATABASE `#{name}` DEFAULT CHARACTER SET `#{options[:charset] || 'utf8'}`"
+      #   end
+      # end
+      # 
+      # def drop_database(name) #:nodoc:
+      #   execute "DROP DATABASE IF EXISTS `#{name}`"
+      # end
+      # 
+      # def current_database
+      #   select_value 'SELECT DATABASE() as db'
+      # end
+
+      # Returns the database character set.
+      # def charset
+      #   if @charset.nil?
+      #     @charset = show_variable 'character_set_database'
+      #   end
+      #   
+      #   @charset
+      # end
+      # 
+      # # Returns the database collation strategy.
+      # def collation
+      #   if @collation.nil?
+      #     @collation = show_variable 'collation_database'
+      #   end
+      #   @collation
+      # end
+
+      def tables(name = nil) #:nodoc:
+        tables = []
+        execute("SHOW TABLES", name).rows.each { |row| tables << row[0] }
+        tables
+      end
+
+      # def drop_table(table_name, options = {})
+      #   super(table_name, options)
+      # end
+
+      def indexes(table_name, name = nil)#:nodoc:
+        indexes = []
+        current_index = nil
+        execute("SHOW KEYS FROM #{quote_table_name(table_name)}", name).rows.each do |row|
+          if current_index != row[2]
+            next if row[2] == "PRIMARY" # skip the primary key
+            current_index = row[2]
+            indexes << IndexDefinition.new(row[0], row[2], row[1] == "0", [])
+          end
+
+          indexes.last.columns << row[4]
+        end
+        indexes
+      end
+
+      def columns(table_name, name = nil)#:nodoc:
+        sql = "SHOW FIELDS FROM #{quote_table_name(table_name)}"
+        columns = []
+        execute(sql, name).rows.each { |row| columns << DbslayerColumn.new(row[0], row[4], row[1], row[2] == "YES") }
+        columns
+      end
+
+      # def create_table(table_name, options = {}) #:nodoc:
+      #   super(table_name, options.reverse_merge(:options => "ENGINE=InnoDB"))
+      # end
+      # 
+      # def rename_table(table_name, new_name)
+      #   execute "RENAME TABLE #{quote_table_name(table_name)} TO #{quote_table_name(new_name)}"
+      # end
+      # 
+      # def change_column_default(table_name, column_name, default) #:nodoc:
+      #   current_type = select_one("SHOW COLUMNS FROM #{quote_table_name(table_name)} LIKE '#{column_name}'")["Type"]
+      # 
+      #   execute("ALTER TABLE #{quote_table_name(table_name)} CHANGE #{quote_column_name(column_name)} #{quote_column_name(column_name)} #{current_type} DEFAULT #{quote(default)}")
+      # end
+      # 
+      # def change_column(table_name, column_name, type, options = {}) #:nodoc:
+      #   unless options_include_default?(options)
+      #     if column = columns(table_name).find { |c| c.name == column_name.to_s }
+      #       options[:default] = column.default
+      #     else
+      #       raise "No such column: #{table_name}.#{column_name}"
+      #     end
+      #   end
+      # 
+      #   change_column_sql = "ALTER TABLE #{quote_table_name(table_name)} CHANGE #{quote_column_name(column_name)} #{quote_column_name(column_name)} #{type_to_sql(type, options[:limit], options[:precision], options[:scale])}"
+      #   add_column_options!(change_column_sql, options)
+      #   execute(change_column_sql)
+      # end
+      # 
+      # def rename_column(table_name, column_name, new_column_name) #:nodoc:
+      #   current_type = select_one("SHOW COLUMNS FROM #{quote_table_name(table_name)} LIKE '#{column_name}'")["Type"]
+      #   execute "ALTER TABLE #{quote_table_name(table_name)} CHANGE #{quote_column_name(column_name)} #{quote_column_name(new_column_name)} #{current_type}"
+      # end
+      
+      # SHOW VARIABLES LIKE 'name'
+      def show_variable(name)
+        variables = select_all("SHOW VARIABLES LIKE '#{name}'")
+        variables.first['Value'] unless variables.empty?
+      end
+
+      # Returns a table's primary key and belonging sequence.
+      def pk_and_sequence_for(table) #:nodoc:
+        keys = []
+        execute("describe #{quote_table_name(table)}").each_hash do |h|
+          keys << h["Field"]if h["Key"] == "PRI"
+        end
+        keys.length == 1 ? [keys.first, nil] : nil
+      end
+
+      private
+        def connect
+          # By default, MySQL 'where id is null' selects the last inserted id.
+          # Turn this off. http://dev.rubyonrails.org/ticket/6778
+          ##FIXME !!! execute("SET SQL_AUTO_IS_NULL=0")
+        end
+
+        def select(sql, name = nil)
+          result = execute(sql, name)
+          rows = result.hash_rows
+          rows
+        end
+
+        # Executes the update statement and returns the number of rows affected.
+        def update_sql(sql, name = nil)
+          execute(sql, name).rows[0][0]
+        end
+        
+        def supports_views?
+          ## get mysql version
+          version[0] >= 5
+        end
+
+        def version
+          @version ||= @connection.server_info.scan(/^(\d+)\.(\d+)\.(\d+)/).flatten.map { |v| v.to_i }
+        end
+    end
+  end
+end