sequel 3.12.1 → 3.13.0

data/doc/sharding.rdoc

@@ -4,7 +4,7 @@ Sequel has support for read only slave databases
 with a writable master database, as well as database sharding (where you can
 pick a server to use for a given dataset).  Support for both
 features is database independent, and should work for all database adapters
-included with Sequel.
+that ship with Sequel.
 
 == The :servers Database option
 

data/doc/sql.rdoc

@@ -1,6 +1,6 @@
 = Sequel for SQL Users
 
-One of the main benefits of Sequel is that it doesn't require the user know SQL in order to use it, though SQL knowledge is certainly helpful.  Unlike most other Sequel documentation, this guide assumes you know SQL, and provides an easy way to discover how to do something in Sequel given the knowledge of how to do so in SQL.
+One of the main benefits of Sequel is that it doesn't require the user to know SQL in order to use it, though SQL knowledge is certainly helpful.  Unlike most other Sequel documentation, this guide assumes you know SQL, and provides an easy way to discover how to do something in Sequel given the knowledge of how to do so in SQL.
 
 == You Can Just Use SQL
 
@@ -32,7 +32,7 @@ What Sequel actually does internally is two separate things.  It first creates a
 
   ds = DB.fetch("SELECT * FROM albums")
 
-Then when you want to retrieve the rows later, you can call +each+ on the dataset to retrieve the rows:
+Then, when you want to retrieve the rows later, you can call +each+ on the dataset to retrieve the rows:
 
   ds.each{|r| puts r[:name]}
 
@@ -78,7 +78,7 @@ Almost everywhere in Sequel, you can drop down to literal SQL by providing a lit
   DB[:albums].select('name') # SELECT 'name' FROM albums
   DB[:albums].select('name'.lit) # SELECT name FROM albums
 
-So you can use Sequel's DSL for everywhere you find it helpful, and fallback to literal SQL if the DSL can't do what you want or you just find literal SQL easier.
+So you can use Sequel's DSL everywhere you find it helpful, and fallback to literal SQL if the DSL can't do what you want or you just find literal SQL easier.
 
 == Translating SQL Expressions into Sequel
 
@@ -315,7 +315,7 @@ Note that the following does not work:
 
   1 + :column # raises TypeError
 
-For commutative operates such as + and *, this isn't a problem as you can just reorder, but non-commutative operators such as - and * cannot be expressed directly.  However, Sequel comes with an +sql_expr+ extension that adds an +sql_expr+ method to all objects, allowing you to do:
+For commutative operates such as + and *, this isn't a problem as you can just reorder, but non-commutative operators such as - and / cannot be expressed directly.  However, Sequel comes with an +sql_expr+ extension that adds an +sql_expr+ method to all objects, allowing you to do:
 
   Sequel.extension :sql_expr
   1.sql_expr / :column # (1 / "column")
@@ -413,7 +413,7 @@ Just like for the mathematical operators, you can use the +sql_expr+ extension t
   Sequel.extension :sql_expr
   'Name - '.sql_expr + :name # ('Name - ' || "name")
   
-Sequel also adds an <tt>Array#sql_string_join</tt> method, which concatentates all of the elements in the array:
+Sequel also adds an <tt>Array#sql_string_join</tt> method, which concatenates all of the elements in the array:
 
   ['Name', :name].sql_string_join # ('Name' || "name")
 

data/doc/validations.rdoc

@@ -20,7 +20,7 @@ database-level constraints.
 
 == Data Integrity
 
-Data Integrity is best handled by the database itself.  For example, if you have
+Data integrity is best handled by the database itself.  For example, if you have
 a date column that should never contain a NULL value, the column should be
 specified in the database as NOT NULL.  If you have an integer column that should
 only have values from 1 to 10, there should be a CHECK constraint that ensures
@@ -52,12 +52,12 @@ instance, before sending the INSERT or UPDATE query to the database,
 <tt>Sequel::Model</tt> will attempt to validate the instance by calling
 +validate+.  If +validate+ does not add any errors to the object, the object is
 considered valid, and <tt>valid?</tt> will return true.  If +validate+ adds any errors
-to the object, <tt>valid?</tt> will return false, and the save will either return raise
+to the object, <tt>valid?</tt> will return false, and the save will either raise
 a <tt>Sequel::ValidationFailed</tt> exception (the default), or return nil (if +raise_on_save_failure+
 is false).
 
 By validating the object before sending the database query, Sequel attempts to
-ensure that invalidate objects are not saved in the database.  However, if you
+ensure that invalid objects are not saved in the database.  However, if you
 are not enforcing the same validations in the database via constraints, it's
 possible that invalid data can get added to the database via some other method.
 This leads to odd cases such as retrieving a model object from the database,
@@ -128,7 +128,7 @@ will be covered later in this guide.
 
 While <tt>Sequel::Model</tt> does provide a validations framework, it does not define
 any built-in validation helper methods that you can call.  However, Sequel ships with a
-plugin call +validation_helpers+ that handles most basic validation needs.  So instead of
+plugin called +validation_helpers+ that handles most basic validation needs.  So instead of
 specifying validations like this:
 
   class Album < Sequel::Model
@@ -168,7 +168,7 @@ The following methods are provided by +validation_helpers+:
 
 === +validates_presence+
 
-This is probably the most commonly used helper method, which checks if the specified attributes are not blank.  In general, if an object responds to <tt>blank?</tt>, it calls the method to determine if the object is blank.  Otherwise, nil is considered blank, empty strings or strings that just contain whitespace are blank, and objects that respond to <tt>empty?</tt> and return a true valid are considered blank.  All other objects are considered non-blank for the purposes of +validates_presence+.  This means that +validates_presence+ is safe to use on boolean columns where you want to ensure that either true or false is used, but not NULL.
+This is probably the most commonly used helper method, which checks if the specified attributes are not blank.  In general, if an object responds to <tt>blank?</tt>, it calls the method to determine if the object is blank.  Otherwise, nil is considered blank, empty strings or strings that just contain whitespace are blank, and objects that respond to <tt>empty?</tt> and return true are considered blank.  All other objects are considered non-blank for the purposes of +validates_presence+.  This means that +validates_presence+ is safe to use on boolean columns where you want to ensure that either true or false is used, but not NULL.
 
   class Album < Sequel::Model
     def validate
@@ -247,7 +247,7 @@ The <tt>raise_on_typecast_failure = false</tt> setting tells Sequel to attempt t
   album.copies_sold = 'banana'
   album.copies_sold # => 'banana'
 
-+validates_not_string+ is designed to be used in web applications where all user input comes in as strings.  When the <tt>raise_on_typecast_failure = false</tt> setting is used in such an application, you can call +validates_not_string+ with all non-string columns.  If any of those columns has a string value, it's because the column was set and Sequel was not able to typecast it correctly, which means it probably isn't valid.  For example, let's say that you want to check that a couple columns contain valid dates:
++validates_not_string+ is designed to be used in web applications where all user input comes in as strings.  When the <tt>raise_on_typecast_failure = false</tt> setting is used in such an application, you can call +validates_not_string+ with all non-string columns.  If any of those columns has a string value, it's because the column was set and Sequel was not able to typecast it correctly, which means it probably isn't valid.  For example, let's say that you want to check that a couple of columns contain valid dates:
 
   class Album < Sequel::Model
     self.raise_on_typecast_failure = false
@@ -264,7 +264,7 @@ The <tt>raise_on_typecast_failure = false</tt> setting tells Sequel to attempt t
   album.valid? # => false
   album.errors # => {:release_date=>["is not a valid date"]}
 
-For web applications, you usually want the <tt>raise_on_typecast_failure = false</tt> setting, so that you can accept all of the input without raising an error, and then present the user with all error messages.  Without the setting, if the user submits any invalid data, Sequel will immediately raise an error.  +validates_not_string+ is helpful because it allows you to check for typecasting errors on non-string columns, and provides a good default error message stating that it the attribute is not of the expected type.
+For web applications, you usually want the <tt>raise_on_typecast_failure = false</tt> setting, so that you can accept all of the input without raising an error, and then present the user with all error messages.  Without the setting, if the user submits any invalid data, Sequel will immediately raise an error.  +validates_not_string+ is helpful because it allows you to check for typecasting errors on non-string columns, and provides a good default error message stating that the attribute is not of the expected type.
 
 === +validates_unique+
 
@@ -293,7 +293,7 @@ Additionally, you can also include an optional options hash as the last argument
 :message :: The message to use
 :only_if_modified :: Only check the uniqueness if the object is new or one of the columns has been modified.
 
-+validates_unique+ is the only method in +validation_helpers+ that checks with the database.  Attempting to validate uniqueness outside of the database suffers from a race condition, so any time you want to add a uniqueness validation, you should make sure to add a uniqueness constraint or unique index on the underlying database table.  See the the {"Migrations and Schema Modification" guide}[link:files/doc/migration_rdoc.html] for details on how to do that.  
++validates_unique+ is the only method in +validation_helpers+ that checks with the database.  Attempting to validate uniqueness outside of the database suffers from a race condition, so any time you want to add a uniqueness validation, you should make sure to add a uniqueness constraint or unique index on the underlying database table.  See the {"Migrations and Schema Modification" guide}[link:files/doc/migration_rdoc.html] for details on how to do that.  
 
 == +validation_helpers+ Options
 
@@ -301,7 +301,7 @@ All +validation_helpers+ methods except +validates_unique+ accept the following
 
 === <tt>:message</tt>
 
-The most commonly used option, used to override the default validation message.  Can be either a string or proc.  If a string, it is used directly.  If a proc, the proc is called and should return a string.  If the validation method takes a argument before the array of attributes, that argument is passed as an argument to the proc. The exception is the +validates_not_string+ method, which doesn�t take an argument, but passes the schema type symbol as the argument to the proc. 
+The most commonly used option, used to override the default validation message.  Can be either a string or a proc.  If a string, it is used directly.  If a proc, the proc is called and should return a string.  If the validation method takes an argument before the array of attributes, that argument is passed as an argument to the proc. The exception is the +validates_not_string+ method, which doesn�t take an argument, but passes the schema type symbol as the argument to the proc. 
 
   class Album < Sequel::Model
     def validate
@@ -460,7 +460,7 @@ This will make sure that all string columns in the model are validated to make s
 If you forget to call +super+, the validations that you defined in <tt>Sequel::Model</tt> will not be enforced.  It's a good idea to call super whenever you override one of <tt>Sequel::Model</tt>'s methods, unless you specifically do not want the default behavior.
 
 == <tt>Sequel::Model::Errors</tt>
-
+'
 As mentioned earlier, <tt>Sequel::Model::Errors</tt> is a subclass of Hash with a few special methods, the most common of which are described here:
 
 === +add+
@@ -483,7 +483,7 @@ Here, you don't care about validating the release date if there were validation
 
 === +full_messages+
 
-+full_messages+ returns an area of error messages for the object.  It's commonly called after validation to get a list of error messages to display to the user:
++full_messages+ returns an array of error messages for the object.  It's commonly called after validation to get a list of error messages to display to the user:
 
   album.errors
   # => {:name=>["cannot be empty"]}

data/lib/sequel/adapters/ado.rb

@@ -68,7 +68,7 @@ module Sequel
       # The ADO adapter's default provider doesn't support transactions, since it 
       # creates a new native connection for each query.  So Sequel only attempts
       # to use transactions if an explicit :provider is given.
-      def _transaction(conn)
+      def _transaction(conn, o={})
         return super if opts[:provider]
         th = Thread.current
         begin

data/lib/sequel/adapters/do.rb

@@ -118,7 +118,7 @@ module Sequel
       # transactions.  Unfortunately, it tries to create a new connection
       # to do a transaction.  So we close the connection created and
       # substitute our own.
-      def begin_transaction(conn)
+      def begin_transaction(conn, opts={})
         return super if supports_savepoints?
         log_yield(TRANSACTION_BEGIN) do
           t = ::DataObjects::Transaction.create_for_uri(uri)
@@ -131,7 +131,7 @@ module Sequel
       
       # DataObjects requires transactions be prepared before being
       # committed, so we do that.
-      def commit_transaction(t)
+      def commit_transaction(t, opts={})
         return super if supports_savepoints?
         log_yield(TRANSACTION_ROLLBACK) do
           t.prepare
@@ -156,7 +156,7 @@ module Sequel
       end
       
       # We use the transactions rollback method to rollback.
-      def rollback_transaction(t)
+      def rollback_transaction(t, opts={})
         return super if supports_savepoints?
         log_yield(TRANSACTION_COMMIT){t.rollback}
       end

data/lib/sequel/adapters/firebird.rb

@@ -102,12 +102,12 @@ module Sequel
         AUTO_INCREMENT
       end
       
-      def begin_transaction(conn)
+      def begin_transaction(conn, opts={})
         log_yield(TRANSACTION_BEGIN){conn.transaction}
         conn
       end
 
-      def commit_transaction(conn)
+      def commit_transaction(conn, opts={})
         log_yield(TRANSACTION_COMMIT){conn.commit}
       end
       
@@ -182,7 +182,7 @@ module Sequel
         "ALTER SEQUENCE #{seq_name} RESTART WITH #{opts[:restart_position]}"
       end
       
-      def rollback_transaction(conn)
+      def rollback_transaction(conn, opts={})
         log_yield(TRANSACTION_ROLLBACK){conn.rollback}
       end
 

data/lib/sequel/adapters/jdbc/h2.rb

@@ -6,6 +6,12 @@ module Sequel
       module DatabaseMethods
         PRIMARY_KEY_INDEX_RE = /\Aprimary_key/i.freeze
       
+        # Commit an existing prepared transaction with the given transaction
+        # identifier string.
+        def commit_prepared_transaction(transaction_id)
+          run("COMMIT TRANSACTION #{transaction_id}")
+        end
+
         # H2 uses the :h2 database type.
         def database_type
           :h2
@@ -16,13 +22,39 @@ module Sequel
           Sequel::JDBC::H2::Dataset.new(self, opts)
         end
         
+        # Rollback an existing prepared transaction with the given transaction
+        # identifier string.
+        def rollback_prepared_transaction(transaction_id)
+          run("ROLLBACK TRANSACTION #{transaction_id}")
+        end
+
         # H2 uses an IDENTITY type
         def serial_primary_key_options
           {:primary_key => true, :type => :identity}
         end
+
+        # H2 supports prepared transactions
+        def supports_prepared_transactions?
+          true
+        end
+        
+        # H2 supports savepoints
+        def supports_savepoints?
+          true
+        end
         
         private
         
+        # If the :prepare option is given and we aren't in a savepoint,
+        # prepare the transaction for a two-phase commit.
+        def commit_transaction(conn, opts={})
+          if opts[:prepare] && Thread.current[:sequel_transaction_depth] <= 1
+            log_connection_execute(conn, "PREPARE COMMIT #{opts[:prepare]}")
+          else
+            super
+          end
+        end
+
         # H2 needs to add a primary key column as a constraint
         def alter_table_sql(table, op)
           case op[:op]
@@ -75,6 +107,7 @@ module Sequel
       # Dataset class for H2 datasets accessed via JDBC.
       class Dataset < JDBC::Dataset
         SELECT_CLAUSE_METHODS = clause_methods(:select, %w'distinct columns from join where group having compounds order limit')
+        BITWISE_METHOD_MAP = {:& =>:BITAND, :| => :BITOR, :^ => :BITXOR}
         
         # Work around H2's lack of a case insensitive LIKE operator
         def complex_expression_sql(op, args)
@@ -83,6 +116,12 @@ module Sequel
             super(:LIKE, [SQL::PlaceholderLiteralString.new("CAST(? AS VARCHAR_IGNORECASE)", [args.at(0)]), args.at(1)])
           when :"NOT ILIKE"
             super(:"NOT LIKE", [SQL::PlaceholderLiteralString.new("CAST(? AS VARCHAR_IGNORECASE)", [args.at(0)]), args.at(1)])
+          when :&, :|, :^
+            literal(SQL::Function.new(BITWISE_METHOD_MAP[op], *args))
+          when :<<
+            "(#{literal(args[0])} * POWER(2, #{literal(args[1])}))"
+          when :>>
+            "(#{literal(args[0])} / POWER(2, #{literal(args[1])}))"
           else
             super(op, args)
           end

data/lib/sequel/adapters/jdbc/mysql.rb

@@ -54,6 +54,11 @@ module Sequel
         def requires_return_generated_keys?
           true
         end
+      
+        # Convert tinyint(1) type to boolean
+        def schema_column_type(db_type)
+          db_type == 'tinyint(1)' ? :boolean : super
+        end
       end
       
       # Dataset class for MySQL datasets accessed via JDBC.

data/lib/sequel/adapters/jdbc/oracle.rb

@@ -19,13 +19,13 @@ module Sequel
         private
         
         # Use JDBC connection's setAutoCommit to false to start transactions
-        def begin_transaction(conn)
+        def begin_transaction(conn, opts={})
           log_yield(TRANSACTION_BEGIN){conn.setAutoCommit(false)}
           conn
         end
         
         # Use JDBC connection's commit method to commit transactions
-        def commit_transaction(conn)
+        def commit_transaction(conn, opts={})
           log_yield(TRANSACTION_COMMIT){conn.commit}
         end
         
@@ -36,7 +36,7 @@ module Sequel
         end
         
         # Use JDBC connection's rollback method to rollback transactions
-        def rollback_transaction(conn)
+        def rollback_transaction(conn, opts={})
           log_yield(TRANSACTION_ROLLBACK){conn.rollback}
         end
       end

data/lib/sequel/adapters/mysql.rb

@@ -115,18 +115,21 @@ module Sequel
           Mysql::CLIENT_MULTI_STATEMENTS +
           (opts[:compress] == false ? 0 : Mysql::CLIENT_COMPRESS)
         )
+        sqls = []
         # Set encoding a slightly different way after connecting,
         # in case the READ_DEFAULT_GROUP overrode the provided encoding.
         # Doesn't work across implicit reconnects, but Sequel doesn't turn on
         # that feature.
-        conn.query("set names #{literal(encoding.to_s)}") if encoding
+        sqls << "SET NAMES #{literal(encoding.to_s)}" if encoding
 
         # increase timeout so mysql server doesn't disconnect us
-        conn.query("set @@wait_timeout = #{opts[:timeout] || 2592000}")
+        sqls << "SET @@wait_timeout = #{opts[:timeout] || 2592000}"
 
         # By default, MySQL 'where id is null' selects the last inserted id
-        conn.query("set SQL_AUTO_IS_NULL=0") unless opts[:auto_is_null]
-        
+        sqls << "SET SQL_AUTO_IS_NULL=0" unless opts[:auto_is_null]
+
+        sqls.each{|sql| log_yield(sql){conn.query(sql)}}
+
         class << conn
           attr_accessor :prepared_statements
         end

data/lib/sequel/adapters/oracle.rb

@@ -74,12 +74,12 @@ module Sequel
 
       private
       
-      def begin_transaction(conn)
+      def begin_transaction(conn, opts={})
         log_yield(TRANSACTION_BEGIN){conn.autocommit = false}
         conn
       end
       
-      def commit_transaction(conn)
+      def commit_transaction(conn, opts={})
         log_yield(TRANSACTION_COMMIT){conn.commit}
       end
 
@@ -92,7 +92,7 @@ module Sequel
         super
       end
       
-      def rollback_transaction(conn)
+      def rollback_transaction(conn, opts={})
         log_yield(TRANSACTION_ROLLBACK){conn.rollback}
       end
     end

data/lib/sequel/adapters/shared/mssql.rb

@@ -39,6 +39,11 @@ module Sequel
         true
       end
       
+      # MSSQL supports transaction isolation levels
+      def supports_transaction_isolation_levels?
+        true
+      end
+
       # Microsoft SQL Server supports using the INFORMATION_SCHEMA to get
       # information on tables.
       def tables(opts={})
@@ -91,7 +96,7 @@ module Sequel
       
       # Commit the active transaction on the connection, does not commit/release
       # savepoints.
-      def commit_transaction(conn)
+      def commit_transaction(conn, opts={})
         log_connection_execute(conn, commit_transaction_sql) unless Thread.current[:sequel_transaction_depth] > 1
       end
 
@@ -215,6 +220,10 @@ module Sequel
           super(:LIKE, args)
         when :"NOT ILIKE"
           super(:"NOT LIKE", args)
+        when :<<
+          "(#{literal(args[0])} * POWER(2, #{literal(args[1])}))"
+        when :>>
+          "(#{literal(args[0])} / POWER(2, #{literal(args[1])}))"
         else
           super(op, args)
         end

data/lib/sequel/adapters/shared/mysql.rb

@@ -30,6 +30,12 @@ module Sequel
         CAST_TYPES[type] || super
       end
 
+      # Commit an existing prepared transaction with the given transaction
+      # identifier string.
+      def commit_prepared_transaction(transaction_id)
+        run("XA COMMIT #{literal(transaction_id)}")
+      end
+
       # MySQL uses the :mysql database type
       def database_type
         :mysql
@@ -52,6 +58,12 @@ module Sequel
         indexes.reject{|k,v| remove_indexes.include?(k)}
       end
 
+      # Rollback an existing prepared transaction with the given transaction
+      # identifier string.
+      def rollback_prepared_transaction(transaction_id)
+        run("XA ROLLBACK #{literal(transaction_id)}")
+      end
+
       # Get version of MySQL server, used for determined capabilities.
       def server_version
         m = /(\d+)\.(\d+)\.(\d+)/.match(get(SQL::Function.new(:version)))
@@ -67,11 +79,21 @@ module Sequel
         metadata_dataset.with_sql('SHOW TABLES').server(opts[:server]).map{|r| m.call(r.values.first)}
       end
       
+      # MySQL supports prepared transactions (two-phase commit) using XA
+      def supports_prepared_transactions?
+        true
+      end
+
       # MySQL supports savepoints
       def supports_savepoints?
         true
       end
 
+      # MySQL supports transaction isolation levels
+      def supports_transaction_isolation_levels?
+        true
+      end
+
       # Changes the database in use by issuing a USE statement.  I would be
       # very careful if I used this.
       def use(db_name)
@@ -117,6 +139,23 @@ module Sequel
         AUTO_INCREMENT
       end
       
+      # MySQL needs to set transaction isolation before begining a transaction
+      def begin_new_transaction(conn, opts)
+        set_transaction_isolation(conn, opts)
+        log_connection_execute(conn, begin_transaction_sql)
+      end
+
+      # Use XA START to start a new prepared transaction if the :prepare
+      # option is given.
+      def begin_transaction(conn, opts={})
+        if s = opts[:prepare]
+          log_connection_execute(conn, "XA START #{literal(s)}")
+          conn
+        else
+          super
+        end
+      end
+
       # MySQL doesn't allow default values on text columns, so ignore if it the
       # generic text type is used
       def column_definition_sql(column)
@@ -124,6 +163,17 @@ module Sequel
         super
       end
       
+      # Prepare the XA transaction for a two-phase commit if the
+      # :prepare option is given.
+      def commit_transaction(conn, opts={})
+        if s = opts[:prepare]
+          log_connection_execute(conn, "XA END #{literal(s)}")
+          log_connection_execute(conn, "XA PREPARE #{literal(s)}")
+        else
+          super
+        end
+      end
+
       # Use MySQL specific syntax for engine type and character encoding
       def create_table_sql(name, generator, options = {})
         engine = options.fetch(:engine, Sequel::MySQL.default_engine)
@@ -162,6 +212,17 @@ module Sequel
         "CREATE #{index_type}INDEX #{index_name}#{using} ON #{quote_schema_table(table_name)} #{literal(index[:columns])}"
       end
       
+      # Rollback the currently open XA transaction
+      def rollback_transaction(conn, opts={})
+        if s = opts[:prepare]
+          log_connection_execute(conn, "XA END #{literal(s)}")
+          log_connection_execute(conn, "XA PREPARE #{literal(s)}")
+          log_connection_execute(conn, "XA ROLLBACK #{literal(s)}")
+        else
+          super
+        end
+      end
+
       # MySQL treats integer primary keys as autoincrementing.
       def schema_autoincrementing_primary_key?(schema)
         super and schema[:db_type] =~ /int/io
@@ -240,6 +301,8 @@ module Sequel
           else
             literal(args.at(0))
           end
+        when :'B~'
+          "CAST(~#{literal(args.at(0))} AS SIGNED INTEGER)"
         else
           super(op, args)
         end