sequel 3.21.0 → 3.39.0

data/doc/testing.rdoc

@@ -0,0 +1,169 @@
+= Testing with Sequel
+
+Whether or not you use Sequel in your application, you are usually going to want to have tests that ensure that your code works.  When you are using Sequel, it's helpful to integrate it into your testing framework, and it's generally best to run each test in its own transaction if possible.  That keeps all tests isolated from each other, and it's simple as it handles all of the cleanup for you.  Sequel doesn't ship with helpers for common libraries, as the exact code you need is often application-specific, but this page offers some examples that you can either use directly or build on.
+
+== Transactional tests
+
+These run each test in its own transaction, the recommended way to test.
+
+Make sure you are using Sequel 3.29.0 or above when using these examples, as older versions don't support the <tt>:rollback=>:always</tt> option.
+
+=== RSpec 1
+
+  class Spec::Example::ExampleGroup
+    def execute(*args, &block)
+      result = nil
+      Sequel::Model.db.transaction(:rollback=>:always){result = super(*args, &block)}
+      result
+    end
+  end
+
+=== RSpec 2, <2.8
+
+  class RSpec::Core::ExampleGroup
+    # Setting an around filter globally doesn't appear to work in <2.8,
+    # so set one up for each subclass.
+    def self.inherited(subclass)
+      super
+      subclass.around do |example|
+        Sequel::Model.db.transaction(:rollback=>:always){example.call}
+      end
+    end
+  end
+
+=== RSpec 2, >=2.8
+
+  # Global around filters should work
+  RSpec.configure do |c|
+    c.around(:each) do |example|
+      DB.transaction(:rollback=>:always){example.run}
+    end
+  end
+
+=== Test::Unit
+
+  # Must use this class as the base class for your tests
+  class SequelTestCase < Test::Unit::TestCase
+    def run(*args, &block)
+      result = nil
+      Sequel::Model.db.transaction(:rollback=>:always){result = super}
+      result
+    end
+  end
+
+  # Or you could override the base implementation like this
+  class Test::Unit::TestCase
+    alias_method :_original_run, :run
+
+    def run(*args, &block)
+      result = nil
+      Sequel::Model.db.transaction(:rollback => :always) do
+        result = _original_run(*args, &block)
+      end
+      result
+    end
+  end
+
+=== MiniTest::Unit
+
+  # Add a subclass
+  # Must use this class as the base class for your tests
+  class SequelTestCase < MiniTest::Unit::TestCase
+    def run(*args, &block)
+      result = nil
+      Sequel::Model.db.transaction(:rollback=>:always){result = super}
+      result
+    end
+  end
+
+  # Or you could override the base implementation like this
+  class MiniTest::Unit::TestCase
+    alias_method :_original_run, :run
+
+    def run(*args, &block)
+      result = nil
+      Sequel::Model.db.transaction(:rollback => :always) do
+        result = _original_run(*args, &block)
+      end
+      result
+    end
+  end
+
+== Transactional testing with multiple databases
+
+You can use the Sequel.transaction method to run a transaction on multiple databases, rolling all of them back.  Instead of:
+
+  Sequel::Model.db.transaction(:rollback=>:always)
+
+Use Sequel.transaction with an array of databases:
+
+  Sequel.transaction([DB1, DB2, DB3], :rollback=>:always)
+
+== Nontransactional tests
+
+In some cases, it is not possible to use transactions.  For example, if you are testing a web application that is running in a separate process, you don't have access to that process's database connections, so you can't run your examples in transactions.  In that case, the best way to handle things is to cleanup after each test by deleting or truncating the database tables used in the test.
+
+The order in which you delete/truncate the tables is important if you are using referential integrity in your database (which you probably should be doing).  If you are using referential integrity, you need to make sure to delete in tables referencing other tables before the tables that are being referenced.  For example, if you have an +albums+ table with an +artist_id+ field referencing the +artists+ table, you want to delete/truncate the +albums+ table before the +artists+ table.  Note that if you have cyclic references in your database, you will probably need to write your own custom cleaning code.
+
+=== RSpec
+
+  class Spec::Example::ExampleGroup
+    after do
+      [:table1, :table2].each{|x| Sequel::Model.db.from(x).truncate}
+      # or
+      [:table1, :table2].each{|x| Sequel::Model.db.from(x).delete}
+    end
+  end
+
+=== Test::Unit
+
+  # Must use this class as the base class for your tests
+  class SequelTestCase < Test::Unit::TestCase
+    def teardown
+      [:table1, :table2].each{|x| Sequel::Model.db.from(x).truncate}
+      # or
+      [:table1, :table2].each{|x| Sequel::Model.db.from(x).delete}
+    end
+  end
+
+= Testing Sequel Itself
+
+Sequel has multiple separate test suites.  All test suites run under either RSpec 1 or RSpec 2.
+
+== rake spec
+
+The +spec+ rake task (which is also the default rake task) runs Sequel's core and model specs.  These specs use a mocked database connection, and test for specific SQL used and for generally correct behavior.
+
+== rake spec_plugin
+
+The +spec_plugin+ rake task runs the specs for the plugins and extensions that ship with Sequel.  These also use a mocked database connection, and operate very similarly to the general Sequel core and model specs.
+
+== rake spec_<i>adapter</i> (e.g. rake spec_postgres)
+
+The <tt>spec_<i>adapter</i></tt> specs run against a real database connection with nothing mocked, and test for correct results.  They are slower than the standard specs, but they will catch errors that are mocked out by the default specs, as well as show issues that only occur on a certain database, adapter, or a combination of the two.
+
+These specs are broken down into two parts.  For each database, there are specific specs that only apply to that database, and these are called the adapter specs.  There are also shared specs that apply to all (or almost all) databases, these are called the integration specs.
+
+== Environment variables
+
+Sequel often uses environment variables when testing to specify either the database to be tested or specify how testing should be done.  You can also specify the databases to test by copying spec/spec_config.rb.example to spec/spec_config.rb and modifying it.  See that file for details.  It may be necessary to use spec_config.rb as opposed to an environment variable if your database connection cannot be specified by a connection string.
+
+=== Connection Strings
+
+The following environment variables specify Database connection URL strings:
+
+* SEQUEL_INTEGRATION_URL: integration specs
+* SEQUEL_FB_SPEC_DB: firebird adapter specs
+* SEQUEL_MSSQL_SPEC_DB: mssql adapter specs
+* SEQUEL_PG_SPEC_DB: postgres adapter specs
+* SEQUEL_SQLITE_SPEC_DB: sqlite adapter specs
+* SEQUEL_MY_SPEC_DB: mysql adapter specs
+* SEQUEL_DB2_SPEC_DB: db2 adapter specs
+
+=== Other
+
+* SEQUEL_MSSQL_SPEC_REQUIRE: Separate file to require when running mssql adapter specs
+* SEQUEL_DB2_SPEC_REQUIRE: Separate file to require when running db2 adapter specs
+* SEQUEL_COLUMNS_INTROSPECTION: Whether to run the specs with the columns_introspection extension loaded by default
+* SEQUEL_NO_PENDING: Don't mark any specs as pending, try running all specs
+* SKIPPED_TEST_WARN: Warn when skipping any tests because libraries aren't available

data/doc/thread_safety.rdoc

@@ -0,0 +1,17 @@
+= Thread Safety
+
+Most Sequel usage (and all common Sequel usage) is thread safe by default.  Specifically, multiple threads can operate on Database instances, Dataset instances, and Model classes concurrently without problems.  In general, Database instance and Model classes are not modified after application startup, and modifying Dataset instances returns modified copies of the dataset instead of mutating it.
+
+== Connection Pool
+
+In order to allow multiple threads to operate on the same database at the same time, Sequel uses a connection pool.  The connection pool is designed so that a thread uses a connection for the minimum amount of time, returning the connection to the pool as soon as it is done using the connection.  If a thread requests a connection and the pool does not have an available connection, a new connection will be created.  If the maximum number of connections in the pool has already been reached, the thread will block (actually busy-wait) until a connection is available or the the connection pool timeout has elapsed (in which case a PoolTimeout error will be raised).
+
+== Exceptions
+
+This is a small list of things that are specifically non thread-safe.  This is not an exhaustive list, there may be cases not mentioned here.
+
+1) Model instances: Model instances are not thread-safe unless they are frozen first.  Multiple threads should not operate on an unfrozen model instance concurrently.
+
+2) Model class modifications: Model class modifications, such as adding associations and loading plugins, are not designed to be thread safe.  You should not modify a class in one thread if any other thread can concurrently access it.  Model subclassing is designed to be thread-safe, so you create a model subclass in a thread and modify it safely.
+
+3) Dataset mutation methods: Dataset mutation methods are not thread safe, you should not call them on datasets that could be accessed by other threads.  It is safe to clone the dataset first inside a thread and call mutation methods on the cloned dataset.

data/doc/transactions.rdoc

@@ -0,0 +1,137 @@
+= Database Transactions 
+
+Sequel uses autocommit mode by default for all of its database adapters, so in general in Sequel if you want to use database transactions, you need to be explicit about it.  There are a few cases where transactions are used implicitly by default:
+
+* Dataset#import to insert many records at once
+* Model#save
+* Model#destroy
+* Migrations if the database supports transactional schema
+* A few model plugins
+
+Everywhere else, it is up to use to use a database transaction if you want to.
+
+== Basic Transaction Usage
+
+In Sequel, the <tt>Database#transaction</tt> method should be called if you want to use a database transaction.  This method must be called with a block.  If the block does not raise an exception, the transaction is committed:
+
+  DB.transaction do # BEGIN
+    DB[:foo].insert(1) # INSERT
+  end # COMMIT
+   
+If the block raises a Sequel::Rollback exception, the transaction is rolled back, but no exception is raised outside the block:
+
+  DB.transaction do # BEGIN
+    raise Sequel::Rollback
+  end # ROLLBACK
+  # no exception raised
+
+If any other exception is raised, the transaction is rolled back, and the exception is raised outside the block:
+
+  # ArgumentError raised
+  DB.transaction do # BEGIN
+    raise ArgumentError
+  end # ROLLBACK
+  # ArgumentError raised
+
+If you want Sequel::Rollback exceptions to be reraised, use the <tt>:rollback => :reraise</tt> option:
+
+  DB.transaction(:rollback => :reraise) do # BEGIN
+    raise Sequel::Rollback
+  end # ROLLBACK
+  # Sequel::Rollback raised
+
+If you always want to rollback (useful for testing), use the <tt>:rollback => :always</tt> option:
+
+  DB.transaction(:rollback => :always) do # BEGIN
+    DB[:foo].insert(1) # INSERT
+  end # ROLLBACK
+
+If you want to check whether you are currently in a transaction, use the Database#in_transaction? method:
+
+  DB.in_transaction? # false
+  DB.transaction do
+    DB.in_transaction? # true
+  end
+
+== Transaction Hooks
+
+You can add hooks to an in progress transaction that are called after the transaction commits or rolls back:
+
+  x = nil
+  DB.transaction do
+    DB.after_commit{x = 1}
+    DB.after_rollback{x = 2}
+    x # nil
+  end
+  x # 1
+  
+  x = nil
+  DB.transaction do
+    DB.after_commit{x = 1}
+    DB.after_rollback{x = 2}
+    raise Sequel::Rollback
+  end
+  x # 2
+
+== Nested Transaction Calls / Savepoints
+
+You can nest calls to transaction, which by default just reuses the existing transaction:
+
+  DB.transaction do # BEGIN
+    DB.transaction do
+      DB[:foo].insert(1) # INSERT
+    end
+  end # COMMIT
+
+You can use the <tt>:savepoint => true</tt> option in the inner transaction to explicitly use a savepoint (if the database supports it):
+
+  DB.transaction do # BEGIN
+    DB.transaction(:savepoint => true) do # SAVEPOINT
+      DB[:foo].insert(1) # INSERT
+    end # RELEASE SAVEPOINT
+  end # COMMIT
+
+If a Sequel::Rollback exception is raised inside the savepoint block, it will only rollback to the savepoint:
+
+  DB.transaction do # BEGIN
+    DB.transaction(:savepoint => true) do # SAVEPOINT
+      raise Sequel::Rollback
+    end # ROLLBACK TO SAVEPOINT
+    # no exception raised
+  end # COMMIT
+
+Other exceptions, unless rescued inside the outer transaction block, will rollback the savepoint and the outer transactions, since they are reraised by the transaction code:
+
+  DB.transaction do # BEGIN
+    DB.transaction(:savepoint => true) do # SAVEPOINT
+      raise ArgumentError
+    end # ROLLBACK TO SAVEPOINT
+  end # ROLLBACK
+  # ArgumentError raised
+
+== Prepared Transactions / Two-Phase Commit
+
+Sequel supports database prepared transactions on PostgreSQL, MySQL, and H2.  With prepared transactions, at the end of the transaction, the transaction is not immediately committed (it acts like a rollback).  Later, you can call +commit_prepared_transaction+ to commit the transaction or +rollback_prepared_transaction+ to roll the transaction back.  Prepared transactions are usually used with distributed databases to make sure all databases commit the same transaction or none of them do.
+
+To use prepared transactions in Sequel, you provide a string as the value of the :prepare option:
+
+  DB.transaction(:prepare => 'foo') do # BEGIN
+    DB[:foo].insert(1) # INSERT
+  end # PREPARE TRANSACTION 'foo'
+   
+Later, you can commit the prepared transaction:
+
+  DB.commit_prepared_transaction('foo')
+
+or roll the prepared transaction back:
+
+  DB.rollback_prepared_transaction('foo')
+
+== Transaction Isolation Levels
+
+The SQL standard supports 4 isolation levels: READ UNCOMMITTED, READ COMMITTED, REPEATABLE READ, and SERIALIZABLE.  Not all databases implement the levels as specified in the standard (or implement the levels at all), but on PostgreSQL, MySQL, and Microsoft SQL Server, you can specify which transaction isolation level you want to use via the :isolation option to <tt>Database#transaction</tt>.  The isolation level is specified as one of the following symbols: :uncommitted, :committed, :repeatable, and :serializable.  Using this option make Sequel use the correct transaction isolation syntax for your database:
+
+  DB.transaction(:isolation => :serializable) do # BEGIN
+    # SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
+    DB[:foo].insert(1) # INSERT
+  end # COMMIT

data/doc/validations.rdoc

@@ -284,7 +284,7 @@ You can mix and match the two approaches.  For example, if all albums should hav
 
 +validates_unique+ also accepts a block to scope the uniqueness constraint.  For example, if you want to ensure that all active albums have a unique name, but inactive albums can duplicate the name:
 
-  validates_unique(:name){|ds| ds.filter(:active)}
+  validates_unique(:name){|ds| ds.where(:active)}
 
 If you provide a block, it is called with the dataset to use for the uniqueness check, which you can then filter to scope the uniqueness validation to a subset of the model's dataset.
 

data/doc/virtual_rows.rdoc

@@ -1,6 +1,6 @@
 = Virtual Row Blocks
 
-Dataset methods filter, order, and select all take blocks that are referred to as
+Dataset methods where, order, and select all take blocks that are referred to as
 virtual row blocks.  Many other dataset methods pass the blocks
 they are given into one of those three methods, so there are actually
 many Sequel::Dataset methods that take virtual row blocks.
@@ -11,7 +11,7 @@ Virtual Rows were created to work around the issue that some parts of
 Sequel's standard DSL could not be used on ruby 1.9.  For example, the
 following Sequel code works on ruby 1.8, but not ruby 1.9:
 
-  dataset.filter(:a > :b[:c])
+  dataset.where(:a > :b[:c])
   # WHERE a > b(c)
   
 This code does not work on ruby 1.9 for two reasons.  First, Symbol#>
@@ -20,30 +20,15 @@ does not override it to return an SQL inequality expression.  Second, Symbol#[]
 is already defined on ruby 1.9, so Sequel does not override it to return an
 SQL function expression.
 
-Prior to the introduction of virtual rows, the way to handle this was
-to use the methods that work on both ruby 1.8 and ruby 1.9:
+It's possible to use Sequel's DSL to represent such expressions, but it is a
+little verbose:
 
-  dataset.filter(:a.identifier > :b.sql_function(:c))
+  dataset.where(Sequel.expr(:a) > Sequel.function(:b, :c))
   # WHERE a > b(c)
 
-However, that code is a little verbose.  The virtual row DSL makes such code
-more concise:
+The virtual row DSL makes such code more concise:
 
-  dataset.filter{a > b(c)}
-
-Another use of virtual rows is when you turn off Sequel's core extensions off
-using the SEQUEL_NO_CORE_EXTENSIONS constant or environment variable.  With
-the core extensions turned off, much of the standard Sequel DSL is not
-available.  For example, you are no longer able to do:
-
-  dataset.filter(:a & (:b | ~:c))
-  # WHERE a AND (b OR NOT c)
-  
-Because Symbol#&, Symbol#| and Symbol#~ are not defined when the core
-extensions are turned off.  However, with virtual rows allow almost the same
-syntax even without the core extensions:
-
-  dataset.filter{a & (b | ~c)}
+  dataset.where{a > b(c)}
 
 == Regular Procs vs Instance Evaled Procs
 
@@ -54,11 +39,11 @@ evaluated in the context of an instance of Sequel::SQL::VirtualRow.
 
   ds = DB[:items]
   # Regular proc
-  ds.filter{|o| o.column > 1}
+  ds.where{|o| o.column > 1}
   # WHERE column > 1
   
   # Instance-evaled proc
-  ds.filter{column > 1}
+  ds.where{column > 1}
   # WHERE column > 1
   
 If you aren't familiar with the difference between regular blocks and instance
@@ -75,21 +60,21 @@ inside the proc.  If that doesn't make sense, maybe this example will help:
   b = 32
   
   # Regular proc
-  ds.filter{|o| o.c > a - b}
+  ds.where{|o| o.c > a - b}
   # WHERE c > 10
   
   # Instance-evaled proc
-  ds.filter{c > a - b}
+  ds.where{c > a - b}
   # WHERE c > (a - 32)
   
-There are two related differences here.  First is the usage of "o.c" vs "c",
-and second is the difference between the the use of "a".  In the regular proc,
-you couldn't call c without an explicit receiver in the proc, unless the self of the
-surrounding scope responded to it.  For a, note how ruby calls the method on
+There are two related differences here.  First is the usage of <tt>o.c</tt> vs +c+,
+and second is the difference between the the use of +a+.  In the regular proc,
+you couldn't call +c+ without an explicit receiver in the proc, unless the self of the
+surrounding scope responded to it.  For +a+, note how ruby calls the method on
 the receiver of the surrounding scope in the regular proc, which returns an integer,
-and does the substraction before Sequel gets access to it.  In the instance evaled
-proc, calling a without a receiver calls the a method on the VirtualRow instance.
-For b, note that it operates the same in both cases, as it is a local variable.
+and does the subtraction before Sequel gets access to it.  In the instance evaled
+proc, calling +a+ without a receiver calls the a method on the VirtualRow instance.
+For +b+, note that it operates the same in both cases, as it is a local variable.
 
 Basically, the choice for whether to use a regular proc or an instance evaled proc is
 completely up to you.  The same things can be accomplished with both.
@@ -105,7 +90,7 @@ variable, you can call it with () to differentiate the method call from the
 local variable access.  This is mostly useful in instance_evaled procs:
 
   b = 32
-  ds.filter{b() > b}
+  ds.where{b() > b}
   # WHERE b > 32
 
 == VirtualRow Methods
@@ -120,8 +105,8 @@ not qualified by any table. You get an SQL::Identifier if the method is called
 without a block or arguments, and doesn't have a double underscore in the method
 name:
 
-  ds.filter{|o| o.column > 1}
-  ds.filter{column > 1}
+  ds.where{|o| o.column > 1}
+  ds.where{column > 1}
   # WHERE column > 1
   
 == SQL::QualifiedIdentifiers - Qualified columns
@@ -131,8 +116,8 @@ are qualified to a specific table.  You get an SQL::QualifiedIdentifier if
 the method is called without a block or arguments, and has a double underscore
 in the method name:
 
-  ds.filter{|o| o.table__column > 1}
-  ds.filter{table__column > 1}
+  ds.where{|o| o.table__column > 1}
+  ds.where{table__column > 1}
   # WHERE table.column > 1
 
 Using the double underscore for SQL::QualifiedIdentifiers was done to make
@@ -144,15 +129,15 @@ into a qualified column.
 SQL::Functions can be thought of as function calls in SQL.  You get a simple
 function call if you call a method with arguments and without a block:
 
-  ds.filter{|o| o.function(1) > 1}
-  ds.filter{function(1) > 1}
+  ds.where{|o| o.function(1) > 1}
+  ds.where{function(1) > 1}
   # WHERE function(1) > 1
   
 To call a SQL function with multiple arguments, just use those arguments in
 your function call:
   
-  ds.filter{|o| o.function(1, o.a) > 1}
-  ds.filter{function(1, a) > 1}
+  ds.where{|o| o.function(1, o.a) > 1}
+  ds.where{function(1, a) > 1}
   # WHERE function(1, a) > 1
 
 If the SQL function does not accept any arguments, you need to provide an empty
@@ -191,7 +176,8 @@ the method call:
 SQL::WindowFunctions can be thought of as calls to SQL window functions.  Not
 all databases support them, but they are very helpful for certain types of
 queries.  To use them, you need to make :over the first argument of the method
-call, with an optional hash as the second argument: Here are some examples of use:
+call, with an optional hash as the second argument, and provide an empty block
+to the method. Here are some examples of use:
 
   ds.select{|o| o.rank(:over){}}
   ds.select{rank(:over){}}
@@ -205,6 +191,55 @@ call, with an optional hash as the second argument: Here are some examples of us
   ds.select{sum(:over, :args=>col1, :partition=>col2, :order=>col3){}}
   # SELECT sum(col1) OVER (PARTITION BY col2 ORDER BY col3)
 
+== Operators
+
+VirtualRows use method_missing to handle almost all method calls.  However, they
+have special handling of some operator methods to make certain things easier. The
+operators all use a prefix form.
+
+=== Math Operators
+
+The standard +, -, *, and / mathematical operators are defined:
+
+  ds.select{|o| o.-(1, o.a).as(b)}
+  ds.select{self.-(1, a).as(b)}
+  # SELECT (1 - a) AS b
+
+=== Boolean Operators
+
+The & and | methods are defined to use AND and OR:
+
+  ds.where{|o| o.&({:a=>:b}, :c)}
+  ds.where{self.&({:a=>:b}, :c)}
+  # WHERE ((a = b) AND c)
+
+The ~ method is defined to do inversion:
+
+  ds.where{|o| o.~({:a=>1, :b=>2})}
+  ds.where{self.~({:a=>1, :b=>2})}
+  # WHERE ((a != 1) OR (b != 2))
+
+=== Inequality Operators
+
+The standard >, <, >=, and <= inequality operators are defined:
+
+  ds.where{|o| o.>(1, :c)}
+  ds.where{self.>(1, :c)}
+  # WHERE (1 > c)
+
+== Literal Strings
+
+The backtick operator can be used inside an instance-evaled
+virtual row block to create a literal string:
+
+  ds.where{a > `some SQL`}
+  # WHERE (a > some SQL)
+
+You can use this on a regular virtual row block too, but it
+doesn't look as nice:
+
+  ds.where{|o| o.>(:a, o.`('some SQL')}
+
 == Returning multiple values
 
 It's common when using select and order virtual row blocks to want to

data/lib/sequel/adapters/ado/mssql.rb

@@ -11,11 +11,6 @@ module Sequel
         # delete query.
         ROWS_AFFECTED = "SELECT @@ROWCOUNT AS AffectedRows"
         
-        # Return instance of Sequel::ADO::MSSQL::Dataset with the given opts.
-        def dataset(opts=nil)
-          Sequel::ADO::MSSQL::Dataset.new(self, opts)
-        end
-        
         # Just execute so it doesn't attempt to return the number of rows modified.
         def execute_ddl(sql, opts={})
           execute(sql, opts)
@@ -37,11 +32,28 @@ module Sequel
             end
           end
         end
+
+        private
+
+        # The ADO adapter's default provider doesn't support transactions, since it 
+        # creates a new native connection for each query.  So Sequel only attempts
+        # to use transactions if an explicit :provider is given.
+        def begin_transaction(conn, opts={})
+          super if @opts[:provider]
+        end
+
+        def commit_transaction(conn, opts={})
+          super if @opts[:provider]
+        end
+
+        def rollback_transaction(conn, opts={})
+          super if @opts[:provider]
+        end
       end
       
       class Dataset < ADO::Dataset
         include Sequel::MSSQL::DatasetMethods
-        
+
         # Use a nasty hack of multiple SQL statements in the same call and
         # having the last one return the most recently inserted id.  This
         # is necessary as ADO's default :provider uses a separate native