sequel 3.10.0 → 3.11.0

data/doc/virtual_rows.rdoc

@@ -1,29 +1,227 @@
 = Virtual Row Blocks
 
-Dataset methods filter, order, and select all take blocks that either yield
-instances of Sequel::SQL::VirtualRow (if the block takes an argument), or
-are evaluated in the context of an instance of Sequel::SQL::VirtualRow.  These are referred to as
+Dataset methods filter, order, and select all take blocks that are referred to as
 virtual row blocks.  Many other dataset methods pass the blocks
 they are given into one of those three methods, so there are actually
-many Sequel methods that take virtual row blocks.
+many Sequel::Dataset methods that take virtual row blocks.
 
-VirtualRow is a class that returns SQL::Indentifiers,
-SQL::QualifiedIdentifiers, SQL::Functions, or SQL::WindowFunctions depending on how it is
-called.  This is best shown by example:
+== Why Virtual Rows
+
+Virtual Rows were created to work around the issue that some parts of
+Sequel's standard DSL could not be used on ruby 1.9.  For example, the
+following Sequel code works on ruby 1.8, but not ruby 1.9:
+
+  dataset.filter(:a > :b[:c])
+  # WHERE a > b(c)
+  
+This code does not work on ruby 1.9 for two reasons.  First, Symbol#>
+(like other inequality methods) is already defined in ruby 1.9, so Sequel
+does not override it to return an SQL inequality expression.  Second, Symbol#[]
+is already defined on ruby 1.9, so Sequel does not override it to return an
+SQL function expression.
+
+Prior to the introduction of virtual rows, the way to handle this was
+to use the methods that work on both ruby 1.8 and ruby 1.9:
+
+  dataset.filter(:a.identifier > :b.sql_function(:c))
+  # WHERE a > b(c)
+
+However, that code is a little verbose.  The virtual row DSL makes such code
+more concise:
+
+  dataset.filter{a > b(c)}
+
+Another use of virtual rows is when you turn off Sequel's core extensions off
+using the SEQUEL_NO_CORE_EXTENSIONS constant or environment variable.  With
+the core extensions turned off, much of the standard Sequel DSL is not
+available.  For example, you are no longer able to do:
+
+  dataset.filter(:a & (:b | ~:c))
+  # WHERE a AND (b OR NOT c)
+  
+Because Symbol#&, Symbol#| and Symbol#~ are not defined when the core
+extensions are turned off.  However, with virtual rows allow almost the same
+syntax even without the core extensions:
+
+  dataset.filter{a & (b | ~c)}
+
+== Regular Procs vs Instance Evaled Procs
+
+Virtual row blocks behave differently depending on whether the block accepts
+an argument.  If the block accepts an argument, it is called with an instance
+of Sequel::SQL::VirtualRow.  If it does not accept an argument, it is
+evaluated in the context of an instance of Sequel::SQL::VirtualRow.
 
   ds = DB[:items]
-  ds.filter{column > 1}        # column > 1
-  ds.filter{table__column > 1} # table.column > 1
-  ds.filter{function(1) > 1}   # function(1) > 1
-  ds.select{[column1, sum(column2).as(sum)]} # column1, sum(column2) AS sum
-  ds.select{version{}}   # version()
-  ds.select{count(:*){}} # count(*)
-  ds.select{count(:distinct, col1){}} # count(DISTINCT col1)
-  ds.select{rank(:over){}} # rank() OVER ()
-  ds.select{count(:over, :*=>true){}} # count(*) OVER ()
-  ds.select{sum(:over, :args=>col1, :partition=>col2, :order=>col3){}} # sum(col1) OVER (PARTITION BY col2 ORDER BY col3)
-  
-Basically, the rules are:
+  # Regular proc
+  ds.filter{|o| o.column > 1}
+  # WHERE column > 1
+  
+  # Instance-evaled proc
+  ds.filter{column > 1}
+  # WHERE column > 1
+  
+If you aren't familiar with the difference between regular blocks and instance
+evaled blocks, you should probably consult a general ruby reference, but briefly,
+with regular procs, methods called without an explicit receiver inside the
+proc call the method on the receiver in the surrounding scope, while instance
+evaled procs call the method on the receiver of the instance_eval call.  However,
+in both cases, local variables available in the surrounding scope will be available
+inside the proc.  If that doesn't make sense, maybe this example will help:
+
+  def self.a
+    42
+  end
+  b = 32
+  
+  # Regular proc
+  ds.filter{|o| o.c > a - b}
+  # WHERE c > 10
+  
+  # Instance-evaled proc
+  ds.filter{c > a - b}
+  # WHERE c > (a - 32)
+  
+There are two related differences here.  First is the usage of "o.c" vs "c",
+and second is the difference between the the use of "a".  In the regular proc,
+you couldn't call c without an explicit receiver in the proc, unless the self of the
+surrounding scope responded to it.  For a, note how ruby calls the method on
+the receiver of the surrounding scope in the regular proc, which returns an integer,
+and does the substraction before Sequel gets access to it.  In the instance evaled
+proc, calling a without a receiver calls the a method on the VirtualRow instance.
+For b, note that it operates the same in both cases, as it is a local variable.
+
+Basically, the choice for whether to use a regular proc or an instance evaled proc is
+completely up to you.  The same things can be accomplished with both.
+Instance evaled procs tend to produce shorter code, but by modifying the scope
+can be more difficult for a new user to understand.  That being said, I usually
+use instance evaled procs unless I need to call methods on the receiver of the
+surrounding scope inside the proc.
+
+== Local Variables vs Method Calls
+
+If you have a method that accepts 0 arguments and has the same name as a local
+variable, you can call it with () to differentiate the method call from the
+local variable access.  This is mostly useful in instance_evaled procs:
+
+  b = 32
+  ds.filter{b() > b}
+  # WHERE b > 32
+
+== VirtualRow Methods
+
+VirtualRow is a class that returns SQL::Identifiers, SQL::QualifiedIdentifiers,
+SQL::Functions, or SQL::WindowFunctions depending on how it is called.
+
+== SQL::Identifiers - Regular columns
+
+SQL::Identifiers can be thought of as regular column references in SQL,
+not qualified by any table. You get an SQL::Identifier if the method is called
+without a block or arguments, and doesn't have a double underscore in the method
+name:
+
+  ds.filter{|o| o.column > 1}
+  ds.filter{column > 1}
+  # WHERE column > 1
+  
+== SQL::QualifiedIdentifiers - Qualified columns
+
+SQL::QualifiedIdentifiers can be thought of as column references in SQL that
+are qualified to a specific table.  You get an SQL::QualifiedIdentifier if
+the method is called without a block or arguments, and has a double underscore
+in the method name:
+
+  ds.filter{|o| o.table__column > 1}
+  ds.filter{table__column > 1}
+  # WHERE table.column > 1
+
+Using the double underscore for SQL::QualifiedIdentifiers was done to make
+usage very similar to using symbols, which also translate the double underscore
+into a qualified column.
+
+== SQL::Functions - SQL function calls
+
+SQL::Functions can be thought of as function calls in SQL.  You get a simple
+function call if you call a method with arguments and without a block:
+
+  ds.filter{|o| o.function(1) > 1}
+  ds.filter{function(1) > 1}
+  # WHERE function(1) > 1
+  
+To call a SQL function with multiple arguments, just use those arguments in
+your function call:
+  
+  ds.filter{|o| o.function(1, o.a) > 1}
+  ds.filter{function(1, a) > 1}
+  # WHERE function(1, a) > 1
+
+If the SQL function does not accept any arguments, you need to provide an empty
+block to the method to distinguish it from a call that will produce an
+SQL::Identifier:
+
+  ds.select{|o| o.version{}}
+  ds.select{version{}}
+  # SELECT version()
+  
+To use the SQL wildcard (*) as the sole argument in a function call (most often
+used with the count function), you should provide :* as the sole argument to
+the method, and provide an empty block to the method:
+  
+  ds.select{|o| o.count(:*){}}
+  ds.select{count(:*){}}
+  # SELECT count(*)
+
+To append the DISTINCT keyword before the method arguments, you need to make
+:distinct the first argument of the method call, and provide an empty block to
+the method:
+
+  ds.select{|o| o.count(:distinct, o.col1){}}
+  ds.select{count(:distinct, col1){}}
+  # SELECT count(DISTINCT col1)
+  
+To use multiple columns with the DISTINCT keyword, use multiple arguments in
+the method call:
+
+  ds.select{|o| o.count(:distinct, o.col1, o.col2){}}
+  ds.select{count(:distinct, col1, col2){}}
+  # SELECT count(DISTINCT col1, col2)
+  
+== SQL::WindowFunctions - SQL window function calls
+
+SQL::WindowFunctions can be thought of as calls to SQL window functions.  Not
+all databases support them, but they are very helpful for certain types of
+queries.  To use them, you need to make :over the first argument of the method
+call, with an optional hash as the second argument: Here are some examples of use:
+
+  ds.select{|o| o.rank(:over){}}
+  ds.select{rank(:over){}}
+  # SELECT rank() OVER ()
+  
+  ds.select{|o| o.count(:over, :*=>true){}}
+  ds.select{count(:over, :*=>true){}}
+  # SELECT count(*) OVER ()
+  
+  ds.select{|o| o.sum(:over, :args=>o.col1, :partition=>o.col2, :order=>o.col3){}}
+  ds.select{sum(:over, :args=>col1, :partition=>col2, :order=>col3){}}
+  # SELECT sum(col1) OVER (PARTITION BY col2 ORDER BY col3)
+
+== Returning multiple values
+
+It's common when using select and order virtual row blocks to want to
+return multiple values.  If you want to do that, you just need to return a single
+array:
+
+  ds.select{|o| [o.column1, o.sum(o.column2).as(o.sum)]}
+  ds.select{[column1, sum(column2).as(sum)]}
+  # SELECT column1, sum(column2) AS sum
+  
+Note that if you forget the array brackets, you'll end up with a syntax error:
+
+  # Invalid ruby syntax
+  ds.select{|o| o.column1, o.sum(o.column2).as(o.sum)}
+  ds.select{column1, sum(column2).as(sum)}
+  
+== Alternative Description of the VirtualRow method call rules
 
 * If a block is given:
   * The block is currently not called.  This may change in a future version.
@@ -45,15 +243,3 @@ Basically, the rules are:
     SQL::QualifiedIdentifier with the table and column.
   * Otherwise, create an SQL::Identifier with the name of the
     method.
-  
-If you use a virtual row block that doesn't take an argument,
-the block is instance_evaled, so you can't reference methods
-in the enclosing scope.  If you need to call methods of the
-enclosing scope, you should assign the results to local variables
-before the block, or just make the block take an argument and use
-that.  If you want to create identifiers or qualified identifiers
-with the same name as existing local variables, make sure ruby
-knows it is a method call instead of a local variable reference
-by not ommiting the parentheses at the end of the method call.
-If you want to return multiple arguments (common for select and
-order), have the block return an array of arguments.

data/lib/sequel/adapters/ado.rb

@@ -16,16 +16,26 @@ module Sequel
         end
       end
 
-      # Connect to the database. In addition to the usual database options,
+      # In addition to the usual database options,
       # the following options have an effect:
       #
-      # * :command_timeout - Sets the time in seconds to wait while attempting
-      #   to execute a command before cancelling the attempt and generating
-      #   an error. Specifically, it sets the ADO CommandTimeout property.
-      #   If this property is not set, the default of 30 seconds is used.
-      # * :conn_string - The full ADO connection string.  If this is provided,
-      #   the usual options are ignored.
-      # * :provider - Sets the Provider of this ADO connection (for example, "SQLOLEDB")
+      # :command_timeout :: Sets the time in seconds to wait while attempting
+      #                     to execute a command before cancelling the attempt and generating
+      #                     an error. Specifically, it sets the ADO CommandTimeout property.
+      #                     If this property is not set, the default of 30 seconds is used.
+      # :driver :: The driver to use in the ADO connection string.  If not provided, a default
+      #            of "SQL Server" is used.
+      # :conn_string :: The full ADO connection string.  If this is provided,
+      #                 the usual options are ignored.
+      # :provider :: Sets the Provider of this ADO connection (for example, "SQLOLEDB").
+      #              If you don't specify a provider, the default one used by WIN32OLE
+      #              has major problems, such as creating a new native database connection
+      #              for every query, which breaks things such as temporary tables.
+      #
+      # Pay special attention to the :provider option, as without specifying a provider,
+      # many things will be broken.  The SQLNCLI10 provider appears to work well if you
+      # are connecting to Microsoft SQL Server, but it is not the default as that would
+      # break backwards compatability.
       def connect(server)
         opts = server_opts(server)
         s = opts[:conn_string] || "driver=#{opts[:driver]};server=#{opts[:host]};database=#{opts[:database]}#{";uid=#{opts[:user]};pwd=#{opts[:password]}" if opts[:user]}"
@@ -41,10 +51,9 @@ module Sequel
       end
     
       def execute(sql, opts={})
-        log_info(sql)
         synchronize(opts[:server]) do |conn|
           begin
-            r = conn.Execute(sql)
+            r = log_yield(sql){conn.Execute(sql)}
             yield(r) if block_given?
           rescue ::WIN32OLERuntimeError => e
             raise_error(e)
@@ -56,9 +65,11 @@ module Sequel
 
       private
       
-      # The ADO adapter doesn't support transactions, since it appears not to
-      # use a single native connection for each connection in the pool
+      # The ADO adapter's default provider doesn't support transactions, since it 
+      # creates a new native connection for each query.  So Sequel only attempts
+      # to use transactions if an explicit :provider is given.
       def _transaction(conn)
+        return super if opts[:provider]
         th = Thread.current
         begin
           @transactions << th
@@ -81,6 +92,11 @@ module Sequel
           s.getRows.transpose.each{|r| yield cols.inject({}){|m,c| m[c] = r.shift; m}} unless s.eof
         end
       end
+      
+      # ADO returns nil for all for delete and update statements.
+      def provides_accurate_rows_matched?
+        false
+      end
     end
   end
 end

data/lib/sequel/adapters/ado/mssql.rb

@@ -7,11 +7,36 @@ module Sequel
     module MSSQL
       module DatabaseMethods
         include Sequel::MSSQL::DatabaseMethods
+        # Query to use to get the number of rows affected by an update or
+        # delete query.
+        ROWS_AFFECTED = "SELECT @@ROWCOUNT AS AffectedRows"
         
         # Return instance of Sequel::ADO::MSSQL::Dataset with the given opts.
         def dataset(opts=nil)
           Sequel::ADO::MSSQL::Dataset.new(self, opts)
         end
+        
+        # Just execute so it doesn't attempt to return the number of rows modified.
+        def execute_ddl(sql, opts={})
+          execute(sql, opts)
+        end
+        alias execute_insert execute_ddl
+        
+        # Issue a separate query to get the rows modified.  ADO appears to
+        # use pass by reference with an integer variable, which is obviously
+        # not supported directly in ruby, and I'm not aware of a workaround.
+        def execute_dui(sql, opts={})
+          return super unless @opts[:provider]
+          synchronize(opts[:server]) do |conn|
+            begin
+              log_yield(sql){conn.Execute(sql)}
+              res = log_yield(ROWS_AFFECTED){conn.Execute(ROWS_AFFECTED)}
+              res.getRows.transpose.each{|r| return r.shift}
+            rescue ::WIN32OLERuntimeError => e
+              raise_error(e)
+            end
+          end
+        end
       end
       
       class Dataset < ADO::Dataset
@@ -19,11 +44,18 @@ module Sequel
         
         # Use a nasty hack of multiple SQL statements in the same call and
         # having the last one return the most recently inserted id.  This
-        # is necessary as ADO doesn't provide a consistent native connection.
+        # is necessary as ADO's default :provider uses a separate native
+        # connection for each query.
         def insert(*values)
           return super if @opts[:sql]
           with_sql("SET NOCOUNT ON; #{insert_sql(*values)}; SELECT CAST(SCOPE_IDENTITY() AS INTEGER)").single_value
         end
+        
+        # If you use a better :provider option for the database, you can get an
+        # accurate number of rows matched.
+        def provides_accurate_rows_matched?
+          !!db.opts[:provider]
+        end
       end
     end
   end

data/lib/sequel/adapters/amalgalite.rb

@@ -84,37 +84,42 @@ module Sequel
       
       # Run the given SQL with the given arguments. Returns nil.
       def execute_ddl(sql, opts={})
-        _execute(sql, opts){|conn| conn.execute_batch(sql);}
+        _execute(sql, opts){|conn| log_yield(sql){conn.execute_batch(sql)}}
         nil
       end
       
       # Run the given SQL with the given arguments and return the number of changed rows.
       def execute_dui(sql, opts={})
-        _execute(sql, opts){|conn| conn.execute_batch(sql); conn.row_changes}
+        _execute(sql, opts){|conn| log_yield(sql){conn.execute_batch(sql)}; conn.row_changes}
       end
       
       # Run the given SQL with the given arguments and return the last inserted row id.
       def execute_insert(sql, opts={})
-        _execute(sql, opts){|conn| conn.execute_batch(sql); conn.last_insert_rowid}
+        _execute(sql, opts){|conn| log_yield(sql){conn.execute_batch(sql)}; conn.last_insert_rowid}
       end
       
       # Run the given SQL with the given arguments and yield each row.
-      def execute(sql, opts={}, &block)
-        _execute(sql, opts){|conn| conn.prepare(sql, &block)}
+      def execute(sql, opts={})
+        _execute(sql, opts) do |conn|
+          begin
+            yield(stmt = log_yield(sql){conn.prepare(sql)})
+          ensure
+            stmt.close if stmt
+          end
+        end
       end
       
       # Run the given SQL with the given arguments and return the first value of the first row.
       def single_value(sql, opts={})
-        _execute(sql, opts){|conn| conn.first_value_from(sql)}
+        _execute(sql, opts){|conn| log_yield(sql){conn.first_value_from(sql)}}
       end
       
       private
       
-      # Log the SQL and yield an available connection.  Rescue
+      # Yield an available connection.  Rescue
       # any Amalgalite::Errors and turn them into DatabaseErrors.
       def _execute(sql, opts)
         begin
-          log_info(sql)
           synchronize(opts[:server]){|conn| yield conn}
         rescue ::Amalgalite::Error, ::Amalgalite::SQLite3::Error => e
           raise_error(e)

data/lib/sequel/adapters/db2.rb

@@ -33,13 +33,12 @@ module Sequel
       end
       
       def execute(sql, opts={})
-        log_info(sql)
         synchronize(opts[:server]) do |conn|
           rc, sth = SQLAllocHandle(SQL_HANDLE_STMT, @handle) 
           check_error(rc, "Could not allocate statement")
 
           begin
-            rc = SQLExecDirect(sth, sql) 
+            rc = log_yield(sql){SQLExecDirect(sth, sql)}
             check_error(rc, "Could not execute statement")
             
             yield(sth) if block_given?