sequel 4.26.0 → 5.37.0

data/doc/virtual_rows.rdoc

@@ -7,28 +7,16 @@ many Sequel::Dataset methods that take virtual row blocks.
 
 == Why Virtual Rows
 
-Virtual Rows were created to work around the issue that some parts of
-Sequel's standard DSL could not be used on ruby 1.9.  For example, the
-following Sequel code historically worked on ruby 1.8, but not ruby 1.9:
+Virtual rows offer a less verbose way to express many queries.  For example,
+by default if you want to express an inequality filter in Sequel, you can do:
 
-  dataset.where(:a > :b[:c])
-  # WHERE a > b(c)
-  
-This code does not work on ruby 1.9 for two reasons.  First, Symbol#>
-(like other inequality methods) is already defined in ruby 1.9, so Sequel
-does not override it to return an SQL inequality expression.  Second, Symbol#[]
-is already defined on ruby 1.9, so Sequel does not override it to return an
-SQL function expression.
-
-It's possible to use Sequel's DSL to represent such expressions, but it is a
-little verbose:
-
-  dataset.where(Sequel.expr(:a) > Sequel.function(:b, :c))
-  # WHERE a > b(c)
+  dataset.where(Sequel[:a] > Sequel.function(:b, :c))
+  # WHERE (a > b(c))
 
-The virtual row DSL makes such code more concise:
+With virtual rows, you can use the less verbose:
 
   dataset.where{a > b(c)}
+  # WHERE (a > b(c))
 
 == Regular Procs vs Instance Evaled Procs
 
@@ -38,96 +26,110 @@ of Sequel::SQL::VirtualRow.  If it does not accept an argument, it is
 evaluated in the context of an instance of Sequel::SQL::VirtualRow.
 
   ds = DB[:items]
-  # Regular proc
+  # Regular block
   ds.where{|o| o.column > 1}
-  # WHERE column > 1
+  # WHERE (column > 1)
   
-  # Instance-evaled proc
+  # Instance-evaled block
   ds.where{column > 1}
-  # WHERE column > 1
+  # WHERE (column > 1)
   
 If you aren't familiar with the difference between regular blocks and instance
-evaled blocks, you should probably consult a general ruby reference, but briefly,
-with regular procs, methods called without an explicit receiver inside the
-proc call the method on the receiver in the surrounding scope, while instance
-evaled procs call the method on the receiver of the instance_eval call.  However,
+evaled blocks, inside regular blocks methods called without an explicit receiver call
+the method on the receiver in the surrounding scope, while instance
+evaled blocks call the method on the receiver of the instance_eval call (the
+Sequel::SQL::VirtualRow instance in this case).
+
 in both cases, local variables available in the surrounding scope will be available
-inside the proc.  If that doesn't make sense, maybe this example will help:
+inside the block.  However, instance variables in the surrounding scope will not
+be available inside the block if using an instance evaled block, and methods called
+without an explicit receiver inside an instance evaled block will not call
+methods in the surrounding scope. For example:
 
   def self.a
     42
   end
   b = 32
+  @d = 100
   
-  # Regular proc
-  ds.where{|o| o.c > a - b}
-  # WHERE c > 10
+  # Regular block
+  ds.where{|o| o.c > a - b + @d}
+  # WHERE (c > 100)
   
-  # Instance-evaled proc
-  ds.where{c > a - b}
-  # WHERE c > (a - 32)
+  # Instance-evaled block
+  ds.where{c > a - b + @d}
+  # WHERE (c > ((a - 32) + NULL))
   
-There are two related differences here.  First is the usage of <tt>o.c</tt> vs +c+,
-and second is the difference between the use of +a+.  In the regular proc,
-you couldn't call +c+ without an explicit receiver in the proc, unless the self of the
-surrounding scope responded to it.  For +a+, note how ruby calls the method on
-the receiver of the surrounding scope in the regular proc, which returns an integer,
+There are three related differences here:
+
+* Regular blocks use +o.c+ instead of just +c+
+* +a+ results in 42 in the regular block, but creates an expression object in the instance evaled block
+* @d results in 100 in the regular block, but nil in the instance evaled block
+
+In the regular block, you need to call +c+ with an explicit receiver (the virtual
+row block argument), while in the instance evaled block +c+ can be called directly,
+as the default receiver has changed inside the block.
+
+For +a+, note how ruby calls the method on
+the receiver of the surrounding scope in the regular block, which returns an integer,
 and does the subtraction before Sequel gets access to it.  In the instance evaled
-proc, calling +a+ without a receiver calls the a method on the VirtualRow instance.
+block, calling +a+ without a receiver calls the a method on the VirtualRow instance.
+For @d, note that in a regular block, the value hasn't changed, but in the instance evaled
+block, instance variable access returns nil.
 For +b+, note that it operates the same in both cases, as it is a local variable.
 
-Basically, the choice for whether to use a regular proc or an instance evaled proc is
-completely up to you.  The same things can be accomplished with both.
-Instance evaled procs tend to produce shorter code, but by modifying the scope
-can be more difficult for a new user to understand.  That being said, I usually
-use instance evaled procs unless I need to call methods on the receiver of the
-surrounding scope inside the proc.
+The choice for whether to use a regular block or an instance evaled block is
+up to you.  The same things can be accomplished with both.
+Instance evaled block tend to produce shorter code, but by modifying the scope
+can be more difficult to understand.
+
+If you are not sure which to use, use instance evaled blocks unless you need to
+call methods or access instance variables of the surrounding scope inside the block.
 
 == Local Variables vs Method Calls
 
 If you have a method that accepts 0 arguments and has the same name as a local
 variable, you can call it with () to differentiate the method call from the
-local variable access.  This is mostly useful in instance_evaled procs:
+local variable access.  This is mostly useful in instance evaled blocks:
 
   b = 32
   ds.where{b() > b}
   # WHERE b > 32
 
+It's also possible to use an explicit self receiver in instance evaled blocks:
+
+  b = 32
+  ds.where{self.b > b}
+  # WHERE b > 32
+
+
 == VirtualRow Methods
 
-VirtualRow is a class that returns SQL::Identifiers, SQL::QualifiedIdentifiers, or
-SQL::Functions depending on how it is called.
+VirtualRow is a class that returns SQL::Identifiers or SQL::Functions depending
+on how it is called.
 
 == SQL::Identifiers - Regular columns
 
 SQL::Identifiers can be thought of as regular column references in SQL,
 not qualified by any table. You get an SQL::Identifier if the method is called
-without a block or arguments, and doesn't have a double underscore in the method
-name:
+without arguments:
 
   ds.where{|o| o.column > 1}
   ds.where{column > 1}
-  # WHERE column > 1
+  # WHERE (column > 1)
   
 == SQL::QualifiedIdentifiers - Qualified columns
 
-SQL::QualifiedIdentifiers can be thought of as column references in SQL that
-are qualified to a specific table.  You get an SQL::QualifiedIdentifier if
-the method is called without a block or arguments, and has a double underscore
-in the method name:
+You can qualified identifiers by calling #[] on an identifier:
 
-  ds.where{|o| o.table__column > 1}
-  ds.where{table__column > 1}
+  ds.where{|o| o.table[:column] > 1}
+  ds.where{table[:column] > 1}
   # WHERE table.column > 1
 
-Using the double underscore for SQL::QualifiedIdentifiers was done to make
-usage very similar to using symbols, which also translate the double underscore
-into a qualified column.
-
 == SQL::Functions - SQL function calls
 
 SQL::Functions can be thought of as function calls in SQL.  You get a simple
-function call if you call a method with arguments and without a block:
+function call if you call a method with arguments:
 
   ds.where{|o| o.function(1) > 1}
   ds.where{function(1) > 1}
@@ -167,8 +169,7 @@ distinct method on the returned Function:
   
 == SQL::Functions with windows - SQL window function calls
 
-Not all databases support window functions, but they are very helpful for certain types of
-queries.  To use them, you should just call the over method on the Function
+To create a window function call, just call the over method on the Function
 object returned, with the options for the window:
 
   ds.select{|o| o.rank.function.over}
@@ -179,19 +180,18 @@ object returned, with the options for the window:
   ds.select{count.function.*.over}
   # SELECT count(*) OVER ()
   
-  ds.select{|o| o.sum(o.col1).over(:partition=>o.col2, :order=>o.col3)}
-  ds.select{sum(col1).over(:partition=>col2, :order=>col3)}
+  ds.select{|o| o.sum(o.col1).over(partition: o.col2, order: o.col3)}
+  ds.select{sum(col1).over(partition: col2, order: col3)}
   # SELECT sum(col1) OVER (PARTITION BY col2 ORDER BY col3)
 
 == Operators
 
 VirtualRows use method_missing to handle almost all method calls.  Since the
-objects given by method_missing are SQL::Identifiers, SQL::QualifiedIdentifiers
-or SQL::Functions, you can use all operators that they provide (see
+objects given by method_missing are SQL::Identifiers or SQL::Functions, you can use all operators that they provide (see
 DatasetFiltering[http://sequel.jeremyevans.net/rdoc/files/doc/dataset_filtering_rdoc.html#label-Filtering+using+expressions]):
 
   ds.select{|o| o.price - 100}
-  ds.select{o.price - 100}
+  ds.select{price - 100}
   # SELECT (price - 100)
 
   ds.where{|o| (o.price < 200) & (o.tax * 100 >= 23)}
@@ -213,14 +213,14 @@ The standard +, -, *, and / mathematical operators are defined:
 
 The & and | methods are defined to use AND and OR:
 
-  ds.where{|o| o.&({:a=>:b}, :c)}
-  ds.where{self.&({:a=>:b}, :c)}
+  ds.where{|o| o.&({a: :b}, :c)}
+  ds.where{self.&({a: :b}, :c)}
   # WHERE ((a = b) AND c)
 
 The ~ method is defined to do inversion:
 
-  ds.where{|o| o.~({:a=>1, :b=>2})}
-  ds.where{self.~({:a=>1, :b=>2})}
+  ds.where{|o| o.~({a: 1, b: 2})}
+  ds.where{self.~({a: 1, b: 2})}
   # WHERE ((a != 1) OR (b != 2))
 
 === Inequality Operators
@@ -231,19 +231,6 @@ The standard >, <, >=, and <= inequality operators are defined:
   ds.where{self.>(1, :c)}
   # WHERE (1 > c)
 
-== Literal Strings
-
-The backtick operator can be used inside an instance-evaled
-virtual row block to create a literal string:
-
-  ds.where{a > `some SQL`}
-  # WHERE (a > some SQL)
-
-You can use this on a regular virtual row block too, but it
-doesn't look as nice:
-
-  ds.where{|o| o.>(:a, o.`('some SQL'))}
-
 == Returning multiple values
 
 It's common when using select and order virtual row blocks to want to
@@ -259,26 +246,20 @@ Note that if you forget the array brackets, you'll end up with a syntax error:
   # Invalid ruby syntax
   ds.select{|o| o.column1, o.sum(o.column2).as(o.sum)}
   ds.select{column1, sum(column2).as(sum)}
-  
-== Alternative Description of the VirtualRow method call rules
-
-* If a block is given:
-  * The block is currently not called.  This may change in a future version.
-  * If there are no arguments, an SQL::Function with the name of
-    method used, and no arguments.
-  * If the first argument is :*, an SQL::Function is created with a single
-    wildcard argument (*).
-  * If the first argument is :distinct, an SQL::Function is created with
-    the keyword DISTINCT prefacing all remaining arguments.
-  * If the first argument is :over, the second argument if provided should
-    be a hash of options to pass to SQL::Window.  The options hash can also
-    contain :*=>true to use a wildcard argument as the function argument, or
-    :args=>... to specify an array of arguments to use as the function arguments.
-* If a block is not given:
-  * If there are arguments, an SQL::Function is returned with the
-    name of the method used and the arguments given.
-  * If there are no arguments and the method contains a double
-    underscore, split on the double underscore and return an
-    SQL::QualifiedIdentifier with the table and column.
-  * Otherwise, create an SQL::Identifier with the name of the
-    method.
+ 
+== Split symbols
+
+Note that if you turn on symbol splitting for backwards compatibility,
+Sequel will split virtual row methods with double underscores and
+return them as qualified identifiers:
+
+  Sequel.split_symbols = true
+  ds.where{|o| o.table__column}
+  ds.where{table__column}
+  WHERE table.column
+
+It's not recommended that you rely on this, it's better to convert the calls
+to the recommended form:
+
+  ds.where{|o| o.table[:column]}
+  ds.where{table[:column]}

data/lib/sequel.rb

@@ -1 +1,3 @@
-require 'sequel/model'
+# frozen-string-literal: true
+
+require_relative 'sequel/model'

data/lib/sequel/adapters/ado.rb

@@ -1,20 +1,101 @@
+# frozen-string-literal: true
+
 require 'win32ole'
 
 module Sequel
   # The ADO adapter provides connectivity to ADO databases in Windows.
   module ADO
-    class Database < Sequel::Database
-      DISCONNECT_ERROR_RE = /Communication link failure/
+    # ADO constants (DataTypeEnum)
+    # Source: https://msdn.microsoft.com/en-us/library/ms675318(v=vs.85).aspx
+    AdBigInt           = 20
+    AdBinary           = 128
+    #AdBoolean          = 11
+    #AdBSTR             = 8
+    #AdChapter          = 136
+    #AdChar             = 129
+    #AdCurrency         = 6
+    #AdDate             = 7
+    AdDBDate           = 133
+    #AdDBTime           = 134
+    AdDBTimeStamp      = 135
+    #AdDecimal          = 14
+    #AdDouble           = 5
+    #AdEmpty            = 0
+    #AdError            = 10
+    #AdFileTime         = 64
+    #AdGUID             = 72
+    #AdIDispatch        = 9
+    #AdInteger          = 3
+    #AdIUnknown         = 13
+    AdLongVarBinary    = 205
+    #AdLongVarChar      = 201
+    #AdLongVarWChar     = 203
+    AdNumeric          = 131
+    #AdPropVariant      = 138
+    #AdSingle           = 4
+    #AdSmallInt         = 2
+    #AdTinyInt          = 16
+    #AdUnsignedBigInt   = 21
+    #AdUnsignedInt      = 19
+    #AdUnsignedSmallInt = 18
+    #AdUnsignedTinyInt  = 17
+    #AdUserDefined      = 132
+    AdVarBinary        = 204
+    #AdVarChar          = 200
+    #AdVariant          = 12
+    AdVarNumeric       = 139
+    #AdVarWChar         = 202
+    #AdWChar            = 130
+
+    bigint = Object.new
+    def bigint.call(v)
+      v.to_i
+    end
 
+    numeric = Object.new
+    def numeric.call(v)
+      if v.include?(',')
+        BigDecimal(v.tr(',', '.'))
+      else
+        BigDecimal(v)
+      end
+    end
+
+    binary = Object.new
+    def binary.call(v)
+      Sequel.blob(v.pack('c*'))
+    end
+
+    date = Object.new
+    def date.call(v)
+      Date.new(v.year, v.month, v.day)
+    end
+
+    CONVERSION_PROCS = {}
+    [
+      [bigint, AdBigInt],
+      [numeric, AdNumeric, AdVarNumeric],
+      [date, AdDBDate],
+      [binary, AdBinary, AdVarBinary, AdLongVarBinary]
+    ].each do |callable, *types|
+      callable.freeze
+      types.each do |i|
+        CONVERSION_PROCS[i] = callable
+      end
+    end
+    CONVERSION_PROCS.freeze
+
+    class Database < Sequel::Database
       set_adapter_scheme :ado
 
+      attr_reader :conversion_procs
+
       # In addition to the usual database options,
       # the following options have an effect:
       #
       # :command_timeout :: Sets the time in seconds to wait while attempting
       #                     to execute a command before cancelling the attempt and generating
       #                     an error. Specifically, it sets the ADO CommandTimeout property.
-      #                     If this property is not set, the default of 30 seconds is used.
       # :driver :: The driver to use in the ADO connection string.  If not provided, a default
       #            of "SQL Server" is used.
       # :conn_string :: The full ADO connection string.  If this is provided,
@@ -26,8 +107,8 @@ module Sequel
       #
       # Pay special attention to the :provider option, as without specifying a provider,
       # many things will be broken.  The SQLNCLI10 provider appears to work well if you
-      # are connecting to Microsoft SQL Server, but it is not the default as that would
-      # break backwards compatability.
+      # are connecting to Microsoft SQL Server, but it is not the default as that is not
+      # always available and would break backwards compatability.
       def connect(server)
         opts = server_opts(server)
         s = opts[:conn_string] || "driver=#{opts[:driver]};server=#{opts[:host]};database=#{opts[:database]}#{";uid=#{opts[:user]};pwd=#{opts[:password]}" if opts[:user]}"
@@ -44,6 +125,11 @@ module Sequel
         nil
       end
 
+      def freeze
+        @conversion_procs.freeze
+        super
+      end
+
       # Just execute so it doesn't attempt to return the number of rows modified.
       def execute_ddl(sql, opts=OPTS)
         execute(sql, opts)
@@ -62,7 +148,7 @@ module Sequel
         return super if opts[:provider]
         synchronize(opts[:server]) do |conn|
           begin
-            log_yield(sql){conn.Execute(sql, 1)}
+            log_connection_yield(sql, conn){conn.Execute(sql, 1)}
             WIN32OLE::ARGV[1]
           rescue ::WIN32OLERuntimeError => e
             raise_error(e)
@@ -73,8 +159,15 @@ module Sequel
       def execute(sql, opts=OPTS)
         synchronize(opts[:server]) do |conn|
           begin
-            r = log_yield(sql){conn.Execute(sql)}
-            yield(r) if block_given?
+            r = log_connection_yield(sql, conn){conn.Execute(sql)}
+            begin
+              yield r if block_given?
+            ensure
+              begin
+                r.close
+              rescue ::WIN32OLERuntimeError
+              end
+            end
           rescue ::WIN32OLERuntimeError => e
             raise_error(e)
           end
@@ -87,22 +180,29 @@ module Sequel
       def adapter_initialize
         case @opts[:conn_string]
         when /Microsoft\.(Jet|ACE)\.OLEDB/io
-          Sequel.require 'adapters/ado/access'
+          require_relative 'ado/access'
           extend Sequel::ADO::Access::DatabaseMethods
           self.dataset_class = ADO::Access::Dataset
         else
           @opts[:driver] ||= 'SQL Server'
           case @opts[:driver]
           when 'SQL Server'
-            Sequel.require 'adapters/ado/mssql'
+            require_relative 'ado/mssql'
             extend Sequel::ADO::MSSQL::DatabaseMethods
             self.dataset_class = ADO::MSSQL::Dataset
             set_mssql_unicode_strings
           end
         end
+
+        @conversion_procs = CONVERSION_PROCS.dup
+
         super
       end
 
+      def dataset_class_default
+        Dataset
+      end
+
       # The ADO adapter's default provider doesn't support transactions, since it 
       # creates a new native connection for each query.  So Sequel only attempts
       # to use transactions if an explicit :provider is given.
@@ -119,7 +219,7 @@ module Sequel
       end
 
       def disconnect_error?(e, opts)
-        super || (e.is_a?(::WIN32OLERuntimeError) && e.message =~ DISCONNECT_ERROR_RE)
+        super || (e.is_a?(::WIN32OLERuntimeError) && e.message =~ /Communication link failure/)
       end
 
       def rollback_transaction(conn, opts=OPTS)
@@ -128,21 +228,53 @@ module Sequel
     end
     
     class Dataset < Sequel::Dataset
-      Database::DatasetClass = self
-
       def fetch_rows(sql)
-        execute(sql) do |s|
-          columns = cols = s.Fields.extend(Enumerable).map{|column| output_identifier(column.Name)}
-          @columns = columns
-          s.getRows.transpose.each do |r|
-            row = {}
-            cols.each{|c| row[c] = r.shift}
-            yield row
-          end unless s.eof
+        execute(sql) do |recordset|
+          cols = []
+          conversion_procs = db.conversion_procs
+
+          ts_cp = nil
+          recordset.Fields.each do |field|
+            type = field.Type
+            cp = if type == AdDBTimeStamp
+              ts_cp ||= begin
+                nsec_div = 1000000000.0/(10**(timestamp_precision))
+                nsec_mul = 10**(timestamp_precision+3)
+                meth = db.method(:to_application_timestamp)
+                lambda do |v|
+                  # Fractional second handling is not correct on ruby <2.2
+                  meth.call([v.year, v.month, v.day, v.hour, v.min, v.sec, (v.nsec/nsec_div).round * nsec_mul])
+                end
+              end
+            else
+              conversion_procs[type]
+            end
+            cols << [output_identifier(field.Name), cp]
+          end
+
+          self.columns = cols.map(&:first)
+          return if recordset.EOF
+          max = cols.length
+
+          recordset.GetRows.transpose.each do |field_values|
+            h = {}
+
+            i = -1
+            while (i += 1) < max
+              name, cp = cols[i]
+              h[name] = if (v = field_values[i]) && cp
+                cp.call(v)
+              else
+                v
+              end
+            end
+            
+            yield h
+          end
         end
       end
       
-      # ADO returns nil for all for delete and update statements.
+      # ADO can return for for delete and update statements, depending on the provider.
       def provides_accurate_rows_matched?
         false
       end