sequel 4.46.0 → 4.49.0

data/doc/release_notes/4.49.0.txt

@@ -0,0 +1,222 @@
+= Forward Compatibility
+
+Sequel 4.49.0 will be the last minor release of Sequel 4. While the
+vast majority of backwards incompatible changes in Sequel 5 have
+deprecation warnings in 4.49.0, there are a few changes that do
+not.  Here is a brief list of changes coming in Sequel 5 that
+do not have deprecation warnings (note that this list may not be
+exhaustive):
+
+* The {before,after,around}_validation hooks will always be called
+  when saving, even if the validate: false option is used.  This
+  will allow you to use the before_validation hook to make changes
+  to the model instance that are required before validation and
+  before saving even if not validating.  Currently, you would have
+  to use both a before_save and before_validation hook, which would
+  both be run on normal instance saving.
+
+* Getting values for newly created model instances after insertion
+  now happens before after_create is called, instead of after.
+  This behavior is currently available via the before_after_save
+  plugin, and and will become the default behavior.
+
+* Sequel will now immediately attempt to the connect to the database
+  when a Database instance is created, in order to fail fast.  This
+  behavior is currently available via the test: true option, and
+  will become the default behavior.  You can force not testing the
+  connection by using the test: false option.
+
+* The validates_unique method in the validation_helpers plugin will
+  now only check for uniqueness by default if the record is new or
+  one of the related columns has been modified by default.  You can
+  use only_if_modified: false to force the uniqueness check.
+
+* Database schema methods and schema generator methods will return
+  nil instead of some internal value.
+
+* Many cases where Sequel uses send internally will be switched to
+  public_send so they only call public methods, unless it is
+  specifically expected that they will call private methods.
+
+* Model association hooks will be nil instead of empty arrays by
+  default.  They will only be arrays if that hook has been set for
+  the association.
+
+* Internal uses of instance_eval with a block will be changed to
+  instance_exec.  This will allow them to be used with lambdas that
+  take no arguments.  Unfortunately, it will break the case where a
+  lambda is currently used that takes one argument.
+
+* Most internal constants will be frozen, unless there is a
+  requirement that they be modified at runtime.
+
+* The @was_new instance variable set during model instance creation
+  will be removed.
+
+= Deprecated Features
+
+* Model association before callbacks returning false canceling the
+  action is now deprecated.  The callbacks should now call
+  Model#cancel_action to cancel the action.
+
+* Loading plugins by requiring them via sequel_#{plugin} is now
+  deprecated.  Affected plugins should move the plugin file so it can
+  be required via sequel/plugins/#{plugin}.
+
+* In the mock adapter, Dataset#autoid=, #_fetch=, and #numrows= are
+  now deprecated.  They modified the dataset itself, which would not
+  work for frozen datasets.  Dataset#with_autoid, #with_fetch,
+  and #with_numrows should be used instead, which return a modified
+  copy.
+
+* In the null_dataset extension, Dataset#nullify! is now deprecated.
+  It modified the dataset itself, which would not work for frozen
+  datasets.  Dataset#nullify should be used instead, which returns a
+  modified copy.
+
+* Modifying the validation_helpers plugin DEFAULT_OPTIONS hash is now
+  deprecated.  Any change to the default options should be done by
+  overriding the Model#default_validation_helpers_options private
+  method.
+
+* Modifying ConnectionPool::CONNECTION_POOL_MAP to support an
+  external connection pool is now deprecated.  To use an external
+  connection pool, pass the pool class via the :pool_class
+  Database option.  Additionally, using a :pool_class option that
+  is not a class or a symbol for one of the default connection
+  pools is also deprecated.
+
+* ConnectionPool#created_count is now deprecated.  This method was
+  misnamed, as it was in alias to size, but the name implies it
+  returns how many connections have been created, as opposed to how
+  many connections are still in the pool.
+
+* Sequel::SQL::Function#f is now deprecated, switch to using #name
+  instead.
+
+* Sequel::SQL::AliasedExpression#aliaz is now deprecated, switch
+  to using #alias instead.
+
+* The :eager_loading_predicate_key association option and
+  eager_loading_predicate_key association method are now deprecated.
+  The predicate_key option and method should be used instead.
+
+* The cti_columns class method in the class_table_inheritance plugin
+  is now deprecated.
+
+* The serialized_columns class method in the serialization plugin
+  is now deprecated.
+
+* Having ds.join_table(:table, :cross, :a=>:b) be treated as an
+  inner join on MySQL is now deprecated.
+
+* Sequel::IBMDB::Connection#prepared_statements= in the ibmdb
+  adapter is now deprecated.
+
+* Additional internal constants are now deprecated.
+
+= New Features
+
+* Database#extend_datasets and Database#with_extend if given a block
+  now use a Dataset::DatasetModule instance instead of a plain Module
+  instance.  Dataset::DatasetModule is a subset of
+  Model::DatasetModule, and allows for the easy creation of dataset
+  methods that can perform caching for frozen datasets.
+
+  Defining dataset methods is done by calling methods with the same
+  name as dataset methods inside the extend_datasets or with_extend
+  block:
+
+    DB.extend_datasets do
+      order :by_id, :id
+      select :with_id_and_name, :id, :name
+      where :active, :active
+    end
+
+  This is equivalent to:
+
+    DB.extend_datasets do
+      def by_id
+        order(:id)
+      end
+
+      def with_id_and_name
+        select(:id, :name)
+      end
+
+      def active
+        where(:active)
+      end
+    end
+
+  Except that for frozen datasets (the default in Sequel 5),
+  code like:
+
+    100.times do
+      DB[:table].active.with_id_and_name.by_id
+    end
+
+  will only allocate 4 datasets instead of 400, and can be
+  3-4 times faster.
+
+* Dataset#where_{all,each,single_value} are now core dataset methods
+  instead of just model dataset methods.  These methods allow you to
+  replace:
+
+    dataset.where(cond).all
+    dataset.where(cond).each{}
+    dataset.where(cond).single_value
+
+  with:
+
+    dataset.where_all(cond)
+    dataset.where_each(cond){}
+    dataset.where_single_value(cond)
+
+  The advantage of #where_{all,each,single_value} is that frozen
+  datasets can take potentially advantage of caching and perform
+  70%-300% faster.
+
+* Oracle 12 native limit/offset support is now supported, which
+  in particular makes offset queries much faster as they don't
+  have to be emulated using the row_number window function.
+
+* Dataset#paged_each in the mysql2 adapter now supports a
+  :stream=>false option to disable streaming and fallback to
+  the default implementation.
+  
+* The postgres adapter now supports the :sslrootcert option
+  directly, you no longer need to specify it using the
+  :driver_options hash.
+
+* The single_table_inheritance plugin now supports an
+  sti_class_from_sti_key method for getting the appropriate
+  subclass for the given key.
+
+= Other Improvements
+
+* Using the dataset_associations plugin with a many_through_many
+  association that joins to the same table multiple times is now
+  handled correctly by automatically aliasing the table
+  appropriately.
+
+* On Ruby 2.1+, Sequel::Error#cause will use wrapped_exception
+  if one is set.  This doesn't result in different behavior in
+  most cases, but it can in cases where nested exception handling
+  is done and Sequel tries to raise the most relevant exception.
+
+* Using the composition plugin with the :mapping option now works
+  correctly when using the column_conflicts plugin.
+
+* The validation_helpers plugin's validates_max_length method
+  now correctly gets the default :nil_message option from
+  the default_validation_helpers_options method instead of
+  looking at the plugin defaults.
+
+* The duplicate_columns_handler extension no longer makes the
+  Dataset#columns= method public.
+
+* On H2 1.4+, alter_table add_primary_key now works correctly.
+
+* The jdbc/sqlserver adapter's datetimeoffset type handling now
+  works with more JDBC driver versions.

data/lib/sequel/adapters/ado/access.rb

@@ -299,7 +299,8 @@ module Sequel
         
         def fetch_ado_schema(type, criteria=[])
           execute_open_ado_schema(type, criteria) do |s|
-            cols = s.Fields.extend(Enumerable).map(&:Name)
+            cols = []
+            s.Fields.each{|f| cols << f.Name}
             s.getRows.transpose.each do |r|
               row = {}
               cols.each{|c| row[c] = r.shift}

data/lib/sequel/adapters/do/postgres.rb

@@ -4,8 +4,6 @@ Sequel::DataObjects.load_driver 'do_postgres'
 Sequel.require 'adapters/shared/postgres'
 
 module Sequel
-  Postgres::CONVERTED_EXCEPTIONS << ::DataObjects::Error
-  
   module DataObjects
     Sequel.synchronize do
       DATABASE_SETUP[:postgres] = proc do |db|
@@ -31,6 +29,11 @@ module Sequel
         
         private
         
+        DATABASE_ERROR_CLASSES = [::DataObjects::Error].freeze
+        def database_error_classes
+          DATABASE_ERROR_CLASSES
+        end
+
         # Extend the adapter with the DataObjects PostgreSQL AdapterMethods
         def setup_connection(conn)
           conn = super(conn)

data/lib/sequel/adapters/ibmdb.rb

@@ -6,12 +6,17 @@ Sequel.require 'adapters/shared/db2'
 module Sequel
 
   module IBMDB
+    # SEQUEL5: Remove
     @convert_smallint_to_bool = true
-
     class << self
-      # Whether to convert smallint values to bool, true by default.
-      # Can also be overridden per dataset.
-      attr_accessor :convert_smallint_to_bool
+      def convert_smallint_to_bool
+        Sequel::Deprecation.deprecate("Sequel::IBMDB.convert_smallint_to_bool", "Call this method on the Database instance")
+        @convert_smallint_to_bool
+      end
+      def convert_smallint_to_bool=(v)
+        Sequel::Deprecation.deprecate("Sequel::IBMDB.convert_smallint_to_bool=", "Call this method on the Database instance")
+        @convert_smallint_to_bool = v
+      end
     end
 
     tt = Class.new do
@@ -32,7 +37,12 @@ module Sequel
     class Connection
       # A hash with prepared statement name symbol keys, where each value is 
       # a two element array with an sql string and cached Statement value.
-      attr_accessor :prepared_statements
+      attr_reader :prepared_statements
+
+      def prepared_statements=(v)
+        Sequel::Deprecation.deprecate("Sequel::IBMDB::Connection#prepared_statements=", "Use replace on the hash instead of reassigning it")
+        @prepared_statements = v
+      end
 
       # Error class for exceptions raised by the connection.
       class Error < StandardError
@@ -187,6 +197,16 @@ module Sequel
 
       # Hash of connection procs for converting
       attr_reader :conversion_procs
+
+      # Whether to convert smallint values to bool for this Database instance
+      #attr_accessor :convert_smallint_to_bool # SEQUEL5
+
+      # SEQUEL5: Remove
+      attr_writer :convert_smallint_to_bool
+      def convert_smallint_to_bool
+        v = @convert_smallint_to_bool
+        v.nil? ? Sequel::IBMDB.instance_variable_get(:@convert_smallint_to_bool) : v
+      end
     
       # Create a new connection object for the given server.
       def connect(server)
@@ -284,6 +304,7 @@ module Sequel
       end
 
       def adapter_initialize
+        #@convert_smallint_to_bool = typecast_value_boolean(opts.fetch(:convert_smallint_to_bool, true)) # SEQUEL5
         @conversion_procs = DB2_TYPES.dup
         @conversion_procs[:timestamp] = method(:to_application_timestamp)
       end
@@ -349,7 +370,7 @@ module Sequel
 
       # Convert smallint type to boolean if convert_smallint_to_bool is true
       def schema_column_type(db_type)
-        if Sequel::IBMDB.convert_smallint_to_bool && db_type =~ /smallint/i 
+        if convert_smallint_to_bool && db_type =~ /smallint/i 
           :boolean
         else
           super
@@ -379,13 +400,14 @@ module Sequel
 
       # Override the default IBMDB.convert_smallint_to_bool setting for this dataset.
       def convert_smallint_to_bool=(v)
+        Sequel::Deprecation.deprecate("Sequel::IBMDB::Dataset#convert_smallint_to_bool=", "Call with_convert_smallint_to_bool instead, which returns a modified copy instead of modifying the object")
         @opts[:convert_smallint_to_bool] = v
       end
 
       # Whether to convert smallint to boolean arguments for this dataset.
       # Defaults to the IBMDB module setting.
       def convert_smallint_to_bool
-        opts.has_key?(:convert_smallint_to_bool) ? opts[:convert_smallint_to_bool] : IBMDB.convert_smallint_to_bool
+        opts.has_key?(:convert_smallint_to_bool) ? opts[:convert_smallint_to_bool] : db.convert_smallint_to_bool
       end
 
       # Return a cloned dataset with the convert_smallint_to_bool option set.
@@ -405,7 +427,7 @@ module Sequel
             type = stmt.field_type(i).downcase.to_sym
             # decide if it is a smallint from precision
             type = :boolean  if type == :int && convert && stmt.field_precision(i) < 8
-            type = :blob if type == :clob && Sequel::DB2.use_clob_as_blob
+            type = :blob if type == :clob && db.use_clob_as_blob
             columns << [key, cps[type]]
           end
           cols = columns.map{|c| c[0]}

data/lib/sequel/adapters/jdbc/as400.rb

@@ -1,6 +1,6 @@
 # frozen-string-literal: true
 
-Sequel::Deprecation.deprecate("The jdbc/as400 adapter", "Please consider maintaining it yourself as an external gem if you want to continue using it")
+Sequel::Deprecation.deprecate("The jdbc/as400 adapter", "This gem will replace it: https://github.com/ecraft/sequel-jdbc-as400")
 
 Sequel::JDBC.load_driver('com.ibm.as400.access.AS400JDBCDriver')
 Sequel.require 'adapters/jdbc/transactions'

data/lib/sequel/adapters/jdbc/db2.rb

@@ -35,7 +35,8 @@ module Sequel
       end
     end
 
-    class TypeConvertor
+    # SEQUEL5: Remove
+    class Type_Convertor
       def DB2Clob(r, i)
         if v = r.getClob(i)
           v = v.getSubString(1, v.length)
@@ -63,7 +64,7 @@ module Sequel
         def set_ps_arg(cps, arg, i)
           case arg
           when Sequel::SQL::Blob
-            if ::Sequel::DB2.use_clob_as_blob
+            if use_clob_as_blob
               cps.setString(i, arg)
             else
               super
@@ -91,7 +92,15 @@ module Sequel
           super
           map = @type_convertor_map
           types = Java::JavaSQL::Types
-          map[types::NCLOB] = map[types::CLOB] = TypeConvertor::INSTANCE.method(:DB2Clob)
+          map[types::NCLOB] = map[types::CLOB] = method(:convert_clob)
+        end
+
+        def convert_clob(r, i)
+          if v = r.getClob(i)
+            v = v.getSubString(1, v.length)
+            v = Sequel::SQL::Blob.new(v) if use_clob_as_blob
+            v
+          end
         end
       end
 

data/lib/sequel/adapters/jdbc/derby.rb

@@ -54,7 +54,7 @@ module Sequel
           end
         end
         
-        # Derby supports transaction DDL statements.
+        # Derby supports transactional DDL statements.
         def supports_transactional_ddl?
           true
         end
@@ -227,8 +227,7 @@ module Sequel
         Sequel::Deprecation.deprecate_constant(self, :EMULATED_FUNCTION_MAP)
 
         # Derby doesn't support an expression between CASE and WHEN,
-        # so remove 
-        # conditions.
+        # so remove conditions.
         def case_expression_sql_append(sql, ce)
           super(sql, ce.with_merged_expression)
         end
@@ -300,7 +299,7 @@ module Sequel
         end
 
         # Derby uses an expression yielding false for false values.
-        # Newer versions can use the FALSE literal, but the latest gem version cannot.
+        # Newer versions can use the FALSE literal, but older versions cannot.
         def literal_false
           if db.svn_version >= 1040133
             'FALSE'
@@ -315,7 +314,7 @@ module Sequel
         end
 
         # Derby uses an expression yielding true for true values.
-        # Newer versions can use the TRUE literal, but the latest gem version cannot.
+        # Newer versions can use the TRUE literal, but older versions cannot.
         def literal_true
           if db.svn_version >= 1040133
             'TRUE'

data/lib/sequel/adapters/jdbc/h2.rb

@@ -30,6 +30,15 @@ module Sequel
           :h2
         end
 
+        def freeze
+          h2_version
+          super
+        end
+
+        def h2_version
+          @h2_version ||= get(Sequel.function(:H2VERSION))
+        end
+
         # Rollback an existing prepared transaction with the given transaction
         # identifier string.
         def rollback_prepared_transaction(transaction_id, opts=OPTS)
@@ -84,7 +93,7 @@ module Sequel
 
               sqls = [super(table, op)]
 
-              if pk && op[:type] != :identity
+              if pk && (h2_version >= '1.4' || op[:type] != :identity)
                 sqls << "ALTER TABLE #{quote_schema_table(table)} ADD PRIMARY KEY (#{quote_identifier(op[:name])})"
               end
 

data/lib/sequel/adapters/jdbc/oracle.rb

@@ -14,7 +14,8 @@ module Sequel
       end
     end
 
-    class TypeConvertor
+    # SEQUEL5: Remove
+    class Type_Convertor
       JAVA_BIG_DECIMAL_CONSTRUCTOR = java.math.BigDecimal.java_class.constructor(Java::long).method(:new_instance)
 
       def OracleDecimal(r, i)
@@ -31,6 +32,19 @@ module Sequel
 
     # Database and Dataset support for Oracle databases accessed via JDBC.
     module Oracle
+      JAVA_BIG_DECIMAL_CONSTRUCTOR = java.math.BigDecimal.java_class.constructor(Java::long).method(:new_instance)
+
+      def self.OracleDecimal(r, i)
+        if v = r.getBigDecimal(i)
+          i = v.long_value
+          if v == JAVA_BIG_DECIMAL_CONSTRUCTOR.call(i)
+            i
+          else
+            BigDecimal.new(v.to_string)
+          end
+        end
+      end 
+
       # Instance methods for Oracle Database objects accessed via JDBC.
       module DatabaseMethods
         PRIMARY_KEY_INDEX_RE = /\Asys_/i.freeze
@@ -110,7 +124,7 @@ module Sequel
 
         def setup_type_convertor_map
           super
-          @type_convertor_map[:OracleDecimal] = TypeConvertor::INSTANCE.method(:OracleDecimal)
+          @type_convertor_map[:OracleDecimal] = Oracle.method(:OracleDecimal)
         end
       end
       

data/lib/sequel/adapters/jdbc/postgresql.rb

@@ -4,8 +4,6 @@ Sequel::JDBC.load_driver('org.postgresql.Driver', :Postgres)
 Sequel.require 'adapters/shared/postgres'
 
 module Sequel
-  Postgres::CONVERTED_EXCEPTIONS << NativeException
-  
   module JDBC
     Sequel.synchronize do
       DATABASE_SETUP[:postgresql] = proc do |db|
@@ -15,11 +13,27 @@ module Sequel
       end
     end
 
-    class TypeConvertor
+    # SEQUEL5: Remove
+    class Type_Convertor
+      def RubyPGArray(r, i)
+        if v = r.getArray(i)
+          v.array.to_ary
+        end
+      end 
+      def RubyPGHstore(r, i)
+        if v = r.getObject(i)
+          v.to_hash
+        end
+      end 
+    end
+
+    # Adapter, Database, and Dataset support for accessing a PostgreSQL
+    # database via JDBC.
+    module Postgres
       # Return PostgreSQL array types as ruby Arrays instead of
       # JDBC PostgreSQL driver-specific array type. Only used if the
       # database does not have a conversion proc for the type.
-      def RubyPGArray(r, i)
+      def self.RubyPGArray(r, i)
         if v = r.getArray(i)
           v.array.to_ary
         end
@@ -28,16 +42,12 @@ module Sequel
       # Return PostgreSQL hstore types as ruby Hashes instead of
       # Java HashMaps.  Only used if the database does not have a
       # conversion proc for the type.
-      def RubyPGHstore(r, i)
+      def self.RubyPGHstore(r, i)
         if v = r.getObject(i)
           v.to_hash
         end
       end 
-    end
 
-    # Adapter, Database, and Dataset support for accessing a PostgreSQL
-    # database via JDBC.
-    module Postgres
       # Methods to add to Database instances that access PostgreSQL via
       # JDBC.
       module DatabaseMethods
@@ -50,6 +60,12 @@ module Sequel
           db.send(:initialize_postgres_adapter)
         end
 
+        # Remove any current entry for the oid in the oid_convertor_map.
+        def add_conversion_proc(oid, *)
+          super
+          Sequel.synchronize{@oid_convertor_map.delete(oid)}
+        end
+
         # See Sequel::Postgres::Adapter#copy_into
         def copy_into(table, opts=OPTS)
           data = opts[:data]
@@ -61,7 +77,7 @@ module Sequel
             raise Error, "Must provide either a :data option or a block to copy_into"
           end
 
-          synchronize(opts) do |conn|
+          synchronize(opts[:server]) do |conn|
             begin
               copy_manager = org.postgresql.copy.CopyManager.new(conn)
               copier = copy_manager.copy_in(copy_into_sql(table, opts))
@@ -73,7 +89,7 @@ module Sequel
                 data.each { |d| copier.writeToCopy(d.to_java_bytes, 0, d.length) }
               end
             rescue Exception => e
-              copier.cancelCopy
+              copier.cancelCopy if copier
               raise
             ensure
               unless e
@@ -105,8 +121,12 @@ module Sequel
                 end
                 b
               end
+            rescue => e
+              raise_error(e, :disconnect=>true)
             ensure
-              raise DatabaseDisconnectError, "disconnecting as a partial COPY may leave the connection in an unusable state" if buf
+              if buf && !e
+                raise DatabaseDisconnectError, "disconnecting as a partial COPY may leave the connection in an unusable state"
+              end
             end
           end
         end
@@ -122,19 +142,24 @@ module Sequel
             else
               false
             end
-             Sequel.synchronize{@oid_convertor_map[oid] = conv}
+            Sequel.synchronize{@oid_convertor_map[oid] = conv}
           end
           conv
         end
 
         private
         
-        # Clear oid convertor map cache when conversion procs are updated.
+        # SEQUEL5: Remove
         def conversion_procs_updated
           super
           Sequel.synchronize{@oid_convertor_map = {}}
         end
 
+        DATABASE_ERROR_CLASSES = [NativeException].freeze
+        def database_error_classes
+          DATABASE_ERROR_CLASSES
+        end
+
         def disconnect_error?(exception, opts)
           super || exception.message =~ /\A(This connection has been closed\.|FATAL: terminating connection due to administrator command|An I\/O error occurred while sending to the backend\.)\z/
         end
@@ -178,8 +203,6 @@ module Sequel
         def setup_type_convertor_map
           super
           @oid_convertor_map = {}
-          @type_convertor_map[:RubyPGArray] = TypeConvertor::INSTANCE.method(:RubyPGArray)
-          @type_convertor_map[:RubyPGHstore] = TypeConvertor::INSTANCE.method(:RubyPGHstore)
         end
       end
       
@@ -188,6 +211,8 @@ module Sequel
         include Sequel::Postgres::DatasetMethods
         APOS = "'".freeze
         Sequel::Deprecation.deprecate_constant(self, :APOS)
+        HSTORE_TYPE = 'hstore'.freeze
+        Sequel::Deprecation.deprecate_constant(self, :HSTORE_TYPE)
         
         private
         
@@ -203,8 +228,9 @@ module Sequel
 
         STRING_TYPE = Java::JavaSQL::Types::VARCHAR
         ARRAY_TYPE = Java::JavaSQL::Types::ARRAY
+        ARRAY_METHOD = Postgres.method(:RubyPGArray)
         PG_SPECIFIC_TYPES = [ARRAY_TYPE, Java::JavaSQL::Types::OTHER, Java::JavaSQL::Types::STRUCT]#.freeze # SEQUEL5
-        HSTORE_TYPE = 'hstore'.freeze
+        HSTORE_METHOD = Postgres.method(:RubyPGHstore)
 
         def type_convertor(map, meta, type, i)
           case type
@@ -213,11 +239,11 @@ module Sequel
             if pr = db.oid_convertor_proc(oid)
               pr
             elsif type == ARRAY_TYPE
-              map[:RubyPGArray]
+              ARRAY_METHOD
             elsif oid == 2950 # UUID
               map[STRING_TYPE]
-            elsif meta.getPGType(i) == HSTORE_TYPE
-              map[:RubyPGHstore]
+            elsif meta.getPGType(i) == 'hstore'
+              HSTORE_METHOD
             else
               super
             end