sequel 4.35.0 → 4.36.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 29e8963c69c4f6e3b0af0840e6cc881934ca608f
-  data.tar.gz: 30d2fa8ead89124fa8a4e8a4582a11a78bf7a5c8
+  metadata.gz: bb4a5a3794c2842c41482c01c74e82c2eed2f548
+  data.tar.gz: b4e04bef12f7580b1fa3857a9a6d820ce5e977df
 SHA512:
-  metadata.gz: 2961452f5e6b3f1ab7e32da5e4a4dd92e9362a13bb561999ebd493ab8877132d803b7c3e22206ded6a9c2e8336a9bde344c3e2707cbe81732a115d45383d4186
-  data.tar.gz: 72c885ba7219f83ac043bf213045f7b67ebbf213c4aa8380224c9c9d76fd1cd446f043e77ba0765e34684cf0fd35300b6b966a688f6e631e83b4b556ecb25b76
+  metadata.gz: b35ce656b26c851e5b5e2bf19c4ce5a40e83f6d79d939561a6e2d584b5210f91cbef788dbbb0a6c47154baa91f039eea6fb81053bdb03f016bf2a40a0da0f5eb
+  data.tar.gz: 829169bda7ea5e43e8ef6d6b9726ba950adb033e6a4f992a3fb677cded90ca1ec842bb90632ba21aa15b4c6184db0fb45d3ce37b908e9d8fcd4f1591e13a7954

data/CHANGELOG

@@ -1,3 +1,35 @@
+=== 4.36.0 (2016-07-01)
+
+* Deprecate use of Bignum class as generic type, since the behavior will change in ruby 2.4 (jeremyevans)
+
+* Don't hold connection pool mutex while disconnecting connections (jeremyevans)
+
+* Don't hold references to disconnected connections in the connection_validator extension (jeremyevans)
+
+* Don't overwrite existing connection_validation_timeout when loading connection_validator extension multiple times (jeremyevans)
+
+* Add connection_expiration extension, for automatically removing connections open for too long (pdrakeweb) (#1208, #1209)
+
+* Handle disconnection errors raised during string literalization in mysql2 and postgres adapters (jeremyevans)
+
+* Add string_agg extension for aggregate string concatenation support on many databases (jeremyevans)
+
+* Add SQL::Function#order for ordered aggregate functions (jeremyevans)
+
+* Support operator validation in constraint_validations for <, <=, >, and >= operators with string and integer arguments (jeremyevans)
+
+* Make validates_operator validation consider nil values invalid unless :allow_nil or similar option is used (jeremyevans)
+
+* Close cursors for non-SELECT queries in the oracle adapter after execution, instead of waiting until GC (jeremyevans) (#1203)
+
+* Add :class_namespace association option for setting default namespace for :class option given as symbol/string (jeremyevans)
+
+* Add Sequel::Model.cache_anonymous_models accessor for changing caching on a per-model basis (jeremyevans)
+
+* Add Sequel::Model.def_Model for adding a Model() method to a module, for easier use of namespaced models (jeremyevans)
+
+* Add Sequel::Model::Model() for creating subclasses of Sequel::Model subclasses, instead of just Sequel::Model itself (jeremyevans)
+
 === 4.35.0 (2016-06-01)
 
 * Add :headline option to PostgreSQL Dataset#full_text_search for adding an extract of the matched text to the SELECT list (jeremyevans)

data/doc/association_basics.rdoc

@@ -907,11 +907,13 @@ you need to setup a delayed evaluation:
 
 This is the class of the associated objects that will be used.  It's
 one of the most commonly used options.  If it is not given, it guesses
-based on the name of the association.  If a *_to_many association is
-used, uses the singular form of the association name.  For example:
+based on the name of the association, including considering the namespace
+of the current model.  If a *_to_many association is used, uses the
+singular form of the association name.  For example:
 
   Album.many_to_one :artist # guesses Artist
   Artist.one_to_many :albums # guesses Album
+  Foo::Artist.one_to_many :albums # guesses Foo::Album
 
 However, for more complex associations, especially ones that add
 additional filters beyond the foreign/primary key relationships, the
@@ -925,10 +927,18 @@ default class guessed will be wrong:
 You can specify the :class option using the class itself, a Symbol,
 or a String:
 
-  Album.many_to_one :artist, :class=>Artist # Class
-  Album.many_to_one :artist, :class=>:Artist # Symbol
+  Album.many_to_one :artist, :class=>Artist   # Class
+  Album.many_to_one :artist, :class=>:Artist  # Symbol
   Album.many_to_one :artist, :class=>"Artist" # String
 
+If you are namespacing your models, and you need to specify the :class
+option, the path you give to the :class option should be the full path
+to the associated class including any namespaces:
+
+  Foo::Album.many_to_one :artist                        # Uses Foo::Artist
+  Foo::Album.many_to_one :artist, :class=>"Artist"      # Uses Artist
+  Foo::Album.many_to_one :artist, :class=>"Foo::Artist" # Uses Foo::Artist
+
 ==== :key
 
 For +many_to_one+ associations, this is the foreign_key in the current model's
@@ -1758,6 +1768,19 @@ a many_to_many association where there is a unique index in the join table
 so that you know only one object will ever be associated through the
 association.
 
+==== :class_namespace
+
+If the :class option is specified as a symbol or string, the default namespace
+in which to look up the class. If the :class option is not specified as a
+symbol or string, this option is ignored.  This namespace can be overridden
+by starting the string or symbol with <tt>::</tt>:
+
+  Foo::Album.many_to_one :artist, :class=>"Artist"                                 # Uses Artist
+  Foo::Album.many_to_one :artist, :class=>"Artist", :class_namespace=>'Foo'        # Uses Foo::Artist
+  Foo::Album.many_to_one :artist, :class=>"Foo::Artist", :class_namespace=>'Foo'   # Uses Foo::Foo::Artist
+  Foo::Album.many_to_one :artist, :class=>"::Artist", :class_namespace=>'Foo'      # Uses Artist
+  Foo::Album.many_to_one :artist, :class=>"::Foo::Artist", :class_namespace=>'Foo' # Uses Foo::Artist
+
 ==== :methods_module
 
 The module that the methods created by the association will be placed

data/doc/migration.rdoc

@@ -611,3 +611,27 @@ or:
 This usage is discouraged in new code, but will continue to be supported indefinitely.
 It is not recommended to convert old-style migration classes to the <tt>Sequel.migration</tt>
 DSL, but it is recommended to use the <tt>Sequel.migration</tt> DSL for all new migrations.
+
+== Database-specific migrations
+
+While not a recommended practice, it is sometimes necessary to have parts of migrations be
+database-specific .  You can use the Sequel::Database#database_type method to check which
+database the migration is being run on, and operate accordingly:
+
+  Sequel.migration do
+    up do
+      if database_type == :mysql
+        run 'MySQL specific code'
+      else
+        run 'Generic SQL code'
+      end
+    end
+  
+    down do
+      if database_type == :mysql
+        run 'MySQL specific code'
+      else
+        run 'Generic SQL code'
+      end
+    end
+  end

data/doc/release_notes/4.36.0.txt

@@ -0,0 +1,116 @@
+= New Features
+
+* Sequel::Model::Model() has been added, which allows for
+  Sequel::Model() like behavior where the base class used is a
+  subclass of Sequel::Model.  To make it easier to use,
+  Sequel::Model.def_Model has also been added, which takes a module
+  and adds a Model() method to the module that calls Model() on the
+  receiver.
+
+  A :class_namespace association option has been added to make it
+  possible to set a default namespace for the :class option if given
+  as a symbol or string.
+
+  Sequel::Model.Model.cache_anonymous_models has been added and
+  controls whether to cache anonymous model subclasses created by
+  Sequel::Model::Model() on a per-class basis.
+
+  These changes are designed to make it easier to use namespaced
+  models, for example:
+
+    module Foo
+       Model = Class.new(Sequel::Model)
+       Model.def_Model(self)
+       DB = Model.db = Sequel.connect(ENV['FOO_DATABASE_URL'])
+       Model.plugin :prepared_statements
+       Model.default_association_options[:class_namespace] = 'Foo'
+
+       class Bar < Model
+         # Uses Foo::DB[:bars] as dataset
+
+         # Implicitly uses Foo::Baz as associated class
+         one_to_many :bazes
+
+         # Uses Foo::Baz due to :class_namespace option
+         one_to_many :oldest_bazes, :class=>:Baz, :order=>:id
+       end
+
+       class Baz < Model(:my_baz)
+         # Uses Foo::DB[:my_baz] as dataset
+
+         # Implicitly uses Foo::Bar as associated class
+         one_to_many :bars
+
+         # Uses Foo::Bar due to :class_namespace option
+         one_to_many :oldest_bars, :class=>:Bar, :order=>:id
+       end
+     end
+
+* A string_agg extension has been added for aggregate string
+  concatentation support on PostgreSQL 9+, SQLAnywhere 12+,
+  Oracle11g+, DB 9.7+, MySQL, HSQLDB, H2, and CUBRID:
+
+    DB.extension :string_agg
+    ds = DB[:table]
+    ds.get(Sequel.string_agg(:c))             # ',' default separator
+    ds.get(Sequel.string_agg(:c, ' - '))      # custom separator
+    ds.get(Sequel.string_agg(:c).order(:bar)) # force order
+    ds.get(Sequel.string_agg(:c).distinct)    # remove duplicates
+
+* A connection_expiration extension has been added, for automatically
+  removing connections from the connection pool after they have been
+  open for a given amount of time (4 hours by default).
+
+* Support for <, <=, >, and >= operator validations when using integer
+  and string arguments has been added to the constraint_validations
+  extension and plugin.
+
+* Sequel::SQL::Function#order has been added to support ordered
+  aggregate functions:
+
+    Sequel.function(:foo, :bar).order(:baz)
+    # foo(bar ORDER BY baz)
+
+= Other Improvements
+
+* The validates_operator validation in validation_helpers now
+  considers nil values as invalid unless :allow_nil or a similar
+  option is used.  Previously, using validates_operator with a nil
+  value would probably raise a NoMethodError.  This makes
+  validates_operator more similar to other validations.
+
+* The threaded connection pools no longer hold the pool mutex when
+  disconnecting connections, which is useful if the driver blocks
+  when disconnecting connections.
+
+* The connection_validator extension no longer holds a reference
+  to connections that have been disconnected.
+
+* The connection_validator extension no longer overwrites the
+  connection_validation_timeout if loaded a second time.
+
+* Sequel now closes cursors as soon as it is done using them in the
+  oracle adapter, instead of waiting for GC to clean them up.
+
+* Sequel now handles disconnect errors that occur when literalizing
+  strings in the mysql2 and postgres adapters.
+
+= Backwards Compatibility
+
+* Using the Bignum class as a generic type is now deprecated.  As
+  announced in the 4.35.0 release notes, ruby 2.4 is unifying the
+  Fixnum and Bignum classes into Integer, which results in the
+  behavior of the Bignum class changing.  4.35.0 added support for
+  using the :Bignum symbol as a generic 64-bit integer type, and
+  Sequel users now need to switch to that to avoid the deprecation
+  warning.
+
+  Sequel 4.41.0 (to be released in December), will drop support
+  for using the Bignum class as a generic type.  This is being done
+  before the release of ruby 2.4 to hopefully make it unlikely that
+  users will be subject to a behavior changes when upgrading ruby
+  versions.
+
+  Related to this change, external adapters need to switch from
+  overriding Database#type_literal_generic_bignum to 
+  Database#type_literal_generic_bignum_symbol.

data/lib/sequel/adapters/jdbc/h2.rb

@@ -147,7 +147,7 @@ module Sequel
 
         # Use BIGINT IDENTITY for identity columns that use bigint, fixes
         # the case where primary_key :column, :type=>:Bignum is used.
-        def type_literal_generic_bignum(column)
+        def type_literal_generic_bignum_symbol(column)
           column[:identity] ? 'BIGINT IDENTITY' : super
         end
       end

data/lib/sequel/adapters/mysql2.rb

@@ -199,7 +199,17 @@ module Sequel
 
       # Handle correct quoting of strings using ::Mysql2::Client#escape.
       def literal_string_append(sql, v)
-        sql << APOS << db.synchronize(@opts[:server]){|c| c.escape(v)} << APOS
+        s = begin
+          db.synchronize(@opts[:server]) do |c| 
+            begin
+              c.escape(v)
+            rescue ::Mysql2::Error => e
+              db.send(:raise_error, e)
+            end
+          end
+        end
+
+        sql << APOS << s << APOS
       end
     end
   end

data/lib/sequel/adapters/oracle.rb

@@ -86,11 +86,7 @@ module Sequel
               r = log_connection_yield(sql, conn){conn.exec(sql)}
             end
             if block_given?
-              begin
-                yield(r)
-              ensure
-                r.close
-              end
+              yield(r)
             elsif type == :insert
               last_insert_id(conn, opts)
             else
@@ -99,6 +95,8 @@ module Sequel
           rescue OCIException, RuntimeError => e
             # ruby-oci8 is naughty and raises strings in some places
             raise_error(e)
+          ensure
+            r.close if r.is_a?(::OCI8::Cursor)
           end
         end
       end

data/lib/sequel/adapters/postgres.rb

@@ -841,12 +841,12 @@ module Sequel
       
       # Use the driver's escape_bytea
       def literal_blob_append(sql, v)
-        sql << APOS << db.synchronize(@opts[:server]){|c| c.escape_bytea(v)} << APOS
+        sql << APOS << db.synchronize(@opts[:server]){|c| c.check_disconnect_errors{c.escape_bytea(v)}} << APOS
       end
       
       # Use the driver's escape_string
       def literal_string_append(sql, v)
-        sql << APOS << db.synchronize(@opts[:server]){|c| c.escape_string(v)} << APOS
+        sql << APOS << db.synchronize(@opts[:server]){|c| c.check_disconnect_errors{c.escape_string(v)}} << APOS
       end
       
       # For each row in the result set, yield a hash with column name symbol

data/lib/sequel/adapters/shared/access.rb

@@ -69,7 +69,7 @@ module Sequel
       
       # Access doesn't have a 64-bit integer type, so use integer and hope
       # the user isn't using more than 32 bits.
-      def type_literal_generic_bignum(column)
+      def type_literal_generic_bignum_symbol(column)
         :integer
       end
 

data/lib/sequel/adapters/shared/oracle.rb

@@ -241,7 +241,7 @@ module Sequel
       # Oracle's integer/:number type handles larger values than
       # most other databases's bigint types, so it should be
       # safe to use for Bignum.
-      def type_literal_generic_bignum(column)
+      def type_literal_generic_bignum_symbol(column)
         :integer
       end
 

data/lib/sequel/adapters/shared/postgres.rb

@@ -1142,7 +1142,7 @@ module Sequel
       end
 
       # Handle bigserial type if :serial option is present
-      def type_literal_generic_bignum(column)
+      def type_literal_generic_bignum_symbol(column)
         column[:serial] ? :bigserial : super
       end
 

data/lib/sequel/adapters/shared/sqlite.rb

@@ -487,7 +487,7 @@ module Sequel
       # SQLite only supports AUTOINCREMENT on integer columns, not
       # bigint columns, so use integer instead of bigint for those
       # columns.
-      def type_literal_generic_bignum(column)
+      def type_literal_generic_bignum_symbol(column)
         column[:auto_increment] ? :integer : super
       end
     end

data/lib/sequel/connection_pool.rb

@@ -97,6 +97,11 @@ class Sequel::ConnectionPool
   end
   
   private
+
+  # Remove the connection from the pool.
+  def disconnect_connection(conn)
+    db.disconnect_connection(conn)
+  end
   
   # Return a new connection by calling the connection proc with the given server name,
   # and checking for connection errors.

data/lib/sequel/connection_pool/sharded_single.rb

@@ -92,7 +92,7 @@ class Sequel::ShardedSingleConnectionPool < Sequel::ConnectionPool
   # Disconnect from the given server, if connected.
   def disconnect_server(server)
     if conn = @conns.delete(server)
-      db.disconnect_connection(conn)
+      disconnect_connection(conn)
     end
   end
 

data/lib/sequel/connection_pool/sharded_threaded.rb

@@ -96,9 +96,9 @@ class Sequel::ShardedThreadedConnectionPool < Sequel::ThreadedConnectionPool
   # :server :: Should be a symbol specifing the server to disconnect from,
   #            or an array of symbols to specify multiple servers.
   def disconnect(opts=OPTS)
-    sync do
-      (opts[:server] ? Array(opts[:server]) : @servers.keys).each do |s|
-        disconnect_server(s)
+    (opts[:server] ? Array(opts[:server]) : sync{@servers.keys}).each do |s|
+      if conns = sync{disconnect_server_connections(s)}
+        disconnect_connections(conns)
       end
     end
   end
@@ -139,11 +139,12 @@ class Sequel::ShardedThreadedConnectionPool < Sequel::ThreadedConnectionPool
   # except that after it is used, future requests for the server will use the
   # :default server instead.
   def remove_servers(servers)
+    conns = nil
     sync do
       raise(Sequel::Error, "cannot remove default server") if servers.include?(:default)
       servers.each do |server|
         if @servers.include?(server)
-          disconnect_server(server)
+          conns = disconnect_server_connections(server)
           @waiters.delete(server) if USE_WAITER
           @available_connections.delete(server)
           @allocated.delete(server)
@@ -151,6 +152,10 @@ class Sequel::ShardedThreadedConnectionPool < Sequel::ThreadedConnectionPool
         end
       end
     end
+
+    if conns
+      disconnect_connections(conns)
+    end
   end
 
   # Return an array of symbols for servers in the connection pool.
@@ -243,16 +248,26 @@ class Sequel::ShardedThreadedConnectionPool < Sequel::ThreadedConnectionPool
     conn
   end
 
-  # Disconnect from the given server.  Disconnects available connections
+  # Clear the array of available connections for the server, returning an array
+  # of previous available connections that should be disconnected (or nil if none should be).
+  # Mark any allocated connections to be removed when they are checked back in. The calling
+  # code should already have the mutex before calling this.
+  def disconnect_server_connections(server)
+    @connections_to_remove.concat(allocated(server).values)
+
+    if dis_conns = available_connections(server)
+      conns = dis_conns.dup
+      dis_conns.clear
+    end
+    conns
+  end
+
+  # Disconnect all given connections.  
   # immediately, and schedules currently allocated connections for disconnection
-  # as soon as they are returned to the pool. The calling code should already
+  # as soon as they are returned to the pool. The calling code should not
   # have the mutex before calling this.
-  def disconnect_server(server)
-    if conns = available_connections(server)
-      conns.each{|conn| db.disconnect_connection(conn)}
-      conns.clear
-    end
-    @connections_to_remove.concat(allocated(server).values)
+  def disconnect_connections(conns)
+    conns.each{|conn| disconnect_connection(conn)}
   end
 
   # Creates a new connection to the given server if the size of the pool for
@@ -318,7 +333,7 @@ class Sequel::ShardedThreadedConnectionPool < Sequel::ThreadedConnectionPool
       conn = allocated(server).delete(thread)
 
       if @connection_handling == :disconnect
-        db.disconnect_connection(conn)
+        disconnect_connection(conn)
       else
         checkin_connection(server, conn)
       end
@@ -330,7 +345,7 @@ class Sequel::ShardedThreadedConnectionPool < Sequel::ThreadedConnectionPool
   def remove(thread, conn, server)
     @connections_to_remove.delete(conn)
     allocated(server).delete(thread) if @servers.include?(server)
-    db.disconnect_connection(conn)
+    disconnect_connection(conn)
   end
   
   CONNECTION_POOL_MAP[[false, true]] = self