sequel 3.35.0 → 3.36.0

data/doc/opening_databases.rdoc

@@ -68,7 +68,7 @@ These options are shared by all adapters unless otherwise noted.
 
 :adapter :: The adapter to use
 :database :: The name of the database to which to connect
-:default_schema :: The database schema to use by default.
+:default_schema :: The database schema to use in schema_modification and introspection.
 :host :: The hostname of the database server to which to connect
 :logger :: An array of SQL loggers to log to
 :loggers :: An array of SQL loggers to log to
@@ -265,6 +265,7 @@ Example connection strings:
   jdbc:oracle:thin:user/password@localhost:1521:database
   jdbc:db2://localhost:3700/database:user=user;password=password;
   jdbc:firebirdsql:localhost/3050:/path/to/database.fdb
+  jdbc:jdbcprogress:T:hostname:port:database
 
 You can also use JNDI connection strings:
 
@@ -294,6 +295,8 @@ The following additional options are supported:
 :config_local_infile :: If provided, sets the Mysql::OPT_LOCAL_INFILE option on the connection with the given value.
 :encoding :: Specify the encoding/character set to use for the connection.
 :socket :: Can be used to specify a Unix socket file to connect to instead of a TCP host and port.
+:sql_mode :: Set the sql_mode(s) for a given connection.  Can be single symbol or string,
+             or an array of symbols or strings (e.g. <tt>:sql_mode=>[:no_zero_date, :pipes_as_concat]</tt>).
 :timeout :: Sets the wait_timeout for the connection, defaults to 1 month.
 :read_timeout :: Set the timeout in seconds for reading back results to a query.
 :connect_timeout :: Set the timeout in seconds before a connection attempt is abandoned.

data/doc/release_notes/3.36.0.txt

@@ -0,0 +1,245 @@
+= New Features
+
+* An eager_each plugin has been added, which automatically makes
+  eagerly loaded datasets do eager loading if you call #each (or
+  another Enumerable method) instead of #all.  By default, if you
+  call #each on an eager dataset, it will not do eager loading,
+  and if you call #each on an eager_graph dataset, you will
+  get plain hashes with columns from all joined tables instead of
+  model objects.  With this plugin, #each on both eager and
+  eager_graph datasets will do eager loading.
+
+* The nested attributes plugin now supports composite primary keys
+  in associated records.  Additionally, it now deals better with
+  natural primary keys in associated records.  There is a new
+  :unmatched_pk option that can be set to :create if you want to
+  create new associated records when the input hash contains
+  primary key information that doesn't match one of the existing
+  associated objects.
+
+  The nested attributes plugin now also supports a :transform option.
+  If given, this option is called with the parent object and the
+  input hash given for each associated record passed into the
+  nested atttributes setter.  The callable should return the hash
+  of attributes to use.
+
+* Model#from_json in the json_serializer plugin now takes an options
+  hash and recognizes the :fields option.  If the :fields option is
+  given, it should be an array of field names, and set_fields is
+  called with the array instead of using set.  This allows you to
+  easily filter which fields in the hash are set in the model
+  instance.  The entire options hash is also passed to set_fields
+  if :fields is present, so you can additionally use the :missing =>
+  :raise or :missing => :skip options that set_fields supports.
+
+* The Dataset#to_json method in the json_serializer plugin now
+  respects :root=>:collection and :root=>:instance options.  If
+  :root=>:collection is given, only the collection is wrapped in a
+  hash, and if :root=>:instance is given, only the instances are
+  wrapped in a hash.  For backwards compatibility, both the
+  instances and collection are wrapped in a hash:
+
+    Model.to_json(:root=>true)
+    # {"models":[{"model":{"id":1}}]}
+
+    Model.to_json(:root=>:collection)
+    # {"models":[{"id":1}]}
+
+    Model.to_json(:root=>:instance)
+    # [{"model":{"id":1}}]
+
+  Wrapping both the collection and instance in a root by default
+  is probably an undesired behavior, so the default for :root=>true
+  may change in the next major version of Sequel.  Users who want
+  the current behavior should switch to using :root=>:both.
+
+* The schema_dumper extension now respects an :index_names option
+  when dumping.  This option can be set to false to never dump the
+  index names.  It can also be set to :namespace, in which case if
+  the database does not have a global index namespace, it will
+  automatically prefix the name of the index with the name of the
+  table.
+
+  Database#global_index_namespace? was added to check if the
+  database uses a global index namespace.  If false, index names are
+  probably namespaced per table (MySQL, MSSQL, Oracle).
+
+* :each is now a valid prepared statement type.  This prepared
+  statement type requires a block when you call the statement, and
+  iterates over the records of the statement a row at a time.
+  Previously, there wasn't a way to iterate over the records of a
+  prepared statement a row at a time, since the :select and :all
+  types collect all rows into an array before iterating over them.
+
+* A :connection_handling=>:queue option is now respected for
+  database objects, and changes the threaded connection pools to use
+  a queue instead of a stack as the data structure for storing
+  available connections.  A queue does not perform as well as a
+  stack, but reduces the likelihood of stale connections.
+
+  It is possible that Sequel will change in the future from using a
+  stack by default to using a queue by default, so any users who
+  specifically desire a stack to be used should specify the
+  :connection_handling=>:stack option.
+
+* Sequel::Migrator now supports is_current? class method to check
+  if there are no outstanding migrations to apply.  It also supports
+  a check_current class method, which raises an exception if there
+  are outstanding migrations to apply.
+
+* A pg_json extension has been added, supporting PostgreSQL's 9.2
+  json type, similarly to the pg_array and pg_hstore extensions.
+  Note that with the current PostgreSQL json code, the root object
+  can be a string or number, but ruby's json library requires the
+  root json value to be an object or array.  So you will probably
+  get an exception if you attempt to retrieve a PostgreSQL json
+  value that ruby's JSON library won't parse.
+
+* A pg_inet extension has been added, which automatically typecasts
+  PostgreSQL inet and cidr types to ruby IPAddr objects on retrieval.
+
+* Database#transaction on PostgreSQL now recognizes :read_only and
+  :deferrable options, and can use them to set the READ ONLY and
+  DEFERRABLE transaction flags.  A :synchronous option is also
+  recognized, which can be set to true, false, :local, or
+  :remote_write, and sets the value of synchronous_commit just for
+  that transaction.
+
+* When adding and dropping indexes on PostgreSQL, a :concurrently
+  option can be used to create or drop the index CONCURRENTLY, which
+  doesn't require a full write table lock.
+
+* When dropping indexes on PostgreSQL, :if_exists and :cascade options
+  are now recognized.
+
+* When using alter_table set_column_type on PostgreSQL, the :using
+  option is respected, and can be used to force a specific conversion
+  from the previous value to the new value with the USING syntax.
+
+* On MySQL, you can now set an :sql_mode option when connecting.  This
+  can be a string or symbol or an array of them, and each should match
+  one of MySQL's sql_modes.  MySQL's default SQL mode is fairly loose,
+  and using one of the strict sql modes is recommended, but for
+  backwards compatibility, Sequel will not set a specific SQL mode by
+  default.  However, that may change in the next major version of
+  Sequel, so to be forwards compatible you should set :sql_mode=>nil
+  if you do not desire a strict SQL mode to be set automatically.
+
+* Partial indexes are now supported on Microsoft SQL Server 2008
+  (SQL Server refers to them as filtered indexes).  Attempting to
+  use a partial index on an earlier version of SQL Server will
+  result in the database raising an exception.
+
+* A jdbc/progress adapter has been added, supporting the Progress
+  database via the jdbc adapter.
+
+= Other Improvements
+
+* Dataset#get now works correctly if you pass it a nil or false
+  argument.  Previously, it ignored the argument and used the block
+  instead.  If you want to use the block argument, you should not
+  pass in a regular argument.
+
+* Database#call now passes any blocks given to it to the underlying
+  prepared statement object.  Before, a passed block was ignored.
+
+* Sequel::Model.db is no longer set automatically when creating
+  an anonymous class with an associated database object.  This fixes
+  cases where a library would create namespaced models, and the
+  database used by the library would be set as the default for the
+  user's application code.
+
+* Model *_to_one association setters are now no-ops if you pass a
+  value that is the same as the cached value.  This fixes issues with
+  reciprocal associations getting reordered, and is better
+  for performance.
+
+  For cases where the old behavior is desired, the
+  set_associated_object_if_same? method can be overridden to return
+  true for object.  If you are manually setting objects in the
+  associations cache before calling the setter method, you may want
+  to set that.
+
+* The dirty plugin no longer affects the return value of refresh
+  and lock!.  Internal changes should now help ensure that plugins
+  don't affect the return values of these methods.
+
+* Sequel now supports JRuby 1.7's new exception handling, fixing
+  exception handling when connecting in the jdbc adapter.
+
+* When dumping unsigned integer types in the schema dumper, if the
+  unsigned values could overflow a 32-bit signed integer type,
+  the generic Bignum class is used as the type.  This should fix
+  issues when copying a database containing an unsigned 32-bit
+  integer column with values between 2^31 and 2^32-1.
+
+* In the optimistic_locking plugin, attempting to refresh and
+  save after a failed save now works correctly.  Before, the second
+  save would never modify a row.
+
+* Time types on jdbc/postgres are now typecasted accurately on
+  retrieval, before they could be off by up to a millisecond due to
+  floating point issues.
+
+* Disconnect detection in the mysql2 adapter has been improved.
+
+* The jdbc/mysql, do/mysql, and swift/mysql adapters all now support
+  the :timeout option to set the MySQL wait_timeout.
+
+* Savepoints in prepared transactions are now supported on MySQL
+  5.5.23+, since the bug that caused them to be unsupported starting
+  in 5.5.13 has been fixed.
+
+* Parsing foreign key metadata for tables with an explicit
+  schema now works correctly on PostgreSQL.
+
+* bin/sequel -C now namespaces indexes automatically when copying
+  from a database without a global index namespace to a database
+  with a global index namespace.
+
+* Indexes are now dropped in reverse order that they were added in
+  the schema_dumper.
+
+* The Model typecasting code works around bugs in objects where
+  object.==('') would raise an exception instead of returning false.
+
+* A better error message is used if an invalid JDBC URL is
+  provided and the JDBC driver's new.connect method returns NULL.
+
+* A document describing Sequel's object model has been added,
+  describing the objects Sequel uses to represent SQL concepts.
+
+* Most adapter specific options to Database methods are now mentioned
+  in the main Database method RDoc.
+
+= Backwards Compatibility
+
+* The nested_attributes plugin internals changed significantly.  If
+  you were overriding one of the nested_attributes* private methods
+  and calling super to get the default behavior, you may have to
+  update your code.
+
+* Database#case_sensitive_like has been removed on SQLite.  This
+  method never worked correctly, it always returned false even if
+  the case_sensitive_like PRAGMA was set.  That's because SQLite
+  doesn't offer a getter for this PRAGMA, only a setter.  Note that
+  Database#case_sensitive_like= still exists and works correctly.
+
+* Database#single_value has been removed from the native SQLite
+  adapter.  This method was designed for internal use, and hasn't
+  been used for some time.  Any current users of the method should
+  switch to Dataset#single_value.
+
+* The private Database#defined_columns_for method in the SQLite
+  adapter no longer takes an options hash.
+
+* A couple jdbc/postgres adapter methods are now private.  Previously,
+  the jdbc/postgres adapter overrode some private superclass methods
+  but left the methods public.
+
+* When using the optimistic_locking plugin, refreshing inside a
+  before_update method after calling super will now result in the
+  lock checking being skipped.
+
+* The private Model#_refresh no longer returns self, so external
+  plugins should no longer rely on that behavior.

data/doc/schema_modification.rdoc

@@ -85,14 +85,8 @@ method, the fourth argument is the options hash.  The following options are supp
 :null :: Mark the column as allowing NULL values (if true),
          or not allowing NULL values (if false).  If unspecified, will default
          to whatever the database default is.
-:size :: The size of the column, generally used with string
-         columns to specify the maximum number of characters the column will hold.
-         An array of two integers can be provided to set the size and the
-         precision, respectively, of decimal columns.
 :unique :: Mark the column as unique, generally has the same effect as
            creating a unique index on the column.
-:unsigned :: Make the column type unsigned, only useful for integer
-             columns.
 
 === Other methods
 

data/lib/sequel/adapters/do/mysql.rb

@@ -18,9 +18,16 @@ module Sequel
           (m = /\/(.*)/.match(URI.parse(uri).path)) && m[1]
         end
 
+        # Recognize the tinyint(1) column as boolean.
         def schema_column_type(db_type)
           db_type == 'tinyint(1)' ? :boolean : super
         end
+
+        # Apply the connectiong setting SQLs for every new connection.
+        def setup_connection(conn)
+          mysql_connection_setting_sqls.each{|sql| log_yield(sql){conn.create_command(sql).execute_non_query}}
+          super
+        end
       end
       
       # Dataset class for MySQL datasets accessed via DataObjects.

data/lib/sequel/adapters/jdbc.rb

@@ -116,6 +116,12 @@ module Sequel
         db.extend(Sequel::JDBC::Firebird::DatabaseMethods)
         db.extend_datasets Sequel::Firebird::DatasetMethods
         org.firebirdsql.jdbc.FBDriver
+      end,
+      :jdbcprogress=>proc do |db|
+        Sequel.ts_require 'adapters/jdbc/progress'
+        db.extend(Sequel::JDBC::Progress::DatabaseMethods)
+        db.extend_datasets Sequel::Progress::DatasetMethods
+        com.progress.sql.jdbc.JdbcProgressDriver
       end
     }
     
@@ -205,7 +211,7 @@ module Sequel
           begin
             JavaSQL::DriverManager.setLoginTimeout(opts[:login_timeout]) if opts[:login_timeout]
             JavaSQL::DriverManager.getConnection(*args)
-          rescue => e
+          rescue JavaSQL::SQLException, NativeException, StandardError => e
             raise e unless driver
             # If the DriverManager can't get the connection - use the connect
             # method of the driver. (This happens under Tomcat for instance)
@@ -217,7 +223,9 @@ module Sequel
             opts[:jdbc_properties].each{|k,v| props.setProperty(k.to_s, v)} if opts[:jdbc_properties]
             begin
               driver.new.connect(args[0], props)
-            rescue => e2
+              raise(Sequel::DatabaseError, 'driver.new.connect returned nil: probably bad JDBC connection string') unless c
+              c
+            rescue JavaSQL::SQLException, NativeException, StandardError => e2
               e.message << "\n#{e2.class.name}: #{e2.message}"
               raise e
             end
@@ -632,7 +640,7 @@ module Sequel
       # Handle type conversions for common Java types.
       class TYPE_TRANSLATOR
         LF = "\n".freeze
-        def time(v) Sequel.string_to_time(v.to_string) + v.getTime.divmod(1000).last/1000.0 end
+        def time(v) Sequel.string_to_time("#{v.to_string}.#{sprintf('%03i', v.getTime.divmod(1000).last)}") end
         def date(v) Date.civil(v.getYear + 1900, v.getMonth + 1, v.getDate) end
         def decimal(v) BigDecimal.new(v.to_string) end
         def byte_array(v) Sequel::SQL::Blob.new(String.from_java_bytes(v)) end

data/lib/sequel/adapters/jdbc/mysql.rb

@@ -53,13 +53,11 @@ module Sequel
           db_type == 'tinyint(1)' ? :boolean : super
         end
       
-        # By default, MySQL 'where id is null' selects the last inserted id.
-        # Turn that off unless explicitly enabled.
+        # Run the default connection setting SQL statements.
+        # Apply the connectiong setting SQLs for every new connection.
         def setup_connection(conn)
+          mysql_connection_setting_sqls.each{|sql| statement(conn){|s| log_yield(sql){s.execute(sql)}}}
           super
-          sql = "SET SQL_AUTO_IS_NULL=0"
-          statement(conn){|s| log_yield(sql){s.execute(sql)}} unless opts[:auto_is_null]
-          conn
         end
       end
     end

data/lib/sequel/adapters/jdbc/postgresql.rb

@@ -43,7 +43,7 @@ module Sequel
       class Dataset < JDBC::Dataset
         include Sequel::Postgres::DatasetMethods
         APOS = Dataset::APOS
-
+        
         class ::Sequel::JDBC::Dataset::TYPE_TRANSLATOR
           # Convert Java::OrgPostgresqlJdbc4::Jdbc4Array to ruby arrays
           def pg_array(v)
@@ -73,6 +73,15 @@ module Sequel
         PG_ARRAY_METHOD = TYPE_TRANSLATOR_INSTANCE.method(:pg_array)
         PG_OBJECT_METHOD = TYPE_TRANSLATOR_INSTANCE.method(:pg_object)
       
+        # Add the shared PostgreSQL prepared statement methods
+        def prepare(*args)
+          ps = super
+          ps.extend(::Sequel::Postgres::DatasetMethods::PreparedStatementMethods)
+          ps
+        end
+
+        private
+        
         # Handle PostgreSQL array and object types. Object types are just
         # turned into strings, similarly to how the native adapter treats
         # the types.
@@ -87,13 +96,6 @@ module Sequel
           end
         end
         
-        # Add the shared PostgreSQL prepared statement methods
-        def prepare(*args)
-          ps = super
-          ps.extend(::Sequel::Postgres::DatasetMethods::PreparedStatementMethods)
-          ps
-        end
-        
         # Literalize strings similar to the native postgres adapter
         def literal_string_append(sql, v)
           sql << APOS << db.synchronize{|c| c.escape_string(v)} << APOS

data/lib/sequel/adapters/jdbc/progress.rb

@@ -0,0 +1,21 @@
+Sequel.require 'adapters/shared/progress'
+Sequel.require 'adapters/jdbc/transactions'
+
+module Sequel
+  module JDBC
+    # Database and Dataset instance methods for Progress v9 specific
+    # support via JDBC.
+    module Progress
+      # Database instance methods for Progress databases accessed via JDBC.
+      module DatabaseMethods
+        include Sequel::Progress::DatabaseMethods
+        include Sequel::JDBC::Transactions
+
+        # Progress DatabaseMetaData doesn't even implement supportsSavepoints()
+        def supports_savepoints?
+          false
+        end
+      end
+    end
+  end
+end

data/lib/sequel/adapters/mock.rb

@@ -22,7 +22,7 @@ module Sequel
 
       # Delegate to the db's #_execute method.
       def execute(sql)
-        @db.send(:_execute, self, sql) 
+        @db.send(:_execute, self, sql, :log=>false) 
       end
     end
 
@@ -200,7 +200,7 @@ module Sequel
         sql += " -- args: #{opts[:arguments].inspect}" if opts[:arguments]
         sql += " -- #{@opts[:append]}" if @opts[:append]
         sql += " -- #{c.server.is_a?(Symbol) ? c.server : c.server.inspect}" if c.server != :default
-        log_info(sql)
+        log_info(sql) unless opts[:log] == false
         @sqls << sql 
 
         ds = opts[:dataset]
@@ -298,10 +298,6 @@ module Sequel
       def disconnect_connection(c)
       end
 
-      def log_connection_execute(c, sql)
-        c.execute(sql)
-      end
-
       def quote_identifiers_default
         shared_adapter? ? super : false
       end