sequel 3.42.0 → 3.43.0

data/CHANGELOG

@@ -1,3 +1,43 @@
+=== 3.43.0 (2013-01-08)
+
+* Move the #meta_def support for Database, Dataset, and Model to the meta_def extension (jeremyevans)
+
+* Fix Database#copy_into on jdbc/postgres when an exception is raised (jeremyevans)
+
+* Add core_refinements extension, providing refinement versions of Sequel's core extensions (jeremyevans)
+
+* Make Database#copy_into raise a DatabaseError if the database signals an error in the postgres adapter (jeremyevans)
+
+* Define respond_to_missing? where method_missing is defined and the object supports respond_to? (jeremyevans)
+
+* Allow lambda procs with 0 arity as virtual row blocks on ruby 1.9 (jeremyevans)
+
+* Handle schema-qualified row_types in the pg_array integration in the pg_row extension (jeremyevans) (#595)
+
+* Support default_schema when reseting primary key sequences on PostgreSQL (jeremyevans) (#596)
+
+* Allow treating tinyint(1) unsigned columns as booleans in the mysql adapters (jeremyevans)
+
+* Support the jdbc-hsqldb gem in the jdbc adapter, since it has been updated to 2.2.9 (jeremyevans)
+
+* Work with new jdbc-* gems that require manual driver loading (kares) (#598)
+
+* Cast blobs correctly on DB2 when use_clob_as_blob is false (mluu, jeremyevans) (#594)
+
+* Add date_arithmetic extension for database-independent date calculations (jeremyevans)
+
+* Make Database#schema handle [host.]database.schema.table qualified tables on Microsoft SQL Server (jeremyevans)
+
+* Add Dataset#split_qualifiers helper method for splitting a qualifier identifier into array of strings (jeremyevans)
+
+* Make Database#schema_and_table always return strings for the schema and table (jeremyevans)
+
+* Skip stripping of blob columns in the string_stripper plugin (jeremyevans) (#593)
+
+* Allow Dataset#get to take an array to return multiple values, similar to map/select_map (jeremyevans)
+
+* Default :prefetch_rows to 100 in the Oracle adapter (andrewhr) (#592)
+
 === 3.42.0 (2012-12-03)
 
 * If an exception occurs while committing a transaction, attempt to rollback (jeremyevans)

data/MIT-LICENSE

@@ -1,5 +1,5 @@
 Copyright (c) 2007-2008 Sharon Rosner
-Copyright (c) 2008-2012 Jeremy Evans
+Copyright (c) 2008-2013 Jeremy Evans
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to

data/Rakefile

@@ -56,7 +56,7 @@ rescue LoadError
   begin
     require "rake/rdoctask"
     Rake::RDocTask
-  rescue LoadError
+  rescue LoadError, StandardError
   end
 end
 

data/doc/opening_databases.rdoc

@@ -338,8 +338,8 @@ The following additional options are supported:
 
 :autosequence :: Set to true to use Sequel's conventions to guess the sequence to use for datasets.  False
                  by default.
-:prefetch_rows :: The number of rows to prefetch.  Defaults to 1, a larger number can be specified
-                  and will improve performance when retrieving a large number of rows.
+:prefetch_rows :: The number of rows to prefetch.  Defaults to 100, a larger number can be specified
+                  and may improve performance when retrieving a large number of rows.
 :privilege :: The Oracle privilege level.
 
 === postgres

data/doc/prepared_statements.rdoc

@@ -75,6 +75,13 @@ and update queries, the hash to insert/update is passed to +prepare+:
   ps2.call(:n=>'Jim', :new_n=>'Bob')
   DB.call(:update_name, :n=>'Jim', :new_n=>'Bob') # same as above
 
+== Implementation Issues
+
+Currently, creating a prepared statement uses Object#extend, which can hurt
+performance.  For high performance applications, it's recommended to create
+all of your prepared statements upon application initialization, and not
+to create prepared statements dynamically at runtime.
+
 == Database support
 
 === PostgreSQL

data/doc/release_notes/3.43.0.txt

@@ -0,0 +1,105 @@
+= New Features
+
+* A core_refinements extension has been added, which offers
+  refinement versions of Sequel's core extensions.  This requires
+  the new experimental refinement support added in ruby 2.0, and
+  allows you to use the Sequel DSL methods in a file without
+  actually modifying the Symbol, String, Array, and Hash classes.
+
+* A date_arithmetic extension has been added for performing
+  database-independent date calculations (adding/subtracting an
+  interval to/from a date):
+
+    Sequel.extension :date_arithmetic
+    e = Sequel.date_add(:date_column, :years=>1, :months=>2, :days=>3)
+    DB[:table].where(e > Sequel::CURRENT_DATE)
+
+  In addition to providing the interval as a hash, you can also
+  provide it as an ActiveSupport::Duration object.  This extension
+  is supported on 11 database types.
+
+* Dataset#get can now take an array of multiple expressions to get
+  an array of values, similar to map/select_map:
+
+    value1, value2 = DB[:table].get([:column1, :column2])
+
+* Sequel can now handle [host.]database.schema.table qualified
+  tables on Microsoft SQL Server.  To implement this support,
+  the split_qualifiers method has been added to Database and
+  Dataset for taking a possibly qualified identifier and splitting
+  it into an array of identifier strings.
+
+* The string_stripper plugin now offers the ability to manually
+  specify which columns to skip stripping for via
+  Model.skip_string_stripping. 
+
+= Other Improvements
+
+* The jdbc adapter now works with the new jdbc-* gems, which require
+  a manual load_driver step that the older jdbc-* gems did not
+  require.
+
+* The string_stripper plugin no longer strips blob columns or values.
+
+* Database#copy_into in both the postgres and jdbc/postgres adapters
+  has been fixed to better handle exceptions.  
+
+* Dataset#hash and Model#hash are now significantly faster.
+
+* Lambda procs with 0 arity can now be used as virtual row blocks
+  on ruby 1.9.  Previously, attempting to use a lambda proc with
+  0 arity as a virtual row block on ruby 1.9 would raise an exception.
+
+* Schema-qualified composite types are now handled correctly in
+  the pg_row extension.
+
+* Database#reset_primary_key_sequence on PostgreSQL now works
+  correctly when a default_schema is set.
+
+* tinyint(1) unsigned columns on MySQL are now parsed as booleans
+  instead of integers on MySQL if converting tinyint to boolean.
+   
+* The jdbc adapter now supports the jdbc-hsqldb gem, so you can
+  now install that instead of having to require the .jar manually.
+
+* Blobs are now casted correctly on DB2 when the use_clob_as_blob
+  setting is false.
+
+* Oracle timestamptz types are now handled correctly in the
+  jdbc/oracle adapter.
+
+* Sequel now defaults to :prefetch_rows = 100 in the oracle
+  adapter, which can significantly improve performance.
+
+* Sequel now defines respond_to_missing? where method_missing? is
+  defined and the object also responds to respond_to?.
+
+* Sequel::BasicObject now responds to instance_exec on ruby 1.8.
+
+= Backwards Compatibility
+
+* The meta_def method that was defined on Database, Dataset, and
+  Model classes and instances has been moved to an extension named
+  meta_def, and is no longer loaded by default.  This method was
+  previously used internally, and it wasn't designed for external
+  use.  If you have code that uses meta_def, you should now load the
+  extension manually:
+
+    Sequel.extension :meta_def
+
+* The private _*_dataset_helper model association methods are no
+  longer defined.  The AssociationReflection#dataset_helper_method
+  public method is also no longer defined.
+
+* Dataset#schema_and_table now always returns strings (or nil).
+  Before, in some cases it would return symbols.
+
+* Using a conditions specifier array with Dataset#get no longer
+  works due to the new multiple values support in Database#get.
+  So code such as:
+
+    DB[:table].get([[:a, 1], [:b, 2]])
+
+  should be changed to:
+
+    DB[:table].get(Sequel.expr([[:a, 1], [:b, 2]]))

data/doc/schema_modification.rdoc

@@ -162,6 +162,25 @@ column, but create a foreign key constraint:
     foreign_key [:artist_name, :artist_location], :artists
   end
 
+When using an array of symbols, you can also provide a :name option to name the constraint:
+
+  create_table(:albums) do
+    String :artist_name
+    String :artist_location
+    String :name
+    foreign_key [:artist_name, :artist_location], :artists, :name=>'albums_artist_name_location_fkey'
+  end
+
+If you want to add a foreign key for a single column with a named constraint, you must use
+the array form with a single symbol:
+
+  create_table(:albums) do
+    primary_key :id
+    Integer :artist_id
+    String :name
+    foreign_key [:artist_id], :artists, :name=>'albums_artist_id_fkey'
+  end
+
 ==== +index+
 
 +index+ creates indexes on the table.  For single columns, calling index is the same as using the

data/lib/sequel/adapters/do/mysql.rb

@@ -20,7 +20,7 @@ module Sequel
 
         # Recognize the tinyint(1) column as boolean.
         def schema_column_type(db_type)
-          db_type == 'tinyint(1)' ? :boolean : super
+          db_type =~ /\Atinyint\(1\)/ ? :boolean : super
         end
 
         # Apply the connectiong setting SQLs for every new connection.

data/lib/sequel/adapters/jdbc.rb

@@ -33,14 +33,14 @@ module Sequel
         Sequel.ts_require 'adapters/jdbc/postgresql'
         db.extend(Sequel::JDBC::Postgres::DatabaseMethods)
         db.dataset_class = Sequel::JDBC::Postgres::Dataset
-        JDBC.load_gem('postgres')
+        JDBC.load_gem(:Postgres)
         org.postgresql.Driver
       end,
       :mysql=>proc do |db|
         Sequel.ts_require 'adapters/jdbc/mysql'
         db.extend(Sequel::JDBC::MySQL::DatabaseMethods)
         db.extend_datasets Sequel::MySQL::DatasetMethods
-        JDBC.load_gem('mysql')
+        JDBC.load_gem(:MySQL)
         com.mysql.jdbc.Driver
       end,
       :sqlite=>proc do |db|
@@ -48,7 +48,7 @@ module Sequel
         db.extend(Sequel::JDBC::SQLite::DatabaseMethods)
         db.extend_datasets Sequel::SQLite::DatasetMethods
         db.set_integer_booleans
-        JDBC.load_gem('sqlite3')
+        JDBC.load_gem(:SQLite3)
         org.sqlite.JDBC
       end,
       :oracle=>proc do |db|
@@ -69,28 +69,28 @@ module Sequel
         db.extend(Sequel::JDBC::JTDS::DatabaseMethods)
         db.dataset_class = Sequel::JDBC::JTDS::Dataset
         db.send(:set_mssql_unicode_strings)
-        JDBC.load_gem('jtds')
+        JDBC.load_gem(:JTDS)
         Java::net.sourceforge.jtds.jdbc.Driver
       end,
       :h2=>proc do |db|
         Sequel.ts_require 'adapters/jdbc/h2'
         db.extend(Sequel::JDBC::H2::DatabaseMethods)
         db.dataset_class = Sequel::JDBC::H2::Dataset
-        JDBC.load_gem('h2')
+        JDBC.load_gem(:H2)
         org.h2.Driver
       end,
       :hsqldb=>proc do |db|
         Sequel.ts_require 'adapters/jdbc/hsqldb'
         db.extend(Sequel::JDBC::HSQLDB::DatabaseMethods)
         db.dataset_class = Sequel::JDBC::HSQLDB::Dataset
-        # Current gem is 1.8.1.3, but Sequel supports 2.2.5
+        JDBC.load_gem(:HSQLDB)
         org.hsqldb.jdbcDriver
       end,
       :derby=>proc do |db|
         Sequel.ts_require 'adapters/jdbc/derby'
         db.extend(Sequel::JDBC::Derby::DatabaseMethods)
         db.dataset_class = Sequel::JDBC::Derby::Dataset
-        JDBC.load_gem('derby')
+        JDBC.load_gem(:Derby)
         org.apache.derby.jdbc.EmbeddedDriver
       end,
       :as400=>proc do |db|
@@ -135,9 +135,14 @@ module Sequel
     # works for PostgreSQL, MySQL, and SQLite.
     def self.load_gem(name)
       begin
-        Sequel.tsk_require "jdbc/#{name}"
+        Sequel.tsk_require "jdbc/#{name.to_s.downcase}"
       rescue LoadError
         # jdbc gem not used, hopefully the user has the .jar in their CLASSPATH
+      else
+        if defined?(::Jdbc) && ( ::Jdbc.const_defined?(name) rescue nil )
+          jdbc_module = ::Jdbc.const_get(name) # e.g. Jdbc::SQLite3
+          jdbc_module.load_driver if jdbc_module.respond_to?(:load_driver)
+        end
       end
     end
 

data/lib/sequel/adapters/jdbc/hsqldb.rb

@@ -138,6 +138,11 @@ module Sequel
           true
         end
 
+        # HSQLDB requires SQL standard datetimes in some places.
+        def requires_sql_standard_datetimes?
+          true
+        end
+
         # HSQLDB does not support IS TRUE.
         def supports_is_true?
           false

data/lib/sequel/adapters/jdbc/mysql.rb

@@ -55,7 +55,7 @@ module Sequel
 
         # Convert tinyint(1) type to boolean
         def schema_column_type(db_type)
-          db_type == 'tinyint(1)' ? :boolean : super
+          db_type =~ /\Atinyint\(1\)/ ? :boolean : super
         end
       
         # Run the default connection setting SQL statements.

data/lib/sequel/adapters/jdbc/oracle.rb

@@ -97,10 +97,16 @@ module Sequel
           db.to_application_timestamp(v.to_string)
         end
       
+        def convert_type_oracle_timestamptz(v)
+          convert_type_oracle_timestamp(db.synchronize{|c| v.timestampValue(c)})
+        end
+      
         def convert_type_proc(v)
           case v
           when JAVA_BIG_DECIMAL
             ORACLE_DECIMAL_METHOD
+          when Java::OracleSql::TIMESTAMPTZ
+            method(:convert_type_oracle_timestamptz)
           when Java::OracleSql::TIMESTAMP
             method(:convert_type_oracle_timestamp)
           else

data/lib/sequel/adapters/jdbc/postgresql.rb

@@ -29,7 +29,7 @@ module Sequel
             raise Error, "Must provide either a :data option or a block to copy_into"
           end
 
-          transaction(opts) do |conn|
+          synchronize(opts) do |conn|
             begin
               copy_manager = org.postgresql.copy.CopyManager.new(conn)
               copier = copy_manager.copy_in(copy_into_sql(table, opts))
@@ -41,10 +41,16 @@ module Sequel
                 data.each { |d| copier.writeToCopy(d.to_java_bytes, 0, d.length) }
               end
             rescue Exception => e
-              copier.endCopy
+              copier.cancelCopy
               raise
             ensure
-              copier.endCopy unless e
+              unless e
+                begin
+                  copier.endCopy
+                rescue NativeException => e2
+                  raise_error(e2)
+                end
+              end
             end
           end
         end

data/lib/sequel/adapters/mysql.rb

@@ -275,7 +275,7 @@ module Sequel
       
       # Convert tinyint(1) type to boolean if convert_tinyint_to_bool is true
       def schema_column_type(db_type)
-        convert_tinyint_to_bool && db_type == 'tinyint(1)' ? :boolean : super
+        convert_tinyint_to_bool && db_type =~ /\Atinyint\(1\)/ ? :boolean : super
       end
     end
     

data/lib/sequel/adapters/mysql2.rb

@@ -124,7 +124,7 @@ module Sequel
 
       # Convert tinyint(1) type to boolean if convert_tinyint_to_bool is true
       def schema_column_type(db_type)
-        convert_tinyint_to_bool && db_type == 'tinyint(1)' ? :boolean : super
+        convert_tinyint_to_bool && db_type =~ /\Atinyint\(1\)/ ? :boolean : super
       end
     end
 

data/lib/sequel/adapters/oracle.rb

@@ -37,7 +37,7 @@ module Sequel
           dbname = opts[:host]
         end
         conn = OCI8.new(opts[:user], opts[:password], dbname, opts[:privilege])
-        conn.prefetch_rows = typecast_value_integer(opts[:prefetch_rows]) if opts[:prefetch_rows]
+        conn.prefetch_rows = typecast_value_integer(opts.fetch(:prefetch_rows, 100))
         conn.autocommit = true
         conn.non_blocking = true
         

data/lib/sequel/adapters/postgres.rb

@@ -328,10 +328,12 @@ module Sequel
               end
             rescue Exception => e
               conn.put_copy_end("ruby exception occurred while copying data into PostgreSQL")
-              raise
             ensure
               conn.put_copy_end unless e
-              conn.get_result
+              while res = conn.get_result
+                raise e if e
+                check_database_errors{res.check}
+              end
             end
           end 
         end

data/lib/sequel/adapters/shared/db2.rb

@@ -216,6 +216,9 @@ module Sequel
       FETCH_FIRST = " FETCH FIRST ".freeze
       ROWS_ONLY = " ROWS ONLY".freeze
       EMPTY_FROM_TABLE = ' FROM "SYSIBM"."SYSDUMMY1"'.freeze
+      HSTAR = "H*".freeze
+      BLOB_OPEN = "BLOB(X'".freeze
+      BLOB_CLOSE = "')".freeze
 
       # DB2 casts strings using RTRIM and CHAR instead of VARCHAR.
       def cast_sql_append(sql, expr, type)
@@ -316,6 +319,15 @@ module Sequel
         BOOL_TRUE
       end
 
+      # DB2 uses a literal hexidecimal number for blob strings
+      def literal_blob_append(sql, v)
+        if ::Sequel::DB2.use_clob_as_blob
+          super
+        else
+          sql << BLOB_OPEN << v.unpack(HSTAR).first << BLOB_CLOSE
+        end
+      end
+
       # Add a fallback table for empty from situation
       def select_from_sql(sql)
         @opts[:from] ? super : (sql << EMPTY_FROM_TABLE)