sequel 3.30.0 → 3.31.0

data/CHANGELOG

@@ -1,3 +1,43 @@
+=== 3.31.0 (2012-01-03)
+
+* Dataset#from no longer handles :a__b__c___d as a.b.c AS d (jeremyevans)
+
+* Support many_to_one associations with the same name as their column, using the :key_column option (jeremyevans)
+
+* Add Model.def_column_alias for defining alias methods for columns (jeremyevans)
+
+* Support :server option in Dataset#import and #multi_insert (jeremyevans)
+
+* Respect existing RETURNING/OUTPUT clauses in #import/#multi_insert on PostgreSQL/MSSQL (jeremyevans)
+
+* Support :return=>:primary_key option to Dataset#import and #multi_insert (jeremyevans)
+
+* Correctly handle return value for Dataset#insert with column array and value array on PostgreSQL <8.2 (jeremyevans)
+
+* Dataset#insert_multiple now returns an array of inserted primary keys (jeremyevans) (#408)
+
+* Support RETURNING with DELETE and UPDATE on PostgreSQL 8.2+ (funny-falcon)
+
+* Raise error if tables from two separate schema are detected when parsing the schema for a single table on PostgreSQL (jeremyevans)
+
+* Handle clob types as string instead of blob on H2 (jeremyevans)
+
+* Add database type support to the mock adapter, e.g. mock://postgres (jeremyevans)
+
+* Allow creation of full text indexes on Microsoft SQL Server, but you need to provide a :key_index option (jeremyevans)
+
+* Allow Dataset#full_text_search usage with prepared statements (jeremyevans)
+
+* Make Dataset#exists use a PlaceholderLiteralString so it works with prepared statements (jeremyevans)
+
+* Fix Dataset#empty? for datasets with offsets when offset support is emulated (jeremyevans)
+
+* Add Dataset#group_rollup and #group_cube methods for GROUP BY ROLLUP and CUBE support (jeremyevans)
+
+* Add support for custom serialization formats to the serialization plugin (jeremyevans)
+
+* Support a :login_timeout option in the jdbc adapter (glebpom) (#406)
+
 === 3.30.0 (2011-12-01)
 
 * Handle usage of on_duplicate_key_update in MySQL prepared statements (jeremyevans) (#404)

data/Rakefile

@@ -117,15 +117,25 @@ begin
     end
   end
 
-  spec_with_cov = lambda do |name, files, d|
+  sc = spec_with_cov = lambda do |name, files, d|
     spec.call(name, files, d)
     t = spec.call("#{name}_cov", files, "#{d} with coverage")
     t.rcov = true
     t.rcov_opts = File.read("spec/rcov.opts").split("\n")
+    t
+  end
+
+  if RUBY_VERSION >= '1.8.7'
+    eval <<-END
+      spec_with_cov = lambda do |*x, &b|
+        t = sc.call(*x)
+        b.call(t) if b
+      end
+    END
   end
   
   task :default => [:spec]
-  spec_with_cov.call("spec", Dir["spec/{core,model}/*_spec.rb"], "Run core and model specs")
+  spec_with_cov.call("spec", Dir["spec/{core,model}/*_spec.rb"], "Run core and model specs"){|t| t.rcov_opts.concat(%w'--exclude "lib/sequel/adapters/([a-ln-z]|m[a-np-z])"')}
   spec.call("spec_core", Dir["spec/core/*_spec.rb"], "Run core specs")
   spec.call("spec_model", Dir["spec/model/*_spec.rb"], "Run model specs")
   spec_with_cov.call("spec_plugin", Dir["spec/extensions/*_spec.rb"], "Run extension/plugin specs")

data/doc/association_basics.rdoc

@@ -1483,6 +1483,34 @@ If the default modification methods would not do what you want, and you
 don't plan on overriding the internal modification methods to do what you
 want, it may be best to set this option to true.
 
+==== :key_column [+many_to_one+]
+
+This option exists to make it possible to create a +many_to_one+ association
+with the same name as the foreign key it uses.  Before this option was
+introduced, this was problematic, since the :key option was used to refer
+both to the model methods and the underlying database columns (which are
+usually the same).
+
+If you have a legacy database with a foreign key that is the same as the name
+of the association you would like to use, you should create a column alias
+for the foreign key and use this option:
+
+  # Example schema:
+  #  albums           artists
+  #   :id        /-->  :id
+  #   :artist --/      :name
+  #   :name
+
+  class Album < Sequel::Model
+    def_column_alias(:artist_id, :artist)
+    many_to_one :artist, :key_column=>:artist
+  end
+
+Note that if you have control over the database, it's better to just rename
+the foreign key column than use this option.  Similarly, if you can think
+of another association name that will work, it's wiser to rename the
+association than use this option.  Think of this option as a last resort.
+
 ==== :validate
 
 Set to false to not validate when implicitly saving any associated object.

data/doc/dataset_filtering.rdoc

@@ -59,6 +59,14 @@ Ranges (both inclusive and exclusive) can also be used:
   items.filter(:price => 100...200).sql
   #=> "SELECT * FROM items WHERE (price >= 100 AND price < 200)"
 
+== Filtering using an array
+
+If you need to select multiple items from a dataset, you can supply an array:
+
+	item_array = [1, 38, 47, 99]
+	items.filter(:id => item_array).sql
+	#=> "SELECT * FROM items WHERE (id IN (1, 38, 47, 99))"
+
 == Filtering using expressions
 
 Sequel allows you to use ruby expressions directly in the call to filter:

data/doc/opening_databases.rdoc

@@ -276,6 +276,7 @@ The following additional options are supported:
                   Setting to false roughly doubles performance when selecting large numbers of rows.
                   Note that you can't provide this option inside the connection string (as that is passed
                   directly to JDBC), you have to pass it as a separate option.
+:login_timeout :: Set the login timeout on the JDBC connection (in seconds).
 
 === mysql 
 

data/doc/release_notes/3.31.0.txt

@@ -0,0 +1,146 @@
+= New Features
+
+* The serialization plugin can now support custom serialization
+  formats, by supplying a serializer/deserializer pair of
+  callable objects.  You can also register custom deserializers
+  via Sequel::Plugins::Serialization.register_format, so that
+  they can be referenced by name.  Example:
+  
+    Sequel::Plugins::Serialization.register_format(:reverse,
+      lambda{|v| v.reverse},
+      lambda{|v| v.reverse})
+    class User < Sequel::Model
+      serialize_attributes :reverse, :password
+    end
+
+* Dataset#import and #multi_insert now support a
+  :return=>:primary_key option.  When this option is used, the
+  methods return an array of primary key values, one for each
+  inserted row.  Usage of this option on MySQL requires that a
+  separate query be issued per row (instead of the single
+  query for all rows that MySQL would usually use).
+  
+* PostgreSQL can now use Dataset#returning in conjunction with
+  import/multi_insert to set a custom column to return.
+  
+* Microsoft SQL Server can now use Dataset#output in conjection with
+  import/multi_insert to set a custom column to return.
+  
+* Dataset#import and #multi_insert now respect a :server option to
+  set the server/shard on which to execute the queries.
+  Additionally, options given to this method are also passed to
+  Dataset#transaction.
+
+* Dataset#insert_multiple now returns an array of inserted primary
+  keys.
+
+* Model.def_column_alias has been added to make it easy to create
+  alias methods for columns.  This is useful if you have a legacy
+  database and want to create friendly method names for the
+  underlying columns.  Note that this alias only affects the setter
+  and getter methods.  This does not affect the dataset level, so you
+  still need to use the actual column names in dataset filters.
+
+* many_to_one associations can now have the same name as the related
+  foreign key column, using the :key_column option.  Use of this
+  feature is not recommended, as it is much better to either rename
+  the column or rename the association.  Here's an example of usage:
+  
+    # Example schema:
+    #  albums           artists
+    #   :id        /-->  :id
+    #   :artist --/      :name
+    #   :name
+    class Album < Sequel::Model
+      def_column_alias(:artist_id, :artist)
+      many_to_one :artist, :key_column=>:artist
+    end
+
+* The mock adapter can now mock out database types, by providing a
+  shared adapter name as the host (e.g. mock://postgres).  This
+  emulation is not perfect, but in most cases it allows you to see
+  what SQL Sequel would generate on a given database without needing
+  to install the required database driver.
+
+* Sequel now supports creating full text indexes on Microsoft SQL
+  Server.  Before using it, you must have previously setup a default
+  full text search catalog, and you need to provide a :key_index
+  option with an index name symbol.
+  
+* Dataset#group_rollup and #group_cube methods have been added for
+  GROUP BY ROLLUP and GROUP BY CUBE support.  These features are in
+  a recent SQL standard, and they are supported to various degrees on
+  Microsoft SQL Server, DB2, Oracle, MySQL, and Derby.
+
+* Dataset#full_text_search on Microsoft SQL Server now supports
+  multiple search terms.
+  
+* The jdbc adapter now supports a :login_timeout option, giving the
+  timeout in seconds.
+
+= Other Improvements
+
+* Dataset#exists can now be used with prepared statement
+  placeholders.
+
+* Dataset#full_text_search can now be used with prepared statement
+  placeholders on PostgreSQL, MySQL, and Microsoft SQL Server.
+  
+* If tables from two separate schema are detected when parsing the
+  schema for a table on PostgreSQL, an error is now raised.
+  Previously, no error was raised, which led to weird errors later,
+  such as duplicate columns in a model's primary_key.
+
+* RETURNING is now supported with UPDATE/DELETE on PostgreSQL 8.2+.
+  Previously, Sequel only supported it on 9.1+, but PostgreSQL
+  introduced support for it in 8.2.
+
+* The shared postgres adapter now correctly handles the return value
+  for Dataset#insert if you provide a separate column array and value
+  array on PostgreSQL < 8.2.
+  
+* Handle case in the PostgreSQL adapter where the server version
+  cannot be determined via a query.
+  
+* H2 clob types are now treated as string instead of as blob.
+  Treating clob as blob breaks on H2, as it doesn't automatically
+  hex-unescape the input for clobs as it does for blobs.
+  
+* Dataset#empty? now works correctly when the dataset has an offset
+  and offset support is being emulated.
+
+* The mock adapter no longer defaults to downcasing identifiers on
+  output.
+
+= Backwards Compatibility
+
+* Dataset#exists now returns a PlaceholderLiteralString instead of a
+  LiteralString, which could potentially break some code.  If you
+  would like a String returned, you can pass the returned object to
+  Dataset#literal:
+  
+    dataset.literal(dataset.exists)
+
+* Dataset#from no longer handles :a__b__c___d as "a.b.c AS d".  This
+  was not the intended behavior, and nowhere else in Sequel is a
+  symbol treated that way.  Now, Dataset#from is consistent with the
+  rest of Sequel, using "a.b__c AS d".  This should only affect
+  people in very rare cases, as most databases don't use three level
+  qualified tables.  One exception is Microsoft SQL Server, which can
+  use three level qualified tables for cross-database access.
+  
+* Previously, Dataset#insert_multiple returned an array of hashes, now
+  it returns an array of primary key values.
+
+* Dataset#EXRACT_CLOSE in the shared sqlite adapter has been renamed to
+  Dataset#EXTRACT_CLOSE.
+  
+* Dataset::StoredProcedureMethods::SQL_QUERY_TYPE and
+  Dataset::ArgumentMapper::SQL_QUERY_TYPE constants have been removed,
+  as have related sql_query_type private methods.
+
+* The serialization plugin was significantly refactored.
+  Model.serialization_map now contains a callable object instead of a
+  Symbol, and Model.serialization_format has been removed.
+  Model.define_serialized_attribute_accessors private method now takes
+  two callable objects before the columns, instead of a single symbol.

data/lib/sequel/adapters/jdbc.rb

@@ -200,6 +200,7 @@ module Sequel
           args = [uri(opts)]
           args.concat([opts[:user], opts[:password]]) if opts[:user] && opts[:password]
           begin
+            JavaSQL::DriverManager.setLoginTimeout(opts[:login_timeout]) if opts[:login_timeout]
             JavaSQL::DriverManager.getConnection(*args)
           rescue => e
             raise e unless driver
@@ -546,17 +547,17 @@ module Sequel
         # Execute the prepared SQL using the stored type and
         # arguments derived from the hash passed to call.
         def execute(sql, opts={}, &block)
-          super(self, {:arguments=>bind_arguments, :type=>sql_query_type}.merge(opts), &block)
+          super(self, {:arguments=>bind_arguments}.merge(opts), &block)
         end
         
         # Same as execute, explicit due to intricacies of alias and super.
         def execute_dui(sql, opts={}, &block)
-          super(self, {:arguments=>bind_arguments, :type=>sql_query_type}.merge(opts), &block)
+          super(self, {:arguments=>bind_arguments}.merge(opts), &block)
         end
         
         # Same as execute, explicit due to intricacies of alias and super.
         def execute_insert(sql, opts={}, &block)
-          super(self, {:arguments=>bind_arguments, :type=>sql_query_type}.merge(opts), &block)
+          super(self, {:arguments=>bind_arguments, :type=>:insert}.merge(opts), &block)
         end
       end
       
@@ -569,17 +570,17 @@ module Sequel
         
         # Execute the database stored procedure with the stored arguments.
         def execute(sql, opts={}, &block)
-          super(@sproc_name, {:args=>@sproc_args, :sproc=>true, :type=>sql_query_type}.merge(opts), &block)
+          super(@sproc_name, {:args=>@sproc_args, :sproc=>true}.merge(opts), &block)
         end
         
         # Same as execute, explicit due to intricacies of alias and super.
         def execute_dui(sql, opts={}, &block)
-          super(@sproc_name, {:args=>@sproc_args, :sproc=>true, :type=>sql_query_type}.merge(opts), &block)
+          super(@sproc_name, {:args=>@sproc_args, :sproc=>true}.merge(opts), &block)
         end
         
         # Same as execute, explicit due to intricacies of alias and super.
         def execute_insert(sql, opts={}, &block)
-          super(@sproc_name, {:args=>@sproc_args, :sproc=>true, :type=>sql_query_type}.merge(opts), &block)
+          super(@sproc_name, {:args=>@sproc_args, :sproc=>true, :type=>:insert}.merge(opts), &block)
         end
       end
       

data/lib/sequel/adapters/jdbc/derby.rb

@@ -169,6 +169,11 @@ module Sequel
           end
         end
 
+        # Derby supports GROUP BY ROLLUP (but not CUBE)
+        def supports_group_rollup?
+          true
+        end
+
         # Derby does not support IS TRUE.
         def supports_is_true?
           false

data/lib/sequel/adapters/jdbc/h2.rb

@@ -100,6 +100,11 @@ module Sequel
         def primary_key_index_re
           PRIMARY_KEY_INDEX_RE
         end
+
+        # Treat clob as string instead of blob
+        def schema_column_type(db_type)
+          db_type == 'clob' ? :string : super
+        end
       end
       
       # Dataset class for H2 datasets accessed via JDBC.
@@ -158,7 +163,7 @@ module Sequel
         #JAVA_H2_CLOB = Java::OrgH2Jdbc::JdbcClob
 
         class ::Sequel::JDBC::Dataset::TYPE_TRANSLATOR
-          def h2_clob(v) Sequel::SQL::Blob.new(v.getSubString(1, v.length)) end
+          def h2_clob(v) v.getSubString(1, v.length) end
         end
 
         H2_CLOB_METHOD = TYPE_TRANSLATOR_INSTANCE.method(:h2_clob)

data/lib/sequel/adapters/mock.rb

@@ -30,6 +30,20 @@ module Sequel
     class Database < Sequel::Database
       set_adapter_scheme :mock
 
+      # Map of database type names to module names, used for handling
+      # mock adapters for specific database types.
+      SHARED_ADAPTERS = {
+        'access'=>'Access',
+        'db2'=>'DB2',
+        'firebird'=>'Firebird',
+        'informix'=>'Informix',
+        'mssql'=>'MSSQL',
+        'mysql'=>'MySQL',
+        'oracle'=>'Oracle',
+        'postgres'=>'Postgres',
+        'sqlite'=>'SQLite'
+      }
+
       # Set the autogenerated primary key integer
       # to be returned when running an insert query.
       # Argument types supported:
@@ -63,7 +77,7 @@ module Sequel
       # Hash :: Always yield a single row with this hash
       # Array of Hashes :: Yield separately for each hash in this array
       # Array (otherwise) :: First retrieval gets the first value
-      #          in the array, second gets the second value, etc.
+      #                      in the array, second gets the second value, etc.
       # Proc :: Called with the select SQL query, uses
       #         the value returned, which should be a hash or
       #         array of hashes.
@@ -95,6 +109,11 @@ module Sequel
       def initialize(opts={})
         super
         opts = @opts
+        if mod_name = SHARED_ADAPTERS[opts[:host]]
+          require "sequel/adapters/shared/#{opts[:host]}"
+          extend Sequel.const_get(mod_name)::DatabaseMethods
+          extend_datasets Sequel.const_get(mod_name)::DatasetMethods
+        end
         self.autoid = opts[:autoid]
         self.columns = opts[:columns]
         self.fetch = opts[:fetch]
@@ -265,7 +284,7 @@ module Sequel
         nil
       end
 
-      def identifier_input_method_default
+      def identifier_output_method_default
         nil
       end
     end

data/lib/sequel/adapters/shared/db2.rb

@@ -226,6 +226,16 @@ module Sequel
         end
       end
 
+      # DB2 supports GROUP BY CUBE
+      def supports_group_cube?
+        true
+      end
+
+      # DB2 supports GROUP BY ROLLUP
+      def supports_group_rollup?
+        true
+      end
+
       # DB2 does not support IS TRUE.
       def supports_is_true?
         false

data/lib/sequel/adapters/shared/mssql.rb

@@ -234,11 +234,11 @@ module Sequel
       # support for clustered index type
       def index_definition_sql(table_name, index)
         index_name = index[:name] || default_index_name(table_name, index[:columns])
-        clustered = index[:type] == :clustered
-        if index[:where]
-          raise Error, "Partial indexes are not supported for this database"
+        raise Error, "Partial indexes are not supported for this database" if index[:where]
+        if index[:type] == :full_text
+          "CREATE FULLTEXT INDEX ON #{quote_schema_table(table_name)} #{literal(index[:columns])} KEY INDEX #{literal(index[:key_index])}"
         else
-          "CREATE #{'UNIQUE ' if index[:unique]}#{'CLUSTERED ' if clustered}INDEX #{quote_identifier(index_name)} ON #{quote_schema_table(table_name)} #{literal(index[:columns])}"
+          "CREATE #{'UNIQUE ' if index[:unique]}#{'CLUSTERED ' if index[:type] == :clustered}INDEX #{quote_identifier(index_name)} ON #{quote_schema_table(table_name)} #{literal(index[:columns])}"
         end
       end
     end
@@ -345,7 +345,8 @@ module Sequel
 
       # MSSQL uses the CONTAINS keyword for full text search
       def full_text_search(cols, terms, opts = {})
-        filter("CONTAINS (#{literal(cols)}, #{literal(terms)})")
+        terms = "\"#{terms.join('" OR "')}\"" if terms.is_a?(Array)
+        filter("CONTAINS (?, ?)", cols, terms)
       end
 
       # Use the OUTPUT clause to get the value of all columns for the newly inserted record.
@@ -418,6 +419,16 @@ module Sequel
         db.server_version(@opts[:server])
       end
 
+      # MSSQL 2005+ supports GROUP BY CUBE.
+      def supports_group_cube?
+        is_2005_or_later?
+      end
+
+      # MSSQL 2005+ supports GROUP BY ROLLUP
+      def supports_group_rollup?
+        is_2005_or_later?
+      end
+
       # MSSQL supports insert_select via the OUTPUT clause.
       def supports_insert_select?
         supports_output_clause? && !opts[:disable_insert_output]
@@ -464,6 +475,24 @@ module Sequel
       end
       
       protected
+      
+      # If returned primary keys are requested, use OUTPUT unless already set on the
+      # dataset.  If OUTPUT is already set, use existing returning values.  If OUTPUT
+      # is only set to return a single columns, return an array of just that column.
+      # Otherwise, return an array of hashes.
+      def _import(columns, values, opts={})
+        if opts[:return] == :primary_key && !@opts[:output]
+          output(nil, [SQL::QualifiedIdentifier.new(:inserted, first_primary_key)])._import(columns, values, opts)
+        elsif @opts[:output]
+          statements = multi_insert_sql(columns, values)
+          @db.transaction(opts.merge(:server=>@opts[:server])) do
+            statements.map{|st| with_sql(st)}
+          end.first.map{|v| v.length == 1 ? v.values.first : v}
+        else
+          super
+        end
+      end
+
       # MSSQL does not allow ordering in sub-clauses unless 'top' (limit) is specified
       def aggregate_dataset
         (options_overlap(Sequel::Dataset::COUNT_FROM_SELF_OPTS) && !options_overlap([:limit])) ? unordered.from_self : super
@@ -496,6 +525,12 @@ module Sequel
       end
       alias update_from_sql delete_from2_sql
       
+      # Return the first primary key for the current table.  If this table has
+      # multiple primary keys, this will only return one of them.  Used by #_import.
+      def first_primary_key
+        @db.schema(self).map{|k, v| k if v[:primary_key] == true}.compact.first
+      end
+
       # MSSQL raises an error if you try to provide more than 3 decimal places
       # for a fractional timestamp.  This probably doesn't work for smalldatetime
       # fields.