sequel 3.2.0 → 3.3.0

data/CHANGELOG

@@ -1,3 +1,43 @@
+=== 3.3.0 (2009-08-03)
+
+* Add an assocation_proxies plugin that uses proxies for associations (jeremyevans)
+
+* Have the add/remove/remove_all methods take additional arguments and pass them to the internal methods (clivecrous)
+
+* Move convert_tinyint_to_bool method from Sequel to Sequel::MySQL (jeremyevans)
+
+* Model associations now default to associating to classes in the same scope (jeremyevans, nougad) (#274)
+
+* Add Dataset#unlimited, similar to unfiltered and unordered (jeremyevans)
+
+* Make Dataset#from_self take an options hash and respect an :alias option, giving the alias to use (Phrogz)
+
+* Make the JDBC adapter accept a :convert_types option to turn off Java type conversion and double performance (jeremyevans)
+
+* Slight increase in ConnectionPool performance (jeremyevans)
+
+* SQL::WindowFunction can now be aliased/casted etc. just like SQL::Function (jeremyevans)
+
+* Model#save no longer attempts to update primary key columns (jeremyevans)
+
+* Sequel will now unescape values provided in connection strings (e.g. ado:///db?host=server%5cinstance) (jeremyevans)
+
+* Significant improvements to the ODBC and ADO adapters in general (jeremyevans)
+
+* The ADO adapter no longer attempts to use database transactions, since they never worked (jeremyevans)
+
+* Much better support for Microsoft SQL Server using the ADO, ODBC, and JDBC adapters (jeremyevans)
+
+* Support rename_column, set_column_null, set_column_type, and add_foreign_key on H2 (jeremyevans)
+
+* Support adding a column with a primary key or unique constraint to an existing table on SQLite (jeremyevans)
+
+* Support altering a column's type, null status, or default on SQLite (jeremyevans)
+
+* Fix renaming a NOT NULL column without a default on MySQL (nougad, jeremyevans) (#273)
+
+* Don't swallow DatabaseConnectionErrors when creating model subclasses (tommy.midttveit)
+
 === 3.2.0 (2009-07-02)
 
 * In the STI plugin, don't overwrite the STI field if it is already set (jeremyevans)

data/Rakefile

@@ -168,7 +168,7 @@ begin
     t.spec_opts  = spec_opts.call
   end
   
-  %w'postgres sqlite mysql informix oracle ado'.each do |adapter|
+  %w'postgres sqlite mysql informix oracle firebird'.each do |adapter|
     desc "Run #{adapter} specs without coverage"
     Spec::Rake::SpecTask.new("spec_#{adapter}") do |t|
       t.spec_files = ["spec/adapters/#{adapter}_spec.rb"] + Dir["spec/integration/*_test.rb"]

data/doc/opening_databases.rdoc

@@ -203,6 +203,13 @@ Example connections strings:
   jdbc:mysql://localhost/test?user=root&password=root
   jdbc:h2:mem:
 
+The following additional options are supported:
+
+* :convert_types - If set to false, does not attempt to convert some Java types to ruby types.
+  Setting to false roughly doubles performance when selecting large numbers of rows.
+  Note that you can't provide this option inside the connection string (as that is passed
+  directly to JDBC), you have to pass it as a separate option.
+
 === mysql 
 
 The MySQL adapter does not support the pure-ruby MySQL adapter that ships with

data/doc/release_notes/3.3.0.txt

@@ -0,0 +1,192 @@
+New Features
+------------
+
+* An association_proxies plugin has been added.  This is not a
+  full-blown proxy implemention, but it allows you to write code
+  such as:
+  
+    artist.albums.filter{num_tracks > 10}
+    
+  Without the plugin, you have to call filter specifically on the
+  association's dataset:
+  
+    artist.albums_dataset.filter{num_tracks > 10}
+
+  The plugin works by proxying array methods to the array of
+  associated objects, and all other methods to the association's
+  dataset.  This results in the following behavior:
+  
+    # Will load the associated objects (unless they are already
+    # cached), and return the length of the array
+    artist.albums.length
+    
+    # Will issue an SQL query with COUNT (even if the association
+    # is already cached), and return the result
+    artist.albums.count
+
+* The add_*/remove_*/remove_all_* association methods now take
+  additional arguments that are passed down to the
+  _add_*/_remove_*/_remove_all_* methods.  One of the things this
+  allows you to do is update additional columns in join tables for
+  many_to_many associations:
+
+    class Album
+      many_to_many :artists
+      def _add_artist(artist, values={})
+        DB[:albums_artists].
+          insert(values.merge(:album_id=>id,
+                              :artist_id=>artist.id))
+      end
+    end
+    
+    album = Album[1]
+    artist1 = Artist[2]
+    artist2 = Artist[3]
+    album.add_artist(artist1, :relationship=>'composer')
+    album.add_artist(artist2, :relationship=>'arranger')
+
+* The JDBC adapter now accepts a :convert_types option to turn off
+  Java type conversion.  The option is true by default for
+  backwards compatibility and correctness, but can be set to false
+  to double performance.  The option can be set at the database
+  and dataset levels:
+
+    DB = Sequel.jdbc('jdbc:postgresql://host/database',
+                     :convert_types=>false)
+    DB.convert_types = true
+    ds = DB[:table]
+    ds.convert_types = false
+
+* Dataset#from_self now takes an option hash and respects an
+  :alias option, giving the table alias to use.
+
+* Dataset#unlimited was added, similar to unfiltered and unordered.
+
+* SQL::WindowFunction is now a subclass of SQL::GenericExpression,
+  so you can alias it and treat it like any other SQL::Function.
+
+Other Improvements
+------------------
+
+* Microsoft SQL Server support is much, much better in Sequel 3.3.0
+  than in previous versions.  Support is pretty good with the ODBC,
+  ADO, and JDBC adapters, close to the level of support for
+  PostreSQL, MySQL, SQLite, and H2.  Improvements are too numerous
+  to list, but here are some highlights:
+  
+  * Dataset#insert now returns the primary key (identity field), so
+    it can be used easier with models.
+    
+  * Transactions can now use savepoints (except on ADO).
+  
+  * Offsets are supported when using SQL Server 2005 or 2008, using
+    a ROW_NUMBER window function.  However, you must specify an
+    order for your dataset (which you probably are already doing if
+    you are using offsets).
+
+  * Schema parsing has been implemented, though it doesn't support
+    primary key parsing (except on JDBC, since the JDBC support is
+    used there).
+  
+  * The SQL syntax Sequel uses is now much more compatible, and
+    most schema modification methods and database types now work
+    correctly.
+
+* The ADO and ODBC adapters both work much better now.  The ADO
+  adapter no longer attempts to use transactions, since I've found
+  that ADO does not give a stable native connection (and hence
+  transactions weren't possible).  I strongly recommend against
+  using the ADO adapter in production.
+
+* The H2 JDBC subadapter now supports rename_column, set_column_null,
+  set_column_type, and add_foreign_key.
+
+* Altering a columns type, null status, or default is now supported
+  on SQLite.  You can also add primary keys and unique columns.
+
+* Both the ADO and ODBC adapters now catch the native exception
+  classes and raise Sequel::DatabaseErrors.
+
+* Model classes now default to associating to other classes in the
+  same scope.  This makes it easier to use namespaced models.
+
+* The schema parser and schema dumper now support the following
+  types: nchar, nvarchar, ntext, smalldatetime, smallmoney, binary,
+  and varbinary.
+
+* You can now specify the null status for a column using :allow_null
+  in addition to :null.  This is to make it easier to use the
+  table creation methods with the results of the schema parser.
+
+* Renaming a NOT NULL column without a default now works on MySQL.
+
+* Model class initialization now raises an exception if there is a
+  problem connecting to the database.
+  
+* Connection pool performance has been increased slightly.
+
+* The literal_time method in the ODBC adapter has been fixed.
+
+* An unlikely but potential bug in the MySQL adapter has been fixed.
+
+Backwards Compatibility
+-----------------------
+
+* The convert_tinyint_to_bool setting moved from the main Sequel
+  module to the Sequel::MySQL module.  The native MySQL adapter is
+  the only adapter that converted tinyint columns to booleans when
+  the rows are returned, so you can only use the setting with the
+  native MySQL adapter.
+  
+  Additionally, the setting's behavior has changed.  When parsing
+  the schema, now only tinyint(1) columns are now considered as
+  boolean, instead of all tinyint columns. This allows you to use
+  tinyint(4) columns for storing small integers and tinyint(1)
+  columns as booleans, and not have the schema parsing support
+  consider the tinyint(4) columns as booleans.  Unfortunately,
+  due to limitations in the native MySQL driver, all tinyint
+  column values are converted to booleans upon retrieval, not just
+  tinyint(1) column values.
+  
+  Unfortunately, the previous Sequel behavior was to use the
+  default tinyint size (tinyint(4)) when creating boolean columns
+  (using the TrueClass or FalseClass generic types).  If you were
+  using the generic type support to create the columns, you should
+  modify your database to change the column type from tinyint(4) to
+  tinyint(1).
+  
+  If you use MySQL with tinyint columns, these changes have the
+  potential to break applications.  Care should be taken when
+  upgrading if these changes apply to you.
+
+* Model classes now default to associating to other classes in the
+  same scope.  It's highly unlikely anyone was relying on the
+  previous behavior, but if you have a model inside a module that
+  you are associating to a model outside of a module, you now need
+  to specify the associated class using the :class option. 
+  
+* Model#save no longer includes the primary key fields in the SET
+  clause of the UPDATE query, only in the WHERE clause.  I'm not
+  sure if this affects backwards compatibility of production code,
+  but it can break tests that expect specific SQL.
+
+* Behavior to handle empty identifiers has now been standardized.
+  If any database adapter returns an empty identifier, Sequel will
+  use 'untitled' as the identifier.  This can break backwards
+  compatibility if the adapter previously used another default and
+  you were relying on that default.  This was necessary to fix any
+  possible "interning empty string" exceptions.
+
+* On MSSQL, Sequel now uses the datetime type instead of the
+  timestamp type for generic DateTimes.  It now uses bit for the
+  TrueClass and FalseClass generic types, and image for the File
+  generic type.
+
+* Sequel now unescapes URL parts:
+
+    Sequel.connect(ado:///db?host=server%5cinstance)
+
+  However, this can break backward compatibility if you previously
+  expected it not to be unescaped.
+
+* The columns_for private SQLite Database method has been removed.

data/lib/sequel/adapters/ado.rb

@@ -1,15 +1,7 @@
 require 'win32ole'
 
 module Sequel
-  # The ADO adapter provides connectivity to ADO databases in Windows. ADO
-  # databases can be opened using a URL with the ado schema:
-  #
-  #   DB = Sequel.connect('ado://mydb')
-  # 
-  # or using the Sequel.ado method:
-  #
-  #   DB = Sequel.ado('mydb')
-  #
+  # The ADO adapter provides connectivity to ADO databases in Windows.
   module ADO
     class Database < Sequel::Database
       set_adapter_scheme :ado
@@ -19,20 +11,19 @@ module Sequel
         opts[:driver] ||= 'SQL Server'
         case opts[:driver]
         when 'SQL Server'
-          Sequel.require 'adapters/shared/mssql'
-          extend Sequel::MSSQL::DatabaseMethods
+          Sequel.require 'adapters/ado/mssql'
+          extend Sequel::ADO::MSSQL::DatabaseMethods
         end
       end
 
       # Connect to the database. In addition to the usual database options,
-      # the following option has effect:
+      # the following options have an effect:
       #
       # * :command_timeout - Sets the time in seconds to wait while attempting
-      #     to execute a command before cancelling the attempt and generating
-      #     an error. Specifically, it sets the ADO CommandTimeout property.
-      #     If this property is not set, the default of 30 seconds is used.
+      #   to execute a command before cancelling the attempt and generating
+      #   an error. Specifically, it sets the ADO CommandTimeout property.
+      #   If this property is not set, the default of 30 seconds is used.
       # * :provider - Sets the Provider of this ADO connection (for example, "SQLOLEDB")
-
       def connect(server)
         opts = server_opts(server)
         s = "driver=#{opts[:driver]};server=#{opts[:host]};database=#{opts[:database]}#{";uid=#{opts[:user]};pwd=#{opts[:password]}" if opts[:user]}"
@@ -50,14 +41,35 @@ module Sequel
       def execute(sql, opts={})
         log_info(sql)
         synchronize(opts[:server]) do |conn|
-          r = conn.Execute(sql)
-          yield(r) if block_given?
-          r
+          begin
+            r = conn.Execute(sql)
+            yield(r) if block_given?
+          rescue ::WIN32OLERuntimeError => e
+            raise_error(e)
+          end
         end
+        nil
       end
-      alias_method :do, :execute
+      alias do execute
 
       private
+      
+      # The ADO adapter doesn't support transactions, since it appears not to
+      # use a single native connection for each connection in the pool
+      def _transaction(conn)
+        th = Thread.current
+        begin
+          @transactions << th
+          yield conn
+        rescue Sequel::Rollback
+        ensure
+          @transactions.delete(th)
+        end
+      end
+      
+      def connection_pool_default_options
+        super.merge(:pool_convert_exceptions=>false)
+      end
 
       def disconnect_connection(conn)
         conn.Close
@@ -67,25 +79,8 @@ module Sequel
     class Dataset < Sequel::Dataset
       def fetch_rows(sql)
         execute(sql) do |s|
-          @columns = s.Fields.extend(Enumerable).map do |column|
-            name = column.Name.empty? ? '(no column name)' : column.Name
-            output_identifier(name)
-          end
-          
-          unless s.eof
-            s.moveFirst
-            s.getRows.transpose.each {|r| yield hash_row(r)}
-          end
-        end
-        self
-      end
-      
-      private
-      
-      def hash_row(row)
-        @columns.inject({}) do |m, c|
-          m[c] = row.shift
-          m
+          @columns = cols = s.Fields.extend(Enumerable).map{|column| output_identifier(column.Name)}
+          s.getRows.transpose.each{|r| yield cols.inject({}){|m,c| m[c] = r.shift; m}} unless s.eof
         end
       end
     end

data/lib/sequel/adapters/ado/mssql.rb

@@ -0,0 +1,30 @@
+Sequel.require 'adapters/shared/mssql'
+
+module Sequel
+  module ADO
+    # Database and Dataset instance methods for MSSQL specific
+    # support via ADO.
+    module MSSQL
+      module DatabaseMethods
+        include Sequel::MSSQL::DatabaseMethods
+        
+        # Return instance of Sequel::ADO::MSSQL::Dataset with the given opts.
+        def dataset(opts=nil)
+          Sequel::ADO::MSSQL::Dataset.new(self, opts)
+        end
+      end
+      
+      class Dataset < ADO::Dataset
+        include Sequel::MSSQL::DatasetMethods
+        
+        # Use a nasty hack of multiple SQL statements in the same call and
+        # having the last one return the most recently inserted id.  This
+        # is necessary as ADO doesn't provide a consistent native connection.
+        def insert(values={})
+          return super if @opts[:sql]
+          with_sql("SET NOCOUNT ON; #{insert_sql(values)}; SELECT CAST(SCOPE_IDENTITY() AS INTEGER)").single_value
+        end
+      end
+    end
+  end
+end

data/lib/sequel/adapters/jdbc.rb

@@ -58,8 +58,8 @@ module Sequel
         Java::oracle.jdbc.driver.OracleDriver
       end,
       :sqlserver=>proc do |db|
-        Sequel.require 'adapters/shared/mssql'
-        db.extend(Sequel::MSSQL::DatabaseMethods)
+        Sequel.require 'adapters/jdbc/mssql'
+        db.extend(Sequel::JDBC::MSSQL::DatabaseMethods)
         com.microsoft.sqlserver.jdbc.SQLServerDriver
       end,
       :h2=>proc do |db|
@@ -88,12 +88,18 @@ module Sequel
       # The type of database we are connecting to
       attr_reader :database_type
       
+      # Whether to convert some Java types to ruby types when retrieving rows.
+      # True by default, can be set to false to roughly double performance when
+      # fetching rows.
+      attr_accessor :convert_types
+      
       # Call the DATABASE_SETUP proc directly after initialization,
       # so the object always uses sub adapter specific code.  Also,
       # raise an error immediately if the connection doesn't have a
       # uri, since JDBC requires one.
       def initialize(opts)
         @opts = opts
+        @convert_types = opts.include?(:convert_types) ? typecast_value_boolean(opts[:convert_types]) : true
         raise(Error, "No connection string specified") unless uri
         if match = /\Ajdbc:([^:]+)/.match(uri) and prok = DATABASE_SETUP[match[1].to_sym]
           prok.call(self)
@@ -279,7 +285,7 @@ module Sequel
                 cps.execute
               when :insert
                 cps.executeUpdate
-                last_insert_id(conn, opts)
+                last_insert_id(conn, opts.merge(:prepared=>true))
               else
                 cps.executeUpdate
               end
@@ -418,6 +424,17 @@ module Sequel
         end
       end
       
+      # Whether to convert some Java types to ruby types when retrieving rows.
+      # Uses the database's setting by default, can be set to false to roughly
+      # double performance when fetching rows.
+      attr_accessor :convert_types
+      
+      # Use the convert_types default setting from the database
+      def initialize(db, opts={})
+        @convert_types = db.convert_types
+        super
+      end
+      
       # Correctly return rows from the database and return them as hashes.
       def fetch_rows(sql, &block)
         execute(sql){|result| process_result_set(result, &block)}
@@ -470,10 +487,16 @@ module Sequel
         i = 0
         meta.getColumnCount.times{cols << [output_identifier(meta.getColumnLabel(i+=1)), i]}
         @columns = cols.map{|c| c.at(0)}
+        row = {}
+        blk = if @convert_types
+          lambda{|n, i| row[n] = convert_type(result.getObject(i))}
+        else
+          lambda{|n, i| row[n] = result.getObject(i)}
+        end
         # get rows
         while result.next
           row = {}
-          cols.each{|n, i| row[n] = convert_type(result.getObject(i))}
+          cols.each(&blk)
           yield row
         end
       end