sequel 3.25.0 → 3.26.0

data/CHANGELOG

@@ -1,3 +1,31 @@
+=== 3.26.0 (2011-08-01)
+
+* Fix bug in default connection pool if a disconnect error is raised and the disconnection_proc also raises an error (jeremyevans)
+
+* Disallow eager loading via eager of many_*_many associations with :eager_graph option (jeremyevans)
+
+* Major speedup in dataset creation (jeremyevans)
+
+* Replace internal implementation of eager_graph with much faster version (jeremyevans)
+
+* Don't treat strings with leading zeros as octal format in the default typecasting (jeremyevans)
+
+* Fix literalization of Date, Time, and DateTime values on Microsoft Access (jeremyevans)
+
+* Fix handling of nil values with the pure-Java version of nokogiri in the xml_serializer plugin (jeremyevans)
+
+* Make identity_map plugin work with standard eager loading of many_to_many and many_through_many associations (jeremyevans)
+
+* Make create_table! only attempt to drop the table if it already exists (jeremyevans)
+
+* Remove custom table_exists? implementations in the oracle and postgres adapters (jeremyevans)
+
+* Handle another type of disconnection in the postgres adapter (jeremyevans)
+
+* Handle disconnections in the ado adapter and do postgres subadapter (jeremyevans)
+
+* Recognize disconnections when issuing BEGIN/ROLLBACK/COMMIT statements (jeremyevans) (#368)
+
 === 3.25.0 (2011-07-01)
 
 * Work with tiny_tds-0.4.5 in the tinytds adapter, older versions are no longer supported (jeremyevans)

data/README.rdoc

@@ -573,7 +573,7 @@ You can execute custom code when creating, updating, or deleting records by defi
 
 Note the use of +super+ if you define your own hook methods.  Almost all <tt>Sequel::Model</tt> class and instance methods (not just hook methods) can be overridden safely, but you have to make sure to call +super+ when doing so, otherwise you risk breaking things.
 
-For the example above, you should probably use a database trigger if you can.  Hooks can be used for data integrity, but they will only enforce that integrity when you are modifying the database through model instances.  If you plan on allowing any other access to the database, it's best to use database triggers for data integrity.
+For the example above, you should probably use a database trigger if you can.  Hooks can be used for data integrity, but they will only enforce that integrity when you are modifying the database through model instances.  If you plan on allowing any other access to the database, it's best to use database triggers and constraints for data integrity.
 
 === Deleting records
 
@@ -681,9 +681,9 @@ Associations can be eagerly loaded via +eager+ and the <tt>:eager</tt> associati
   # Loads all people, their posts, their posts' tags, replies to those posts,
   # the person for each reply, the tag for each reply, and all posts and
   # replies that have that tag.  Uses a total of 8 queries.
-  Person.eager(:posts=>{:replies=>[:person, {:tags=>{:posts, :replies}}]}).all
+  Person.eager(:posts=>{:replies=>[:person, {:tags=>[:posts, :replies]}]}).all
 
-In addition to using +eager+, you can also use +eager_graph+, which will use a single query to get the object and all associated objects.  This may be necessary if you want to filter or order the result set based on columns in associated tables.  It works with cascading as well, the syntax is exactly the same.  Note that using eager_graph to eagerly load multiple *_to_many associations will cause the result set to be a cartesian product, so you should be very careful with your filters when using it in that case.
+In addition to using +eager+, you can also use +eager_graph+, which will use a single query to get the object and all associated objects.  This may be necessary if you want to filter or order the result set based on columns in associated tables.  It works with cascading as well, the API is very similar.  Note that using +eager_graph+ to eagerly load multiple <tt>*_to_many</tt> associations will cause the result set to be a cartesian product, so you should be very careful with your filters when using it in that case.
 
 You can dynamically customize the eagerly loaded dataset by using using a proc.  This proc is passed the dataset used for eager loading, and should return a modified copy of that dataset:
 

data/Rakefile

@@ -1,11 +1,6 @@
 require "rake"
 require "rake/clean"
 require "rake/gempackagetask"
-begin
-  require "hanna/rdoctask"
-rescue LoadError
-  require "rake/rdoctask"
-end
 
 NAME = 'sequel'
 VERS = lambda do
@@ -13,8 +8,6 @@ VERS = lambda do
   Sequel.version
 end
 CLEAN.include ["**/.*.sw?", "sequel-*.gem", ".config", "rdoc", "coverage", "www/public/*.html", "www/public/rdoc*", '**/*.rbc']
-RDOC_DEFAULT_OPTS = ["--quiet", "--line-numbers", "--inline-source", '--title', 'Sequel: The Database Toolkit for Ruby']
-RDOC_OPTS = RDOC_DEFAULT_OPTS + ['--main', 'README.rdoc']
 
 # Gem Packaging and Release
 
@@ -40,7 +33,20 @@ end
 
 ### RDoc
 
-Rake::RDocTask.new do |rdoc|
+RDOC_DEFAULT_OPTS = ["--quiet", "--line-numbers", "--inline-source", '--title', 'Sequel: The Database Toolkit for Ruby']
+
+rdoc_task_class = begin
+  require "rdoc/task"
+  RDOC_DEFAULT_OPTS.concat(['-f', 'hanna'])
+  RDoc::Task
+rescue LoadError
+  require "rake/rdoctask"
+  Rake::RDocTask
+end
+
+RDOC_OPTS = RDOC_DEFAULT_OPTS + ['--main', 'README.rdoc']
+
+rdoc_task_class.new do |rdoc|
   rdoc.rdoc_dir = "rdoc"
   rdoc.options += RDOC_OPTS
   rdoc.rdoc_files.add %w"README.rdoc CHANGELOG MIT-LICENSE lib/**/*.rb doc/*.rdoc doc/release_notes/*.txt"
@@ -56,19 +62,19 @@ end
 desc "Make rdoc for website"
 task :website_rdoc=>[:website_rdoc_main, :website_rdoc_adapters, :website_rdoc_plugins]
 
-Rake::RDocTask.new(:website_rdoc_main) do |rdoc|
+rdoc_task_class.new(:website_rdoc_main) do |rdoc|
   rdoc.rdoc_dir = "www/public/rdoc"
   rdoc.options += RDOC_OPTS
   rdoc.rdoc_files.add %w"README.rdoc CHANGELOG MIT-LICENSE lib/*.rb lib/sequel/*.rb lib/sequel/{connection_pool,dataset,database,model}/*.rb doc/*.rdoc doc/release_notes/*.txt lib/sequel/extensions/migration.rb"
 end
 
-Rake::RDocTask.new(:website_rdoc_adapters) do |rdoc|
+rdoc_task_class.new(:website_rdoc_adapters) do |rdoc|
   rdoc.rdoc_dir = "www/public/rdoc-adapters"
   rdoc.options += RDOC_DEFAULT_OPTS + %w'--main Sequel'
   rdoc.rdoc_files.add %w"lib/sequel/adapters/**/*.rb"
 end
 
-Rake::RDocTask.new(:website_rdoc_plugins) do |rdoc|
+rdoc_task_class.new(:website_rdoc_plugins) do |rdoc|
   rdoc.rdoc_dir = "www/public/rdoc-plugins"
   rdoc.options += RDOC_DEFAULT_OPTS + %w'--main Sequel'
   rdoc.rdoc_files.add %w"lib/sequel/{extensions,plugins}/**/*.rb"

data/doc/release_notes/3.26.0.txt

@@ -0,0 +1,88 @@
+= Performance Enhancements
+
+* The internal implementation of eager_graph has been made 75% to
+  225% faster than before, with greater benefits to more complex
+  graphs.
+
+* Dataset creation has been made much faster (2.5x on 1.8 and 4.4x on
+  1.9), and dataset cloning has been made significantly faster (40%
+  on 1.8 and 20% on 1.9).
+
+= Other Improvements
+
+* Strings passed to setter methods for integer columns are no longer
+  considered to be in octal format if they include leading zeroes.
+  The previous behavior was never intended, but was a side effect of
+  using Kernel#Integer.  Strings with leading zeroes are now treated
+  as decimal, and you can still use the 0x prefix to treat them as
+  hexidecimal.  If anyone was relying on the old octal behavior, let
+  me know and I'll add an extension that restores the octal behavior.
+  
+* The identity_map plugin now works with the standard eager loading
+  of many_to_many and many_through_many associations.
+  
+* Database#create_table! now only attempts to drop the table if it
+  already exists.  Previously, it attempted to drop the table
+  unconditionally ignoring any errors, which resulted in misleading
+  error messages if dropping the table raised an error caused by
+  permissions or referential integrity issues.
+
+* The default connection pool now correctly handles the case where a
+  disconnect error is raised and an exception is raised while
+  running the disconnection proc.
+  
+* Disconnection errors are now detected when issuing transaction
+  statements such as BEGIN/ROLLBACK/COMMIT.  Previously, these
+  statements did not handle disconnect errors on most adapters.
+  
+* More disconnection errors are now detected.  Specifically, the ado
+  adapter and do postgres subadapter now handle disconnect errors,
+  and the postgres adapter handles more types of disconnect errors.
+  
+* Database#table_exists? now always issues a query to select from the
+  table, it no longer attempts to parse the schema to determine the
+  information on PostgreSQL and Oracle.
+
+* Date, DateTime, and Time values are now literalized correctly on
+  Microsoft Access.
+
+* Connecting with the mysql adapter with an options hash now works if
+  the :port option is a string, which makes it easier to use when the
+  connection information is stored in YAML.
+
+* The xml_serializer plugin now works around a bug in pure-Java
+  nokogiri regarding the handling of nil values.
+
+* Nicer error messages are now used if there is an attempt to call
+  an invalid or restricted setter method.
+
+* The RDocs are now formatted with hanna-nouveau, which allows for
+  section ordering, so the Database and Dataset RDoc pages are
+  more friendly.
+
+= Backwards Compatibility
+
+* If you call a Dataset method such as #each on an eager_graphed
+  dataset, you now get plain hashes that have column alias symbol
+  keys and their values. Previously, you got a graphed response with
+  table alias keys and model values.  It's not wise to depend on the
+  behavior, the only supported way of returning records when eager
+  loading is to use #all.
+
+* An error is now raised if you attempt to eager load via
+  Dataset#eager a many_to_many association that includes an
+  :eager_graph option.  Previously, incorrect SQL would have been
+  generated and an error raised by the database.
+  
+* Datasets are no longer guaranteed to have @row_proc,
+  @indentifier_input_method, and @identifier_output_method defined
+  as instance variables.  You should be be using methods to access
+  them anyway.
+
+* Database#table_exists? on PostgreSQL no longer accepts an options
+  hash.  Previously, you could use a :schema option.  You must now
+  provide the schema inside the table argument (e.g. :schema__table).
+
+* If you want to use the rdoc tasks in Sequel's Rakefile, and you are
+  still using the hanna RDoc template with RDoc 2.3, you need to
+  upgrade to using hanna-nouveau with RDoc 3.8+.

data/lib/sequel/adapters/ado.rb

@@ -4,6 +4,8 @@ module Sequel
   # The ADO adapter provides connectivity to ADO databases in Windows.
   module ADO
     class Database < Sequel::Database
+      DISCONNECT_ERROR_RE = /Communication link failure/
+
       set_adapter_scheme :ado
 
       def initialize(opts)
@@ -87,9 +89,17 @@ module Sequel
         end
       end
       
+      def database_error_classes
+        [::WIN32OLERuntimeError]
+      end
+
       def disconnect_connection(conn)
         conn.Close
       end
+
+      def disconnect_error?(e, opts)
+        super || (e.is_a?(::WIN32OLERuntimeError) && e.message =~ DISCONNECT_ERROR_RE)
+      end
     end
     
     class Dataset < Sequel::Dataset

data/lib/sequel/adapters/do.rb

@@ -36,6 +36,8 @@ module Sequel
     # Sequel::DataObjects::Database object, or hack DataObjects (or Extlib) to
     # use a pool size at least as large as the pool size being used by Sequel.
     class Database < Sequel::Database
+      DISCONNECT_ERROR_RE = /terminating connection due to administrator command/
+
       set_adapter_scheme :do
       
       # Call the DATABASE_SETUP proc directly after initialization,
@@ -145,10 +147,20 @@ module Sequel
         :execute_non_query
       end
       
+      # dataobjects uses the DataObjects::Error class as the main error class.
+      def database_error_classes
+        [::DataObjects::Error]
+      end
+
       # Close the given database connection.
       def disconnect_connection(c)
         c.close
       end
+
+      # Recognize DataObjects::ConnectionError instances as disconnect errors.
+      def disconnect_error?(e, opts)
+        super || (e.is_a?(::DataObjects::Error) && (e.is_a?(::DataObjects::ConnectionError) || e.message =~ DISCONNECT_ERROR_RE))
+      end
       
       # Execute SQL on the connection by creating a command first
       def log_connection_execute(conn, sql)

data/lib/sequel/adapters/jdbc.rb

@@ -285,6 +285,12 @@ module Sequel
         c.close
       end
       
+      # Raise a disconnect error if the SQL state of the cause of the exception indicates so.
+      def disconnect_error?(exception, opts)
+        cause = exception.respond_to?(:cause) ? exception.cause : exception
+        super || (cause.respond_to?(:getSQLState) && cause.getSQLState =~ /^08/)
+      end
+
       # Execute the prepared statement.  If the provided name is a
       # dataset, use that as the prepared statement, otherwise use
       # it as a key to look it up in the prepared_statements hash.
@@ -400,12 +406,6 @@ module Sequel
         end
       end
 
-      # Treat SQLExceptions with a "Connection Error" SQLState as disconnects
-      def raise_error(exception, opts={})
-        cause = exception.respond_to?(:cause) ? exception.cause : exception
-        super(exception, {:disconnect => cause.respond_to?(:getSQLState) && cause.getSQLState =~ /^08/}.merge(opts))
-      end
-
       # Java being java, you need to specify the type of each argument
       # for the prepared statement, and bind it individually.  This
       # guesses which JDBC method to use, and hopefully JRuby will convert

data/lib/sequel/adapters/mysql.rb

@@ -139,7 +139,7 @@ module Sequel
           opts[:user],
           opts[:password],
           opts[:database],
-          opts[:port],
+          (opts[:port].to_i if opts[:port]),
           opts[:socket],
           Mysql::CLIENT_MULTI_RESULTS +
           Mysql::CLIENT_MULTI_STATEMENTS +
@@ -220,7 +220,7 @@ module Sequel
             end
           end
         rescue Mysql::Error => e
-          raise_error(e, :disconnect=>MYSQL_DATABASE_DISCONNECT_ERRORS.match(e.message))
+          raise_error(e)
         ensure
           r.free if r
           # Use up all results to avoid a commands out of sync message.
@@ -248,6 +248,12 @@ module Sequel
       def database_error_classes
         [Mysql::Error]
       end
+
+      # Raise a disconnect error if the exception message matches the list
+      # of recognized exceptions.
+      def disconnect_error?(e, opts)
+        super || (e.is_a?(::Mysql::Error) && MYSQL_DATABASE_DISCONNECT_ERRORS.match(e.message))
+      end
       
       # The database name when using the native adapter is always stored in
       # the :database option.

data/lib/sequel/adapters/mysql2.rb

@@ -94,7 +94,7 @@ module Sequel
             yield conn
           end
         rescue ::Mysql2::Error => e
-          raise_error(e, :disconnect=>MYSQL_DATABASE_DISCONNECT_ERRORS.match(e.message))
+          raise_error(e)
         end
       end
 
@@ -108,6 +108,10 @@ module Sequel
         [::Mysql2::Error]
       end
 
+      def disconnect_error?(e, opts)
+        super || (e.is_a?(::Mysql2::Error) && MYSQL_DATABASE_DISCONNECT_ERRORS.match(e.message))
+      end
+
       # The database name when using the native adapter is always stored in
       # the :database option.
       def database_name

data/lib/sequel/adapters/odbc.rb

@@ -52,7 +52,7 @@ module Sequel
             r = log_yield(sql){conn.run(sql)}
             yield(r) if block_given?
           rescue ::ODBC::Error, ArgumentError => e
-            raise_error(e, :disconnect=>DISCONNECT_ERRORS.match(e.message))
+            raise_error(e)
           ensure
             r.drop if r
           end
@@ -65,7 +65,7 @@ module Sequel
           begin
             log_yield(sql){conn.do(sql)}
           rescue ::ODBC::Error, ArgumentError => e
-            raise_error(e, :disconnect=>DISCONNECT_ERRORS.match(e.message))
+            raise_error(e)
           end
         end
       end
@@ -77,9 +77,17 @@ module Sequel
         :do
       end
 
+      def database_error_classes
+        [::ODBC::Error]
+      end
+
       def disconnect_connection(c)
         c.disconnect
       end
+
+      def disconnect_error?(e, opts)
+        super || (e.is_a?(::ODBC::Error) && DISCONNECT_ERRORS.match(e.message))
+      end
     end
     
     class Dataset < Sequel::Dataset

data/lib/sequel/adapters/oracle.rb

@@ -78,7 +78,7 @@ module Sequel
             yield(r) if block_given?
             r
           rescue OCIException => e
-            raise_error(e, :disconnect=>CONNECTION_ERROR_CODES.include?(e.code))
+            raise_error(e)
           end
         end
       end
@@ -100,6 +100,10 @@ module Sequel
       rescue OCIInvalidHandle
         nil
       end
+
+      def disconnect_error?(e, opts)
+        super || (e.is_a?(::OCIException) && CONNECTION_ERROR_CODES.include?(e.code))
+      end
       
       def remove_transaction(conn)
         conn.autocommit = true if conn

data/lib/sequel/adapters/postgres.rb

@@ -136,7 +136,10 @@ module Sequel
     # PGconn subclass for connection specific methods used with the
     # pg, postgres, or postgres-pr driver.
     class Adapter < ::PGconn
+      DISCONNECT_ERROR_RE = /\Acould not receive data from server: Software caused connection abort/
+      
       include Sequel::Postgres::AdapterMethods
+
       self.translate_results = false if respond_to?(:translate_results=)
       
       # Hash of prepared statements for this connection.  Keys are
@@ -155,20 +158,23 @@ module Sequel
         end
         @prepared_statements = {} if SEQUEL_POSTGRES_USES_PG
       end
-      
+
       # Raise a Sequel::DatabaseDisconnectError if a PGError is raised and
       # the connection status cannot be determined or it is not OK.
       def check_disconnect_errors
         begin
           yield
         rescue PGError =>e
+          disconnect = false
           begin
             s = status
           rescue PGError
-            raise Sequel.convert_exception_class(e, Sequel::DatabaseDisconnectError)
+            disconnect = true
           end
           status_ok = (s == Adapter::CONNECTION_OK)
-          status_ok ? raise : raise(Sequel.convert_exception_class(e, Sequel::DatabaseDisconnectError))
+          disconnect ||= !status_ok
+          disconnect ||= e.message =~ DISCONNECT_ERROR_RE
+          disconnect ? raise(Sequel.convert_exception_class(e, Sequel::DatabaseDisconnectError)) : raise
         ensure
           block if status_ok
         end
@@ -181,7 +187,7 @@ module Sequel
         begin
           block_given? ? yield(q) : q.cmd_tuples
         ensure
-          q.clear
+          q.clear if q
         end
       end