sequel 3.34.1 → 3.35.0

data/CHANGELOG

@@ -1,3 +1,55 @@
+=== 3.35.0 (2012-05-01)
+
+* Correctly handle parsing schema for tables in other databases on MySQL (jeremyevans)
+
+* Add DSL support for the modulus operator (%), similar to the bitwise operators (jeremyevans)
+
+* Fix possible thread-safety issues on non-GVL ruby implementations (jeremyevans)
+
+* Allow truncation of multiple tables at the same time on PostgreSQL (jeremyevans)
+
+* Allow truncate to take a :cascade, :only, and :restart options on PostgreSQL (hgimenez, jeremyevans)
+
+* Allow json and xml serializers to support :array option in class to_json method to serialize existing array of model instances (jeremyevans)
+
+* Add dirty plugin, which saves the initial value of the column when the value is changed (jeremyevans)
+
+* create_table now supports an :as option to create a table directly from the results of a query (jeremyevans)
+
+* The :index option when creating columns in the schema generator can now be a hash of options passed to index (jeremyevans)
+
+* Parsing the default column values in the oracle adapter no longer requires superuser privileges (Jason Hines)
+
+* Add Database#cache_schema to allow schema caching to be turned of, useful for development modes where models are reloaded (jeremyevans)
+
+* Correctly handle errors that occur when rolling back transactions (jeremyevans)
+
+* Recognize identity type in the schema dumper (jeremyevans) (#468)
+
+* Don't assign instance variables to Java objects, for future JRuby 2.0 support (jeremyevans) (#466)
+
+* Use date and timestamp formats that are multilanguage and not DATEFORMAT dependent on Microsoft SQL Server (jeremyevans)
+
+* Add Database#log_exception, which logs when a query raises an exception, for easier overriding (jeremyevans) (#465)
+
+* Make the migrators only use transactions by default if the database supports transactional DDL (jeremyevans)
+
+* Add Database#supports_transactional_ddl? for checking if DDL statements can be rolled back in transactions (jeremyevans)
+
+* Don't use auto parameterization when using cursors in the pg_auto_parameterize extension (jeremyevans) (#463)
+
+* No longer escape backslashes in strings by default, fixes doubled backslashes on some adapters (jeremyevans)
+
+* Escape blackslash-carriage return-line feed in strings on Microsoft SQL Server (mluu, jeremyevans) (#462, #461)
+
+* Remove Array#all_two_pairs? (jeremyevans)
+
+* Remove Dataset#disable_insert_returning on PostgreSQL (jeremyevans)
+
+* Remove support for PostgreSQL <8.2 (jeremyevans)
+
+* Remove support for Ruby <1.8.7 (jeremyevans)
+
 === 3.34.1 (2012-04-02)
 
 * Fix bug in optimization of primary key lookup (jeremyevans) (#460)

data/README.rdoc

@@ -64,12 +64,14 @@ If you have any comments or suggestions please post to the Google group.
 
 == The Sequel Console
 
-Sequel includes an IRB console for quick access to databases. You can use it like this:
+Sequel includes an IRB console for quick access to databases (usually referred to as <tt>bin/sequel</tt>). You can use it like this:
 
   sequel sqlite://test.db # test.db in current directory
 
 You get an IRB session with the database object stored in DB.
 
+In addition to providing an IRB shell (the default behavior), bin/sequel also has support for migrating databases (-m and -M), dumping schema migrations (-d and -D), and copying databases (-C).
+
 == An Introduction
 
 Sequel is designed to take the hassle away from connecting to databases and manipulating them. Sequel deals with all the boring stuff like maintaining connections, formatting SQL correctly and fetching records so you can concentrate on your application.

data/Rakefile

@@ -118,23 +118,15 @@ begin
     end
   end
 
-  sc = spec_with_cov = lambda do |name, files, d|
+  spec_with_cov = lambda do |name, files, d, &b|
     spec.call(name, files, d)
     t = spec.call("#{name}_cov", files, "#{d} with coverage")
     t.rcov = true
     t.rcov_opts = File.read("spec/rcov.opts").split("\n")
+    b.call(t) if b
     t
   end
 
-  if RUBY_VERSION >= '1.8.7'
-    eval <<-END
-      spec_with_cov = lambda do |*x, &b|
-        t = sc.call(*x)
-        b.call(t) if b
-      end
-    END
-  end
-  
   task :default => [:spec]
   spec_with_cov.call("spec", Dir["spec/{core,model}/*_spec.rb"], "Run core and model specs"){|t| t.rcov_opts.concat(%w'--exclude "lib/sequel/adapters/([a-ln-z]|m[a-np-z])"')}
   spec.call("spec_core", Dir["spec/core/*_spec.rb"], "Run core specs")

data/doc/active_record.rdoc

@@ -694,6 +694,7 @@ ActiveRecord Method :: Sequel Method
 +set_primary_key+ :: +set_primary_key+
 +sum+ :: +sum+
 +table_name+ :: +table_name+
++unescoped+ :: +unfiltered+
 
 === Class Methods without an Equivalent
 

data/doc/migration.rdoc

@@ -173,9 +173,10 @@ which just executes the code on the underlying database.
 
 == Errors when running migrations
 
-Sequel attempts to run migrations inside of a transaction.  Some databases do not support
-schema modifications made in transactions, and if the migration raises an error, it will
-not rollback the previous schema changes made by the migration.  In that case, you will
+Sequel attempts to run migrations inside of a transaction if the database supports
+transactional DDL statements.  On the databases that don't support transactional DDL
+statements, if there is an error while running a migration, it will not rollback the
+previous schema changes made by the migration.  In that case, you will
 need to update the database by hand.
 
 It's recommended to always run migrations on a test database and ensure they work
@@ -183,9 +184,18 @@ before running them on any production database.
 
 == Transactions
 
-As mentioned above, Sequel attempts to run migrations inside of a transaction by default.
-However, you can disable this on a per migration basis by calling +no_transaction+ inside
-of the Sequel.migration block:
+You can manually specify to use transactions on a per migration basis.  For example,
+if you want to force transaction use for a particular migration, call the transaction
+method in the Sequel.migration block:
+
+  Sequel.migration do
+    transaction
+    change do
+      # ...
+    end
+  end
+
+Likewise, you can disable transaction use via no_transaction:
 
   Sequel.migration do
     no_transaction
@@ -195,7 +205,8 @@ of the Sequel.migration block:
   end
 
 This is necessary in some cases, such as when attempting to use CREATE INDEX CONCURRENTLY
-on PostgreSQL.
+on PostgreSQL (which supports transactional schema, but not that statement inside a
+transaction).
 
 You can also override the transactions setting at the migrator level, either by forcing
 transactions even if no_transaction is set, or by disabling transactions all together:

data/doc/model_hooks.rdoc

@@ -69,6 +69,12 @@ Note that these hooks are only called when using <tt>Model#destroy</tt>, they ar
 
 <tt>Sequel::Model</tt> does support a few additional hooks. One is +after_intialize+, which is called after the model object has been initalized.  It can be used to set default attribute values for new objects, since by default new <tt>Sequel::Model</tt> objects have no attributes, and the attributes are not filled in until the model object is saved.  You should be careful when you are using +after_initialize+, since it is called for every created record.  So if you run a query that returns 1000 model objects, it will be called 1000 times.  If you only want to change the behavior for new records, you can override the +initialize_set+ private method, which is called with the hash passed to +initialize+.
 
+== Special Hook-Related Instance Variables
+
+For after_save hooks, a @was_new instance variable is present that indicates whether the record was a new record that was just inserted, or an existing record that was updated.  Sequel marks a record as existing as soon as it inserts the record, so in an after_save or after_create hook, the instance is no longer considered new.  You have to check @was_new to see if the record was inserted.  This exists so that you don't have to have separate after_create and after_update hooks that are mostly the same and only differ slightly depending on whether the record was a new record.
+
+For after_update hooks, a @columns_updated instance variable is present that is a hash of the values used to update the row (keys are column symbols, values are column values).  This should be used by any code that wants to check what columns and values were used during the update.  You can't just check the current values of the instance, since Sequel offers ways to manually specify which columns to use during the save.
+
 == Transaction-related Hooks
 
 There are four other model hooks that <tt>Sequel::Model</tt> supports, all related to transactions.  These are +after_commit+, +after_rollback+, +after_destroy_commit+, and +after_destroy_rollback+.  +after_commit+ is called after the transaction in which you saved the object commits, only if it commits.  +after_rollback+ is called after the transaction in which you saved the object rolls back, if it rolls back.  +after_destroy_commit+ is called after the transaction in which you destroyed the object commits, if it commits.  +after_destroy_rollback+ is called after the transaction in which you destroyed the object rolls back, if it rolls back.

data/doc/opening_databases.rdoc

@@ -338,6 +338,9 @@ Requires: pg (or postgres if pg is not available)
 
 The Sequel postgres adapter works with the pg, postgres, and postgres-pr ruby libraries.
 The pg library is the best supported, as it supports real bound variables and prepared statements.
+If the pg library is being used, Sequel will also attempt to load the sequel_pg library, which is
+a C extension that optimizes performance when Sequel is used with pg.  All users of Sequel who
+use pg are encouraged to install sequel_pg.
 
 The following additional options are supported:
 

data/doc/prepared_statements.rdoc

@@ -82,7 +82,6 @@ and update queries, the hash to insert/update is passed to +prepare+:
 If you are using the ruby-postgres or postgres-pr driver, PostgreSQL uses the
 default emulated support.  If you are using ruby-pg, there is native support,
 but it may require type specifiers on some old versions (generally not anymore).
-direct control over the SQL string, but since Sequel abstracts that, the types
 You can add a __* suffix to the placeholder symbol to specify a type, which
 casts to that type in the SQL (e.g.  :$name__text, which will be compiled to
 "$1::text" in the SQL).  Prepared statements are always server side.

data/doc/release_notes/3.35.0.txt

@@ -0,0 +1,144 @@
+= New Features
+
+* A dirty plugin has been added, which saves the initial value of
+  the column when the column is changed, similar to
+  ActiveModel::Dirty:
+
+    artist.name                   # => 'Foo'
+    artist.name = 'Bar'
+    artist.initial_value(:name)   # 'Foo'
+    artist.column_change(:name)   # ['Foo', 'Bar']
+    artist.column_changes         # {:name => ['Foo', 'Bar']}
+    artist.column_changed?(:name) # true
+    artist.reset_column(:name)
+    artist.name                   # => 'Foo'
+    artist.column_changed?(:name) # false
+    artist.update(:name=>'Bar')
+    artist.column_changes         # => {}
+    artist.previous_changes       # => {:name=>['Foo', 'Bar']}
+
+* Database#create_table now respects an :as option to create a
+  database based on the results of a query.  The :as option value
+  should either be an SQL string or a dataset.
+
+    DB.create_table(:new_foos, :as=>DB[:foos].where(:new=>true))
+
+* The json_serializer and xml_serializer plugins can now serialize
+  arbitrary arrays of model objects by passing an :array option
+  to the to_json class method.  This works around an issue in
+  ruby's JSON library where Array#to_json does not pass arguments
+  given to it to the members of the array.
+
+    Artist.to_json(:array=>[Artist[1]], :include=>:albums)
+
+* You can now use the % (modulus) operator in the same way you
+  can use the bitwise operators in Sequel:
+
+    :column.sql_number % 1 # (column % 1)
+
+* On PostgreSQL, you can now provide :only, :cascade, and :restart
+  options to Dataset#truncate to use ONLY, CASCADE, and
+  RESTART IDENTITY.  Additionally, you can now truncate multiple
+  tables at the same time:
+
+    DB.from(:table1, :table2).truncate(:cascade=>true)
+
+* The :index option when creating columns in the schema generator
+  can now take a hash of index options:
+
+    DB.create_table(:foo){Integer :bar, :index=>{:unique=>true}}
+
+* A Database#cache_schema accessor has been added, it can be set
+  to false to have the Database never cache schema results.  This
+  can be useful in Rails development mode, so that you don't need to
+  restart a running server to have models pick up the new schema.
+
+* Database#log_exception has been added for easier instrumentation.
+  It is called with the exception and SQL query string for all
+  queries that raise an exception.
+
+* The Sequel.migration DSL now has a transaction method that forces
+  transaction use for the given migration.
+
+= Other Improvements
+
+* Many theoretical thread-safety issues have been fixed for ruby
+  implementations that don't use a global interpreter lock.
+  Previously, Sequel relied on MRI's global interpreter lock for
+  part of its thread safety, now it does manually locking in more
+  places to avoid thread-safety issues on JRuby (and other ruby
+  implementations without a global interpreter lock).
+
+  No Sequel user ever reported a production error related to the
+  previous thread-safety issues, and most of the issues fixed
+  were so difficult to hit that even tests specifically designed
+  to raise errors were unable to do so.
+
+* Sequel.single_threaded = true now disables the mutex
+  synchronization that enforces thread safety for additional
+  performance in single threaded mode.
+
+* Sequel's migrators now only attempt to use transactions by
+  default if the underlying database supports transactional DDL.
+  SQLite does support transactional DDL, but Sequel will not
+  use transactions for SQLite migrations as it causes issues
+  when emulating alter_table operations for tables with foreign
+  keys.
+
+* Errors that occur when rolling back database transactions are
+  now handled correctly.  Previously, the underlying exception was
+  raised, it wasn't correctly wrapped in a Sequel::DatabaseError,
+  and if it was due to a database disconnection, the connection
+  wasn't removed from the pool.
+
+* Sequel no longer sets ruby instance variables on java objects,
+  fixing warnings on JRuby 1.7 and attempting to be forward
+  compatible with JRuby 2.0.
+
+* Sequel now uses date and timestamp formats that are multilanguage
+  and not DATEFORMAT dependent on Microsoft SQL Server.
+
+* Sequel now correctly escapes blackslash-carriage return-line feed
+  on Microsoft SQL Server.
+
+* Parsing the column default values in the oracle adapter no longer
+  requires database superuser privileges.
+
+* Sequel now correctly handles parsing schema for tables in other
+  databases on MySQL.  Previously, it would always look in the
+  current database.
+
+* Sequel no longer doubles backslashes in strings by default.  It
+  now only does so on MySQL, since that is the only database that
+  appears to use backslashes for escaping.  This fixes issues with
+  backslashes being doubled on some of the less commonly used
+  adapters.
+
+* The pg_auto_parameterize extension now works correctly when
+  using cursors.
+
+* Dataset#truncate now raises an Error if you attempt to do so
+  on a dataset that uses HAVING.  Previously, it only checked for
+  WHERE.
+
+* The schema dumper now recognized the identity type.
+
+= Backwards Compatibility
+
+* Association reflections now store cached information in a
+  separate subhash due to the thread-safety changes.  Any code
+  accessing an association reflection should always call the
+  related method to get the cached data instead of checking
+  for a specific location in the hash.
+
+* Association reflection internals for many_through_many associations
+  changed significantly, any code that accesses the edge information
+  in the reflection will need to be changed to use the new methods
+  instead of accessing the old values directly.
+
+* The features deprecated in 3.34.0 have now been removed:
+  * Ruby <1.8.7 support
+  * PostgreSQL <8.2 support
+  * Dataset#disable_insert_returning on PostgreSQL
+  * Array#all_two_pairs? and #sql_expr_if_all_two_pairs
+  

data/doc/schema_modification.rdoc

@@ -80,7 +80,8 @@ When using the type name as method, the third argument is an options hash, and w
 method, the fourth argument is the options hash.  The following options are supported:
 
 :default :: The default value for the column.
-:index :: Create an index on this column.
+:index :: Create an index on this column. If given a hash, use the hash as the
+          options for the index.
 :null :: Mark the column as allowing NULL values (if true),
          or not allowing NULL values (if false).  If unspecified, will default
          to whatever the database default is.
@@ -175,6 +176,13 @@ column, but create a foreign key constraint:
     index :id
   end
 
+  create_table(:a){Integer :id, :index=>{:unique=>true}}
+  # Same as:
+  create_table(:a) do
+    Integer :id
+    index :id, :unique=>true
+  end
+
 Similar to the +primary_key+ and +foreign_key+ methods, calling +index+ with an array of symbols
 will create a multiple column index:
 
@@ -269,6 +277,13 @@ which expands to:
     index [:artist_id, :album_id]
   end
 
+== <tt>create_table :as=></tt>
+
+To create a table from the result of a SELECT query, instead of passing a block
+to +create_table+, provide a dataset to the :as option:
+
+  create_table(:older_items, :as=>DB[:items].where{updated_at < Date.today << 6})
+
 == +alter_table+
 
 +alter_table+ is used to alter existing tables, changing their columns, indexes,

data/doc/thread_safety.rdoc

@@ -0,0 +1,17 @@
+= Thread Safety
+
+Most Sequel usage (and all common Sequel usage) is thread safe by default.  Specifically, multiple threads can operate on Database instances, Dataset instances, and Model classes concurrently without problems.  In general, Database instance and Model classes are not modified after application startup, and modifying Dataset instances returns modified copies of the dataset instead of mutating it.
+
+== Connection Pool
+
+In order to allow multiple threads to operate on the same database at the same time, Sequel uses a connection pool.  The connection pool is designed so that a thread uses a connection for the minimum amount of time, returning the connection to the pool as soon as it is done using the connection.  If a thread requests a connection and the pool is empty, the thread will block (actually busy-wait) until a connection is available or the the connection pool timeout has elapsed (in which case a PoolTimeout error will be raised).
+
+== Exceptions
+
+This is a small list of things that are specifically non thread-safe.  This is not an exhaustive list, there may be cases not mentioned here.
+
+1) Model instances: Model instances are not thread-safe unless they are frozen first.  Multiple threads should not operate on an unfrozen model instance concurrently.
+
+2) Model class modifications: Model class modifications, such as adding associations and loading plugins, are not designed to be thread safe.  You should not modify a class in one thread if any other thread can concurrently access it.  Model subclassing is designed to be thread-safe, so you create a model subclass in a thread and modify it safely.
+
+3) Dataset mutation methods: Dataset mutation methods are not thread safe, you should not call them on datasets that could be accessed by other threads.  It is safe to clone the dataset first inside a thread and call mutation methods on the cloned dataset.

data/lib/sequel/adapters/do.rb

@@ -17,7 +17,7 @@ module Sequel
         Sequel.tsk_require 'do_postgres'
         Sequel.ts_require 'adapters/do/postgres'
         db.extend(Sequel::DataObjects::Postgres::DatabaseMethods)
-        db.dataset_class = Sequel::DataObjects::Postgres::Dataset
+        db.extend_datasets Sequel::Postgres::DatasetMethods
       end,
       :mysql=>proc do |db|
         Sequel.tsk_require 'do_mysql'
@@ -29,7 +29,7 @@ module Sequel
         Sequel.tsk_require 'do_sqlite3'
         Sequel.ts_require 'adapters/do/sqlite'
         db.extend(Sequel::DataObjects::SQLite::DatabaseMethods)
-        db.dataset_class = Sequel::DataObjects::SQLite::Dataset
+        db.extend_datasets Sequel::SQLite::DatasetMethods
         db.set_integer_booleans
       end
     }

data/lib/sequel/adapters/do/postgres.rb

@@ -7,40 +7,6 @@ module Sequel
     # Adapter, Database, and Dataset support for accessing a PostgreSQL
     # database via DataObjects.
     module Postgres
-      # Methods to add to the DataObjects adapter/connection to allow it to work
-      # with the shared PostgreSQL code.
-      module AdapterMethods
-        include Sequel::Postgres::AdapterMethods
-        
-        # Give the DataObjects adapter a direct execute method, which creates
-        # a statement with the given sql and executes it.
-        def execute(sql, args=nil)
-          command = create_command(sql)
-          begin
-            if block_given?
-              begin
-                yield(reader = @db.log_yield(sql){command.execute_reader})
-              ensure
-                reader.close if reader
-              end
-            else
-              @db.log_yield(sql){command.execute_non_query}
-            end
-          rescue ::DataObjects::Error => e
-            @db.send(:raise_error, e)
-          end
-        end
-        
-        private
-        
-        # DataObjects specific method of getting specific values from a result set.
-        def single_value(reader)
-          while(reader.next!) do
-            return reader.values.at(0)
-          end
-        end
-      end
-    
       # Methods to add to Database instances that access PostgreSQL via
       # DataObjects.
       module DatabaseMethods
@@ -55,32 +21,15 @@ module Sequel
           end
         end
         
-        # Run the INSERT sql on the database and return the primary key
-        # for the record.
-        def execute_insert(sql, opts={})
-          synchronize(opts[:server]) do |conn|
-            com = conn.create_command(sql)
-            log_yield(sql){com.execute_non_query}
-            insert_result(conn, opts[:table], opts[:values])
-          end
-        end
-        
         private
         
         # Extend the adapter with the DataObjects PostgreSQL AdapterMethods
         def setup_connection(conn)
           conn = super(conn)
-          conn.extend(Sequel::DataObjects::Postgres::AdapterMethods)
-          conn.db = self
-          conn.apply_connection_settings
+          connection_configuration_sqls.each{|sql| log_yield(sql){conn.create_command(sql).execute_non_query}}
           conn
         end
       end
-      
-      # Dataset subclass used for datasets that connect to PostgreSQL via DataObjects.
-      class Dataset < DataObjects::Dataset
-        include Sequel::Postgres::DatasetMethods
-      end
     end
   end
 end