sequel 2.5.0 → 2.6.0

data/CHANGELOG

@@ -1,3 +1,51 @@
+=== 2.6.0 (2008-10-11)
+
+* Make the sqlite adapter respect the Sequel.datetime_class setting, for timestamp and datetime types (jeremyevans)
+
+* Enhance the CASE statement support to include an optional expression (jarredholman)
+
+* Default to using the simple language if no language is specified for a full text index on PostgreSQL (michalbugno)
+
+* Add Model.raise_on_typecast_failure=, which makes it possible to not raise errors on invalid typecasts (michalbugno)
+
+* Add schema.rdoc file, which provides an brief description of the various parts of Sequel related to schema modification (jeremyevans)
+
+* Fix constraint generation when not using a proc or interpolated string (jeremyevans)
+
+* Make eager_graph respect associations' :order options (use :order_eager_graph=>false to disable) (jeremyevans)
+
+* Cache negative lookup when eagerly loading many_to_one associations where no objects have an associated object (jeremyevans)
+
+* Allow string keys to be used when using Dataset#multi_insert (jeremyevans)
+
+* Fix join_table when doing the first join for a dataset where the first source is a dataset when using unqualified columns (jeremyevans)
+
+* Fix a few corner cases in eager_graph (jeremyevans)
+
+* Support transactions on MSSQL (jeremyevans)
+
+* Use string literals in AS clauses on SQLite (jeremyevans) (#241)
+
+* AlterTableGenerator#set_column_allow_null was added to SET/DROP NOT NULL for columns (divoxx)
+
+* Database#tables now works for MySQL databases using the JDBC adapter (jeremyevans)
+
+* Database#drop_view can now take multiple arguments to drop multiple views at once (jeremyevans)
+
+* Schema modification methods (e.g. drop_table, create_table!) now remove the cached schema entry (jeremyevans)
+
+* Models can now determine their primary keys by looking at the schema (jeremyevans)
+
+* No longer include :numeric_precision and :max_chars entries in the schema column hashes, use the :db_type entry instead (jeremyevans)
+
+* Make schema parsing on PostgreSQL handle implicit schemas (e.g. schema(:schema__table)), so it works with models for tables outside the public schema (jeremyevans)
+
+* Significantly speed up schema parsing on MySQL (jeremyevans)
+
+* Include primary key information when parsing the schema (jeremyevans)
+
+* Fix schema generation of composite foreign keys on MySQL (clivecrous, jeremyevans)
+
 === 2.5.0 (2008-09-03)
 
 * Add Dataset #set_defaults and #set_overrides, used for scoping the values used in insert/update statements (jeremyevans)

data/Rakefile

@@ -1,14 +1,18 @@
 require "rake"
 require "rake/clean"
 require "rake/gempackagetask"
-require "rake/rdoctask"
-require "fileutils"
 require "spec/rake/spectask"
+begin
+  require "hanna/rdoctask"
+rescue LoadError
+  require "rake/rdoctask"
+end
+require "fileutils"
 
 include FileUtils
 
 NAME = 'sequel'
-VERS = '2.5.0'
+VERS = '2.6.0'
 CLEAN.include ["**/.*.sw?", "pkg", ".config", "rdoc", "coverage", "www/public/*.html"]
 RDOC_OPTS = ["--quiet", "--line-numbers", "--inline-source", '--title', \
   'Sequel: The Database Toolkit for Ruby', '--main', 'README']
@@ -73,14 +77,20 @@ end
 
 ### Website
 
-desc "Update sequel.rubyforge.org"
-task :website => [:rdoc]
-task :website do
+desc "Update Non-RDoc section of sequel.rubyforge.org"
+task :website_base do
   sh %{www/make_www.rb}
   sh %{scp -r www/public/* rubyforge.org:/var/www/gforge-projects/sequel/}
+end
+
+desc "Update RDoc section of sequel.rubyforge.org"
+task :website_rdoc=>[:rerdoc] do
   sh %{scp -r rdoc/* rubyforge.org:/var/www/gforge-projects/sequel/rdoc/}
 end
 
+desc "Update sequel.rubyforge.org"
+task :website=>[:website_base, :website_rdoc]
+
 ### Specs
 
 lib_dir = File.join(File.dirname(__FILE__), 'lib')

data/doc/cheat_sheet.rdoc

@@ -5,10 +5,10 @@
   require 'rubygems'
   require 'sequel'
 
-  DB = Sequel.sqlite 'my_blog.db'
-  DB = Sequel('postgres://user:password@localhost/my_db')
-  DB = Sequel.mysql 'my_db', :user => 'user', :password => 'password', :host => 'localhost'
-  DB = Sequel.ado 'mydb'
+  DB = Sequel.sqlite('my_blog.db')
+  DB = Sequel.connect('postgres://user:password@localhost/my_db')
+  DB = Sequel.mysql('my_db', :user => 'user', :password => 'password', :host => 'localhost')
+  DB = Sequel.ado('mydb')
 
 == Open an SQLite memory database
 

data/doc/schema.rdoc

@@ -0,0 +1,9 @@
+= Schema Modifications with Sequel
+
+In addition to its support for getting data from databases, Sequel has good support for modifying the schema of databases.  Unfortunately, the documentation from this is spread out in various places in the Sequel RDoc.
+
+The recommended way to set up schema modifications in Sequel is through migrations. Migrations are modifications to the schema that handle modifying the schema in two directions, up and down, with down reversing the changes made by up. Migrations are stored in sequentially numbered files inside a directory (see the Sequel::Migrator documentation for file naming format details).  The migrations can be run on the database using the sequel command line tool (the -m and -M switches).
+
+The format of the individual migration files themselves is explained in the Sequel::Migration documentation.  Each migration file contains a single migration class.  The migration class acts a proxy for the related database (given on the command line if the sequel command line tool is used, or by the db argument to Sequel::Migration#apply if the API is used). The methods that can be used inside Sequel::Migration#up or Sequel::Migration#down are just Sequel::Database instance methods, such as create_table, drop_table, and alter_table.  Most database methods that alter the schema take regular arguments, but create_table and alter_table take a block.  The methods you can use inside the create_table block are documented in Sequel::Schema::Generator, and the methods you can use inside the alter_table block are documented in Sequel::Schema::AlterTableGenerator.
+
+Migrations are not required, you can just call the schema modification methods directly on the database object.  This is often done in test code and examples.  However, it is recommended that you use the migration framework unless the database schema will not be changing in the future, as it provides a way to easily handle modifications to existing database schema.

data/lib/sequel_core/adapters/jdbc.rb

@@ -154,18 +154,18 @@ module Sequel
           return yield(conn) if @transactions.include?(Thread.current)
           stmt = conn.createStatement
           begin
-            log_info(Sequel::Database::SQL_BEGIN)
-            stmt.execute(Sequel::Database::SQL_BEGIN)
+            log_info(begin_transaction_sql)
+            stmt.execute(begin_transaction_sql)
             @transactions << Thread.current
             yield(conn)
           rescue Exception => e
-            log_info(Sequel::Database::SQL_ROLLBACK)
-            stmt.execute(Sequel::Database::SQL_ROLLBACK)
-            raise e unless Error::Rollback === e
+            log_info(rollback_transaction_sql)
+            stmt.execute(rollback_transaction_sql)
+            transaction_error(e)
           ensure
             unless e
-              log_info(Sequel::Database::SQL_COMMIT)
-              stmt.execute(Sequel::Database::SQL_COMMIT)
+              log_info(commit_transaction_sql)
+              stmt.execute(commit_transaction_sql)
             end
             stmt.close
             @transactions.delete(Thread.current)

data/lib/sequel_core/adapters/mysql.rb

@@ -146,28 +146,23 @@ module Sequel
         @server_version ||= (synchronize(server){|conn| conn.server_version if conn.respond_to?(:server_version)} || super)
       end
       
-      # Return an array of symbols specifying table names in the current database.
-      def tables(server=nil)
-        synchronize(server){|conn| conn.list_tables.map {|t| t.to_sym}}
-      end
-      
       # Support single level transactions on MySQL.
       def transaction(server=nil)
         synchronize(server) do |conn|
           return yield(conn) if @transactions.include?(Thread.current)
-          log_info(SQL_BEGIN)
-          conn.query(SQL_BEGIN)
+          log_info(begin_transaction_sql)
+          conn.query(begin_transaction_sql)
           begin
             @transactions << Thread.current
             yield(conn)
           rescue ::Exception => e
-            log_info(SQL_ROLLBACK)
-            conn.query(SQL_ROLLBACK)
+            log_info(rollback_transaction_sql)
+            conn.query(rollback_transaction_sql)
             transaction_error(e, Mysql::Error)
           ensure
             unless e
-              log_info(SQL_COMMIT)
-              conn.query(SQL_COMMIT)
+              log_info(commit_transaction_sql)
+              conn.query(commit_transaction_sql)
             end
             @transactions.delete(Thread.current)
           end

data/lib/sequel_core/adapters/shared/mssql.rb

@@ -2,16 +2,36 @@ module Sequel
   module MSSQL
     module DatabaseMethods
       AUTO_INCREMENT = 'IDENTITY(1,1)'.freeze
+      SQL_BEGIN = "BEGIN TRANSACTION".freeze
+      SQL_COMMIT = "COMMIT TRANSACTION".freeze
+      SQL_ROLLBACK = "ROLLBACK TRANSACTION".freeze
       
       def auto_increment_sql
         AUTO_INCREMENT
       end
-      
+
       def dataset(opts = nil)
         ds = super
         ds.extend(DatasetMethods)
         ds
       end
+
+      private
+
+      # SQL to BEGIN a transaction.
+      def begin_transaction_sql
+        SQL_BEGIN
+      end
+
+      # SQL to COMMIT a transaction.
+      def commit_transaction_sql
+        SQL_COMMIT
+      end
+      
+      # SQL to ROLLBACK a transaction.
+      def rollback_transaction_sql
+        SQL_ROLLBACK
+      end
     end
   
     module DatasetMethods

data/lib/sequel_core/adapters/shared/mysql.rb

@@ -7,9 +7,6 @@ module Sequel
       NOT_NULL = Sequel::Schema::SQL::NOT_NULL
       NULL = Sequel::Schema::SQL::NULL
       PRIMARY_KEY = Sequel::Schema::SQL::PRIMARY_KEY
-      SQL_BEGIN = Sequel::Database::SQL_BEGIN
-      SQL_COMMIT = Sequel::Database::SQL_COMMIT
-      SQL_ROLLBACK = Sequel::Database::SQL_ROLLBACK
       TYPES = Sequel::Schema::SQL::TYPES
       UNIQUE = Sequel::Schema::SQL::UNIQUE
       UNSIGNED = Sequel::Schema::SQL::UNSIGNED
@@ -38,7 +35,7 @@ module Sequel
       
       # Handle MySQL specific syntax for column references
       def column_references_sql(column)
-        ", FOREIGN KEY (#{quote_identifier(column[:name])})#{super(column)}"
+        "#{", FOREIGN KEY (#{quote_identifier(column[:name])})" unless column[:type] == :check}#{super(column)}"
       end
       
       # Handle MySQL specific index SQL syntax
@@ -62,6 +59,11 @@ module Sequel
         @server_version ||= (m[1].to_i * 10000) + (m[2].to_i * 100) + m[3].to_i
       end
       
+      # Return an array of symbols specifying table names in the current database.
+      def tables(server=nil)
+        self['SHOW TABLES'].server(server).map{|r| r.values.first.to_sym}
+      end
+      
       # Changes the database in use by issuing a USE statement.  I would be
       # very careful if I used this.
       def use(db_name)
@@ -73,27 +75,18 @@ module Sequel
       
       private
       
-      # Always quote identifiers for the schema parser dataset.
-      def schema_ds_dataset
-        ds = schema_utility_dataset.clone
-        ds.quote_identifiers = true
-        ds
-      end
-      
-      # Allow other database schema's to be queried using the :database
-      # option.  Allow all database's schema to be used by setting
-      # the :database option to nil.  If the database option is not specified,
-      # uses the currently connected database.
-      def schema_ds_filter(table_name, opts)
-        filt = super
-        # Restrict it to the given or current database, unless specifically requesting :database = nil
-        filt = SQL::BooleanExpression.new(:AND, filt, {:c__table_schema=>opts[:database] || database_name}) if opts[:database] || !opts.include?(:database)
-        filt
-      end
-
-      # MySQL doesn't support table catalogs, so just join on schema and table name.
-      def schema_ds_join(table_name, opts)
-        [:information_schema__columns, {:table_schema => :table_schema, :table_name => :table_name}, :c]
+      # Use the MySQL specific DESCRIBE syntax to get a table description.
+      def schema_parse_table(table_name, opts)
+        self["DESCRIBE ?", table_name].map do |row|
+          row.delete(:Extra)
+          row[:allow_null] = row.delete(:Null) == 'YES'
+          row[:default] = row.delete(:Default)
+          row[:primary_key] = row.delete(:Key) == 'PRI'
+          row[:default] = nil if row[:default].blank?
+          row[:db_type] = row.delete(:Type)
+          row[:type] = schema_column_type(row[:db_type])
+          [row.delete(:Field).to_sym, row]
+        end
       end
     end
   
@@ -193,9 +186,8 @@ module Sequel
       
       # MySQL specific syntax for inserting multiple values at once.
       def multi_insert_sql(columns, values)
-        columns = column_list(columns)
         values = values.map {|r| literal(Array(r))}.join(COMMA_SEPARATOR)
-        ["INSERT INTO #{source_list(@opts[:from])} (#{columns}) VALUES #{values}"]
+        ["INSERT INTO #{source_list(@opts[:from])} (#{identifier_list(columns)}) VALUES #{values}"]
       end
       
       # MySQL uses the nonstandard ` (backtick) for quoting identifiers.

data/lib/sequel_core/adapters/shared/postgres.rb

@@ -128,7 +128,7 @@ module Sequel
         filter = " WHERE #{filter_expr(filter)}" if filter
         case index_type
         when :full_text
-          lang = index[:language] ? "#{literal(index[:language])}, " : ""
+          lang = "#{literal(index[:language] || 'simple')}, "
           cols = index[:columns].map {|c| literal(c)}.join(" || ")
           expr = "(to_tsvector(#{lang}#{cols}))"
           index_type = :gin
@@ -150,6 +150,22 @@ module Sequel
         synchronize(server){|conn| primary_key_for_table(conn, table)}
       end
 
+      # Support :schema__table format for table
+      def schema(table_name=nil, opts={})
+        case table_name
+        when Symbol
+          t, c, a = dataset.send(:split_symbol, table_name)
+          opts[:schema] ||= t
+          table_name = c
+        when SQL::QualifiedIdentifier
+          opts[:schema] ||= table_name.table
+          table_name = table_name.column
+        when SQL::Identifier
+          table_name = table_name.value
+        end
+        super(table_name, opts)
+      end
+
       # PostgreSQL uses SERIAL psuedo-type instead of AUTOINCREMENT for
       # managing incrementing primary keys.
       def serial_primary_key_options
@@ -267,14 +283,55 @@ module Sequel
         @primary_key_sequences.include?(table) ? @primary_key_sequences[table] : (@primary_key_sequences[table] = conn.sequence(table))
       end
       
-      # When the :schema option is used, use the the given schema.
-      # When the :schema option is nil, return results for all schemas.
-      # If the :schema option is not used, use the public schema.
-      def schema_ds_filter(table_name, opts)
-        filt = super
-        # Restrict it to the given or public schema, unless specifically requesting :schema = nil
-        filt = SQL::BooleanExpression.new(:AND, filt, {:c__table_schema=>opts[:schema] || 'public'}) if opts[:schema] || !opts.include?(:schema)
-        filt
+      # Set the default of the row to NULL if it is blank, and set
+      # the ruby type for the column based on the database type.
+      def schema_parse_rows(rows)
+        rows.map do |row|
+          row[:default] = nil if row[:default].blank?
+          row[:type] = schema_column_type(row[:db_type])
+          [row.delete(:name).to_sym, row]
+        end
+      end
+
+      # Parse the schema for a single table.
+      def schema_parse_table(table_name, opts)
+        schema_parse_rows(schema_parser_dataset(table_name, opts))
+      end
+
+      # Parse the schema for multiple tables.
+      def schema_parse_tables(opts)
+        schemas = {}
+        schema_parser_dataset(nil, opts).each do |row|
+          (schemas[row.delete(:table).to_sym] ||= []) << row
+        end
+        schemas.each do |table, rows|
+          schemas[table] = schema_parse_rows(rows)
+        end
+        schemas
+      end
+
+      # The dataset used for parsing table schemas, using the pg_* system catalogs.
+      def schema_parser_dataset(table_name, opts)
+        ds = dataset.select(:pg_attribute__attname___name,
+            SQL::Function.new(:format_type, :pg_type__oid, :pg_attribute__atttypmod).as(:db_type),
+            SQL::Function.new(:pg_get_expr, :pg_attrdef__adbin, :pg_class__oid).as(:default),
+            SQL::BooleanExpression.new(:NOT, :pg_attribute__attnotnull).as(:allow_null),
+            SQL::Function.new(:COALESCE, {:pg_attribute__attnum => SQL::Function.new(:ANY, :pg_index__indkey)}.sql_expr, false).as(:primary_key)).
+          from(:pg_class).
+          join(:pg_attribute, :attrelid=>:oid).
+          join(:pg_type, :oid=>:atttypid).
+          left_outer_join(:pg_attrdef, :adrelid=>:pg_class__oid, :adnum=>:pg_attribute__attnum).
+          left_outer_join(:pg_index, :indrelid=>:pg_class__oid, :indisprimary=>true).
+          filter(:pg_attribute__attisdropped=>false).
+          filter(:pg_attribute__attnum > 0).
+          order(:pg_attribute__attnum)
+        if table_name
+          ds.filter!(:pg_class__relname=>table_name.to_s)
+        else
+          ds.select_more!(:pg_class__relname___table)
+        end
+        ds.join!(:pg_namespace, :oid=>:pg_class__relnamespace, :nspname=>opts[:schema] || 'public') if opts[:schema] || !opts.include?(:schema)
+        ds
       end
     end
     
@@ -409,9 +466,8 @@ module Sequel
         return super if server_version < 80200
         
         # postgresql 8.2 introduces support for multi-row insert
-        columns = column_list(columns)
         values = values.map {|r| literal(Array(r))}.join(COMMA_SEPARATOR)
-        ["INSERT INTO #{source_list(@opts[:from])} (#{columns}) VALUES #{values}"]
+        ["INSERT INTO #{source_list(@opts[:from])} (#{identifier_list(columns)}) VALUES #{values}"]
       end
       
       # PostgreSQL assumes unquoted identifiers are lower case by default,

data/lib/sequel_core/adapters/shared/sqlite.rb

@@ -2,7 +2,6 @@ module Sequel
   module SQLite
     module DatabaseMethods
       AUTO_VACUUM = {'0' => :none, '1' => :full, '2' => :incremental}.freeze
-      SCHEMA_TYPE_RE = /\A(\w+)\((\d+)\)\z/
       SYNCHRONOUS = {'0' => :off, '1' => :normal, '2' => :full}.freeze
       TABLES_FILTER = "type = 'table' AND NOT name = 'sqlite_sequence'"
       TEMP_STORE = {'0' => :default, '1' => :file, '2' => :memory}.freeze
@@ -86,31 +85,16 @@ module Sequel
       # SQLite supports schema parsing using the table_info PRAGMA, so
       # parse the output of that into the format Sequel expects.
       def schema_parse_table(table_name, opts)
-        rows = self["PRAGMA table_info(?)", table_name].collect do |row|
+        self["PRAGMA table_info(?)", table_name].map do |row|
           row.delete(:cid)
-          row[:column] = row.delete(:name)
-          row[:allow_null] = row.delete(:notnull).to_i == 0 ? 'YES' : 'NO'
+          row[:allow_null] = row.delete(:notnull).to_i == 0
           row[:default] = row.delete(:dflt_value)
-          row[:primary_key] = row.delete(:pk).to_i == 1 ? true : false 
+          row[:primary_key] = row.delete(:pk).to_i == 1
+          row[:default] = nil if row[:default].blank?
           row[:db_type] = row.delete(:type)
-          if m = SCHEMA_TYPE_RE.match(row[:db_type])
-            row[:db_type] = m[1]
-            row[:max_chars] = m[2].to_i
-          else
-             row[:max_chars] = nil
-          end
-          row[:numeric_precision] = nil
-          row
+          row[:type] = schema_column_type(row[:db_type])
+          [row.delete(:name).to_sym, row]
         end
-        schema_parse_rows(rows)
-      end
-      
-      # SQLite doesn't support getting the schema of all tables at once,
-      # so loop through the output of #tables to get them.
-      def schema_parse_tables(opts)
-        schemas = {}
-        tables.each{|table| schemas[table] = schema_parse_table(table, opts)}
-        schemas
       end
     end
     
@@ -169,6 +153,11 @@ module Sequel
       
       private
       
+      # SQLite uses string literals instead of identifiers in AS clauses.
+      def as_sql(expression, aliaz)
+        "#{expression} AS #{literal(aliaz.to_s)}"
+      end
+
       # Call execute_insert on the database with the given SQL.
       def execute_insert(sql, opts={})
         @db.execute_insert(sql, {:server=>@opts[:server] || :default}.merge(opts))