sequel 3.21.0 → 3.31.0

data/doc/release_notes/3.31.0.txt

@@ -0,0 +1,146 @@
+= New Features
+
+* The serialization plugin can now support custom serialization
+  formats, by supplying a serializer/deserializer pair of
+  callable objects.  You can also register custom deserializers
+  via Sequel::Plugins::Serialization.register_format, so that
+  they can be referenced by name.  Example:
+  
+    Sequel::Plugins::Serialization.register_format(:reverse,
+      lambda{|v| v.reverse},
+      lambda{|v| v.reverse})
+    class User < Sequel::Model
+      serialize_attributes :reverse, :password
+    end
+
+* Dataset#import and #multi_insert now support a
+  :return=>:primary_key option.  When this option is used, the
+  methods return an array of primary key values, one for each
+  inserted row.  Usage of this option on MySQL requires that a
+  separate query be issued per row (instead of the single
+  query for all rows that MySQL would usually use).
+  
+* PostgreSQL can now use Dataset#returning in conjunction with
+  import/multi_insert to set a custom column to return.
+  
+* Microsoft SQL Server can now use Dataset#output in conjection with
+  import/multi_insert to set a custom column to return.
+  
+* Dataset#import and #multi_insert now respect a :server option to
+  set the server/shard on which to execute the queries.
+  Additionally, options given to this method are also passed to
+  Dataset#transaction.
+
+* Dataset#insert_multiple now returns an array of inserted primary
+  keys.
+
+* Model.def_column_alias has been added to make it easy to create
+  alias methods for columns.  This is useful if you have a legacy
+  database and want to create friendly method names for the
+  underlying columns.  Note that this alias only affects the setter
+  and getter methods.  This does not affect the dataset level, so you
+  still need to use the actual column names in dataset filters.
+
+* many_to_one associations can now have the same name as the related
+  foreign key column, using the :key_column option.  Use of this
+  feature is not recommended, as it is much better to either rename
+  the column or rename the association.  Here's an example of usage:
+  
+    # Example schema:
+    #  albums           artists
+    #   :id        /-->  :id
+    #   :artist --/      :name
+    #   :name
+    class Album < Sequel::Model
+      def_column_alias(:artist_id, :artist)
+      many_to_one :artist, :key_column=>:artist
+    end
+
+* The mock adapter can now mock out database types, by providing a
+  shared adapter name as the host (e.g. mock://postgres).  This
+  emulation is not perfect, but in most cases it allows you to see
+  what SQL Sequel would generate on a given database without needing
+  to install the required database driver.
+
+* Sequel now supports creating full text indexes on Microsoft SQL
+  Server.  Before using it, you must have previously setup a default
+  full text search catalog, and you need to provide a :key_index
+  option with an index name symbol.
+  
+* Dataset#group_rollup and #group_cube methods have been added for
+  GROUP BY ROLLUP and GROUP BY CUBE support.  These features are in
+  a recent SQL standard, and they are supported to various degrees on
+  Microsoft SQL Server, DB2, Oracle, MySQL, and Derby.
+
+* Dataset#full_text_search on Microsoft SQL Server now supports
+  multiple search terms.
+  
+* The jdbc adapter now supports a :login_timeout option, giving the
+  timeout in seconds.
+
+= Other Improvements
+
+* Dataset#exists can now be used with prepared statement
+  placeholders.
+
+* Dataset#full_text_search can now be used with prepared statement
+  placeholders on PostgreSQL, MySQL, and Microsoft SQL Server.
+  
+* If tables from two separate schema are detected when parsing the
+  schema for a table on PostgreSQL, an error is now raised.
+  Previously, no error was raised, which led to weird errors later,
+  such as duplicate columns in a model's primary_key.
+
+* RETURNING is now supported with UPDATE/DELETE on PostgreSQL 8.2+.
+  Previously, Sequel only supported it on 9.1+, but PostgreSQL
+  introduced support for it in 8.2.
+
+* The shared postgres adapter now correctly handles the return value
+  for Dataset#insert if you provide a separate column array and value
+  array on PostgreSQL < 8.2.
+  
+* Handle case in the PostgreSQL adapter where the server version
+  cannot be determined via a query.
+  
+* H2 clob types are now treated as string instead of as blob.
+  Treating clob as blob breaks on H2, as it doesn't automatically
+  hex-unescape the input for clobs as it does for blobs.
+  
+* Dataset#empty? now works correctly when the dataset has an offset
+  and offset support is being emulated.
+
+* The mock adapter no longer defaults to downcasing identifiers on
+  output.
+
+= Backwards Compatibility
+
+* Dataset#exists now returns a PlaceholderLiteralString instead of a
+  LiteralString, which could potentially break some code.  If you
+  would like a String returned, you can pass the returned object to
+  Dataset#literal:
+  
+    dataset.literal(dataset.exists)
+
+* Dataset#from no longer handles :a__b__c___d as "a.b.c AS d".  This
+  was not the intended behavior, and nowhere else in Sequel is a
+  symbol treated that way.  Now, Dataset#from is consistent with the
+  rest of Sequel, using "a.b__c AS d".  This should only affect
+  people in very rare cases, as most databases don't use three level
+  qualified tables.  One exception is Microsoft SQL Server, which can
+  use three level qualified tables for cross-database access.
+  
+* Previously, Dataset#insert_multiple returned an array of hashes, now
+  it returns an array of primary key values.
+
+* Dataset#EXRACT_CLOSE in the shared sqlite adapter has been renamed to
+  Dataset#EXTRACT_CLOSE.
+  
+* Dataset::StoredProcedureMethods::SQL_QUERY_TYPE and
+  Dataset::ArgumentMapper::SQL_QUERY_TYPE constants have been removed,
+  as have related sql_query_type private methods.
+
+* The serialization plugin was significantly refactored.
+  Model.serialization_map now contains a callable object instead of a
+  Symbol, and Model.serialization_format has been removed.
+  Model.define_serialized_attribute_accessors private method now takes
+  two callable objects before the columns, instead of a single symbol.

data/doc/schema_modification.rdoc

@@ -0,0 +1,547 @@
+= Schema modification methods
+
+Here's a brief description of the most common schema modification methods:
+
+== +create_table+
+
++create_table+ is the most common schema modification method, and it's used for adding new tables
+to the schema.  You provide it with the name of the table as a symbol, as well a block:
+
+  create_table(:artists) do
+    primary_key :id
+    String :name
+  end
+
+Not that if you want a primary key for the table, you need to specify it, Sequel does not create one
+by default.
+
+=== Column types
+
+Most method calls inside the create_table block will create columns, since +method_missing+ calls +column+.
+Columns are generally created by specifying the column type as the method
+name, followed by the column name symbol to use, and after that any options that should be used.
+If the method is a ruby class name that Sequel recognizes, Sequel will transform it into the appropriate
+type for the given database.  So while you specified +String+, Sequel will actually use +varchar+ or
++text+ depending on the underlying database.  Here's a list of all of ruby classes that Sequel will
+convert to database types:
+
+  create_table(:columns_types) do       # common database type used
+    Integer :a0                         # integer
+    String :a1                          # varchar(255)
+    String :a2, :size=>50               # varchar(50)
+    String :a3, :fixed=>true            # char(255)
+    String :a4, :fixed=>true, :size=>50 # char(50)
+    String :a5, :text=>true             # text
+    File :b,                            # blob
+    Fixnum :c                           # integer
+    Bignum :d                           # bigint
+    Float :e                            # double precision
+    BigDecimal :f                       # numeric
+    BigDecimal :f2, :size=>10           # numeric(10)
+    BigDecimal :f3, :size=>[10, 2]      # numeric(10, 2)
+    Date :g                             # date
+    DateTime :h                         # timestamp
+    Time :i                             # timestamp
+    Time :i2, :only_time=>true          # time
+    Numeric :j                          # numeric
+    TrueClass :k                        # boolean
+    FalseClass :l                       # boolean
+  end
+
+Note that in addition to the ruby class name, Sequel also pays attention to the column options when
+determining which database type to use.  Also note that for boolean columns, you can use either
+TrueClass or FalseClass, they are treated the same way (ruby doesn't have a Boolean class).
+
+Also note that this conversion is only done if you use a supported ruby class name.  In all other
+cases, Sequel uses the type specified verbatim:
+
+  create_table(:columns_types) do  # database type used
+    string :a1                     # string
+    datetime :a2                   # datetime
+    blob :a3                       # blob
+    inet :a4                       # inet
+  end
+
+In addition to specifying the types as methods, you can use the +column+ method and specify the types
+as the second argument, either as ruby classes, symbols, or strings:
+
+  create_table(:columns_types) do  # database type used
+    column :a1, :string            # string
+    column :a2, String             # varchar(255)
+    column :a3, 'string'           # string
+    column :a4, :datetime          # datetime
+    column :a5, DateTime           # timestamp
+    column :a6, 'timestamp(6)'     # timestamp(6)
+  end
+
+=== Column options
+
+When using the type name as method, the third argument is an options hash, and when using the +column+
+method, the fourth argument is the options hash.  The following options are supported:
+
+:default :: The default value for the column.
+:index :: Create an index on this column.
+:null :: Mark the column as allowing NULL values (if true),
+         or not allowing NULL values (if false).  If unspecified, will default
+         to whatever the database default is.
+:size :: The size of the column, generally used with string
+         columns to specify the maximum number of characters the column will hold.
+         An array of two integers can be provided to set the size and the
+         precision, respectively, of decimal columns.
+:unique :: Mark the column as unique, generally has the same effect as
+           creating a unique index on the column.
+:unsigned :: Make the column type unsigned, only useful for integer
+             columns.
+
+=== Other methods
+
+In addition to the +column+ method and other methods that create columns, there are a other methods that can be used:
+
+==== +primary_key+
+
+You've seen this one used already.  It's used to create an autoincrementing integer primary key column.
+
+  create_table(:a0){primary_key :id}
+
+If you want to create a primary key column that doesn't use an autoincrementing integer, you should
+not use this method.  Instead, you should use the :primary_key option to the +column+ method or type
+method:
+
+  create_table(:a1){Integer :id, :primary_key=>true} # Non autoincrementing integer primary key
+  create_table(:a2){String :name, :primary_key=>true} # varchar(255) primary key
+
+If you want to create a composite primary key, you should call the +primary_key+ method with an
+array of column symbols:
+
+  create_table(:items) do
+    Integer :group_id
+    Integer :position
+    primary_key [:group_id, :position]
+  end 
+
+If provided with an array, +primary_key+ does not create a column, it just sets up the primary key constraint.
+
+==== +foreign_key+
+
++foreign_key+ is used to create a foreign key column that references a column in another table (or the same table).
+It takes the column name as the first argument, the table it references as the second argument, and an options hash
+as it's third argument.  A simple example is:
+
+  create_table(:albums) do
+    primary_key :id
+    foreign_key :artist_id, :artists
+    String :name
+  end
+
++foreign_key+ accepts some specific options:
+
+:deferrable :: Makes the foreign key constraint checks deferrable, so they aren't checked
+               until the end of the transaction.
+:key :: For foreign key columns, the column in the associated table
+        that this column references.  Unnecessary if this column
+        references the primary key of the associated table, at least
+        on most databases.
+:on_delete :: Specify the behavior of this foreign key column when the row with the primary key
+              it references is deleted , can be :restrict, :cascade, :set_null, or :set_default.
+:on_update :: Specify the behavior of this foreign key column when the row with the primary key
+              it references modifies the value of the primary key, can be
+              :restrict, :cascade, :set_null, or :set_default.
+
+Like +primary_key+, if you provide +foreign_key+ with an array of symbols, it will not create a
+column, but create a foreign key constraint:
+
+  create_table(:artists) do
+    String :name
+    String :location
+    primary_key [:name, :location]
+  end
+  create_table(:albums) do
+    String :artist_name
+    String :artist_location
+    String :name
+    foreign_key [:artist_name, :artist_location], :artists
+  end
+
+==== +index+
+
++index+ creates indexes on the table.  For single columns, calling index is the same as using the
+<tt>:index</tt> option when creating the column:
+
+  create_table(:a){Integer :id, :index=>true}
+  # Same as:
+  create_table(:a) do
+    Integer :id
+    index :id
+  end
+
+Similar to the +primary_key+ and +foreign_key+ methods, calling +index+ with an array of symbols
+will create a multiple column index:
+
+  create_table(:albums) do
+    primary_key :id
+    foreign_key :artist_id, :artists
+    Integer :position
+    index [:artist_id, :position]
+  end
+
+The +index+ method also accepts some options:
+
+:name :: The name of the index (generated based on the table and column names if not provided).
+:type :: The type of index to use (only supported by some databases)
+:unique :: Make the index unique, so duplicate values are not allowed.
+:where :: Create a partial index (only supported by some databases)
+
+==== +unique+
+
+The +unique+ method creates a unique constraint on the table.  A unique constraint generally
+operates identically to a unique index, so the following three +create_table+ blocks are
+pretty much identical:
+
+  create_table(:a){Integer :a, :unique=>true}
+
+  create_table(:a) do
+    Integer :a
+    index :a, :unique=>true
+  end
+
+  create_table(:a) do
+    Integer :a
+    unique :a
+  end
+
+Just like +index+, +unique+ can set up a multiple column unique constraint, where the
+combination of the columns must be unique:
+
+  create_table(:a) do
+    Integer :a
+    Integer :b
+    unique [:a, :b]
+  end
+
+==== +full_text_index+ and +spatial_index+
+
+Both of these create specialized index types supported by some databases.  They
+both take the same options as +index+.
+
+==== +constraint+
+
++constraint+ creates a named table constraint:
+
+  create_table(:artists) do
+    primary_key :id
+    String :name
+    constraint(:name_min_length){char_length(name) > 2}
+  end
+  
+Instead of using a block, you can use arguments that will be handled similarly
+to <tt>Dataset#filter</tt>:
+
+  create_table(:artists) do
+    primary_key :id
+    String :name
+    constraint(:name_length_range, :char_length.sql_function(:name)=>3..50)
+  end
+  
+==== +check+
+
++check+ operates just like +constraint+, except that it doesn't take a name
+and it creates an unnamed constraint
+
+  create_table(:artists) do
+    primary_key :id
+    String :name
+    check{char_length(name) > 2}
+  end
+  
+== +alter_table+
+
++alter_table+ is used to alter existing tables, changing their columns, indexes,
+or constraints.  It it used just like +create_table+, accepting a block which
+is instance_evaled, and providing its own methods:
+
+=== +add_column+
+
+One of the most common methods, +add_column+ is used to add a column to the table.
+Its API is similar to that of +create_table+'s +column+ method, where the first
+argument is the column name, the second is the type, and the third is an options
+hash:
+
+  alter_table(:albums) do
+    add_column :copies_sold, Integer, :default=>0
+  end
+
+When adding a column, it's a good idea to provide a default value, unless you
+want the value for all rows to be set to NULL.
+
+=== +drop_column+
+
+As you may expect, +drop_column+ takes a column name and drops the column.  It's
+often used in the +down+ block of a migration to drop a column added in an +up+ block:
+
+  alter_table(:albums) do
+    drop_column :copies_sold
+  end
+
+=== +rename_column+
+
++rename_column+ is used to rename a column.  It takes the old column name as the first
+argument, and the new column name as the second argument:
+
+  alter_table(:albums) do
+    rename_column :copies_sold, :total_sales
+  end
+
+=== +add_primary_key+
+
+If you forgot to include a primary key on the table, and want to add one later, you
+can use +add_primary_key+.  A common use of this is to make many_to_many association
+join tables into real models:
+
+  alter_table(:albums_artists) do
+    add_primary_key :id
+  end
+
+Just like +create_table+'s +primary_key+ method, if you provide an array of symbols,
+Sequel will not add a column, but will add a composite primary key constraint:
+
+  alter_table(:albums_artists) do
+    add_primary_key [:album_id, :artist_id]
+  end
+
+If you just want to take an existing single column and make it a primary key, call
++add_primary_key+ with an array with a single symbol:
+
+  alter_table(:artists) do
+    add_primary_key [:id]
+  end
+
+=== +add_foreign_key+
+
++add_foreign_key+ can be used to add a new foreign key column or constraint to a table.
+Like +add_primary_key+, if you provide it with a symbol as the first argument, it
+creates a new column:
+
+  alter_table(:albums) do
+    add_foreign_key :artist_id, :artists
+  end
+
+If you want to add a new foreign key constraint to an existing column, you provide an
+array with a single element:
+
+  alter_table(:albums) do
+    add_foreign_key [:artist_id], :artists
+  end
+
+To set up a multiple column foreign key constraint, use an array with multiple column
+symbols:
+
+  alter_table(:albums) do
+    add_foreign_key [:artist_name, :artist_location], :artists
+  end
+
+=== +add_index+
+
++add_index+ works just like +create_table+'s +index+ method, creating a new index on
+the table:
+
+  alter_table(:albums) do
+    add_index :artist_id
+  end
+
+It accepts the same options as +create_table+'s +index+ method, and you can set up
+a multiple column index using an array:
+
+  alter_table(:albums_artists) do
+    add_index [:album_id, :artist_id], :unique=>true
+  end
+
+=== +drop_index+
+
+As you may expect, +drop_index+ drops an existing index:
+
+  alter_table(:albums) do
+    drop_index :artist_id
+  end
+
+Just like +drop_column+, it is often used in the +down+ block of a migration.
+
+To drop an index with a specific name, use the <tt>:name</tt> option:
+
+  alter_table(:albums) do
+    drop_index :artist_id, :name=>:artists_id_index
+  end
+
+=== +add_full_text_index+, +add_spatial_index+
+
+Corresponding to +create_table+'s +full_text_index+ and +spatial_index+ methods,
+these two methods create new indexes on the table.
+
+=== +add_constraint+
+
+This adds a named constraint to the table, similar to +create_table+'s +constraint+
+method:
+
+  alter_table(:albums) do
+    add_constraint(:name_min_length){char_length(name) > 2}
+  end
+
+There is no method to add an unnamed constraint, but you can pass nil as the first
+argument of +add_constraint+ to do so.  However, it's not recommend to do that
+as it is difficult to drop such a constraint.
+
+=== +add_unique_constraint+
+
+This adds a unique constraint to the table, similar to +create_table+'s +unique+
+method.  This usually has the same effect as adding a unique index.
+
+  alter_table(:albums) do
+    add_unique_constraint [:artist_id, :name]
+  end
+
+=== +drop_constraint+
+
+This method drops an existing named constraint:
+
+  alter_table(:albums) do
+    drop_constraint(:name_min_length)
+  end
+
+There is no database independent method to drop an unnamed constraint.  Generally, the
+database will give it a name automatically, and you will have to figure out what it is.
+For that reason, you should not add unnamed constraints that you ever might need to remove.
+
+On MySQL, you must specify the type of constraint via a <tt>:type</tt> option:
+
+  alter_table(:albums) do
+    drop_constraint(:albums_pk, :type=>:primary_key)
+    drop_constraint(:albums_fk, :type=>:foreign_key)
+    drop_constraint(:albums_uk, :type=>:unique)
+  end
+
+=== +set_column_default+
+
+This modifies the default value of a column:
+
+  alter_table(:albums) do
+    set_column_default :copies_sold, 0
+  end
+
+=== +set_column_type+
+
+This modifies a column's type.  Most databases will attempt to convert existing values in
+the columns to the new type:
+
+  alter_table(:albums) do
+    set_column_type :copies_sold, Bignum
+  end
+
+You can specify the type as a string or symbol, in which case it is used verbatim, or as a supported
+ruby class, in which case it gets converted to an appropriate database type.
+
+=== +set_column_allow_null+
+
+This changes the NULL or NOT NULL setting of a column:
+
+  alter_table(:albums) do
+    set_column_allow_null :artist_id, true    # NULL
+    set_column_allow_null :copies_sold, false # NOT NULL
+  end
+
+== Other +Database+ schema modification methods
+
+<tt>Sequel::Database</tt> has many schema modification instance methods,
+most of which are shortcuts to the same methods in +alter_table+.  The
+following +Database+ instance methods just call +alter_table+ with a
+block that calls the method with the same name inside the +alter_table+
+block with all arguments after the first argument (which is used as
+the table name):
+
+* +add_column+
+* +drop_column+
+* +rename_column+
+* +add_index+
+* +drop_index+
+* +set_column_default+
+* +set_column_type+
+
+For example, the following two method calls do the same thing:
+
+  alter_table(:artists){add_column :copies_sold, Integer}
+  add_column :artists, :copies_sold, Integer
+
+There are some other schema modification methods that have no +alter_table+
+counterpart:
+
+=== +drop_table+
+
++drop_table+ takes multiple arguments and treats all arguments as a
+table name to drop:
+
+  drop_table(:albums_artists, :albums, :artists)
+
+Note that when dropping tables, you may need to drop them in a specific order
+if you are using foreign keys and the database is enforcing referential
+integrity.  In general, you need to drop the tables containing the foreign
+keys before the tables containing the primary keys they reference.
+
+=== +rename_table+
+
+You can rename an existing table using +rename_table+.  Like +rename_column+,
+the first argument is the current name, and the second is the new name:
+
+  rename_table(:artist, :artists)
+
+=== <tt>create_table!</tt>
+
+<tt>create_table!</tt> with the bang drops the table if it exists
+before attempting to create it, so:
+
+  create_table!(:artists)
+    primary_key :id
+  end
+
+is the same as:
+
+  drop_table(:artists) if table_exists?(:artists)
+  create_table(:artists)
+    primary_key :id
+  end
+
+It should not be used inside migrations, as if the table does not exist, it may
+mess up the migration.
+
+=== <tt>create_table?</tt>
+
+<tt>create_table?</tt> with a question mark only creates the table if it does
+not already exist, so:
+
+  create_table!(:artists)
+    primary_key :id
+  end
+
+is the same as:
+
+  create_table(:artists)
+    primary_key :id
+  end unless table_exists?(:artists)
+
+Like <tt>create_table!</tt>, it should not be used inside migrations.
+
+=== +create_view+ and +create_or_replace_view+
+
+These can be used to create views.  The difference between them is that
++create_or_replace_view+ will unconditionally replace an existing view of
+the same name, while +create_view+ will probably raise an error.  Both methods
+take the name as the first argument, and either an string or a dataset as the
+second argument:
+
+  create_view(:gold_albums, DB[:albums].filter{copies_sold > 500000})
+  create_or_replace_view(:gold_albums, "SELECT * FROM albums WHERE copies_sold > 500000")
+
+=== +drop_view+
+
++drop_view+ drops existing views.  Just like +drop_table+, it can accept multiple
+arguments:
+
+  drop_view(:gold_albums, :platinum_albums)
+

data/doc/sharding.rdoc

@@ -10,7 +10,13 @@ that ship with Sequel.
 
 Both features use the :servers Database option.  The :servers option should
 be a hash with symbol keys and values that are either hashes or procs that
-return hashes.  Note that all servers should have the same schema for all
+return hashes. These hashes are merged into the Database object's default
+options hash to get the connection options for the shard, so you don't need
+to override all options, just the ones that need to be modified.  For example,
+if you are using the same user, password, and database name and just the
+host is changing, you only need a :host entry in each shard's hash.
+
+Note that all servers should have the same schema for all
 tables you are accessing, unless you really know what you are doing.
 
 == Master and Slave Database Configurations