sequel 3.32.0 → 3.33.0

data/CHANGELOG

@@ -1,3 +1,45 @@
+=== 3.33.0 (2012-03-01)
+
+* Add ability to force or disable transactions completely in the migrators using the :use_transactions option (jeremyevans)
+
+* Add ability to turn off transactions for migrations by calling no_transaction inside the Sequel.migration block (jeremyevans)
+
+* Allow specifically choosing which migrator to use via TimestampMigrator.apply or IntegerMigrator.apply (jeremyevans)
+
+* Add arbitrary_servers extension to allow the use of arbitrary servers/shards by providing a hash of options as the server (jeremyevans)
+
+* Add server_block extension to scope database access inside the block to a specific default server/shard (jeremyevans)
+
+* Respect :collate column option on MySQL (jeremyevans) (#445)
+
+* Use Mysql2::Client::FOUND_ROWS to get accurate number of rows matched in the mysql2 adapter (jeremyevans)
+
+* Use Mysql#info to get accurate number of rows matched in the mysql adapter (jeremyevans)
+
+* Make mock adapter with specific SQL dialect use appropriate defaults for quoting identifiers (jeremyevans)
+
+* Make list plugin automatically set position field value on creation if not already set (jeremyevans)
+
+* Add Database#integer_booleans setting on SQLite to store booleans as integers (jeremyevans)
+
+* Typecast columns stored as integers/floats in the SQLite adapter (jeremyevans)
+
+* In the instance_hooks plugin, (before|after)_*_hook instance methods now return self (jeremyevans)
+
+* Handle NaN, Infinity, and -Infinity floats on PostgreSQL (kf8a, jeremyevans) (#444)
+
+* Support an :sslmode option when using the postgres adapter with the pg driver (jeremyevans)
+
+* Add Database#create_schema and #drop_schema to the shared postgres adapter (tkellen, jeremyevans) (#440)
+
+* Add Database#supports_savepoints_in_prepared_transactions?, false on MySQL >=5.5.12 (jeremyevans) (#437)
+
+* Support an identifier output method in the mysql2 adapter (jeremyevans)
+
+* Make foreign key creation work on MySQL with InnoDB engine without specifying :key option (jeremyevans)
+
+* Allow disabling use of sudo with SUDO='' when running the rake install/uninstall tasks (jeremyevans) (#433)
+
 === 3.32.0 (2012-02-01)
 
 * Make serialization_modification_detection plugin work correctly with new objects and after saving existing objects (jeremyevans) (#432)

data/Rakefile

@@ -7,6 +7,7 @@ VERS = lambda do
   Sequel.version
 end
 CLEAN.include ["**/.*.sw?", "sequel-*.gem", ".config", "rdoc", "coverage", "www/public/*.html", "www/public/rdoc*", '**/*.rbc']
+SUDO = ENV['SUDO'] || 'sudo'
 
 # Gem Packaging and Release
 
@@ -18,12 +19,12 @@ end
 
 desc "Install sequel gem"
 task :install=>[:package] do
-  sh %{sudo gem install ./#{NAME}-#{VERS.call} --local}
+  sh %{#{SUDO} gem install ./#{NAME}-#{VERS.call} --local}
 end
 
 desc "Uninstall sequel gem"
 task :uninstall=>[:clean] do
-  sh %{sudo gem uninstall #{NAME}}
+  sh %{#{SUDO} gem uninstall #{NAME}}
 end
 
 desc "Upload sequel gem to gemcutter"

data/doc/migration.rdoc

@@ -2,7 +2,6 @@
 
 This guide is based on http://guides.rubyonrails.org/migrations.html
 
-
 == Overview
 
 Migrations make it easy to alter your database's schema in a systematic manner.
@@ -14,20 +13,20 @@ create the necessary database structure manually using Sequel's schema
 modification methods or another database tool.  However, if you are dealing
 with other developers, you'll have to send them all of the changes you are
 making.  Even if you aren't dealing with other developers, you generally have
-to make the schema changes in 3 places (development, testing, and 
+to make the schema changes in 3 places (development, testing, and
 production), and it's probably easier to use the migrations system to apply
 the schema changes than it is to keep track of the changes manually and
 execute them manually at the appropriate time.
 
 Sequel tracks which migrations you have already run, so to apply migrations
-you generally need to use run Sequel's migrator with <tt>bin/sequel -m</tt>:
+you generally need to run Sequel's migrator with <tt>bin/sequel -m</tt>:
 
   sequel -m path/to/migrations postgres://host/database
 
 Migrations in Sequel use a very simple DSL via the <tt>Sequel.migration</tt>
 method, and inside the DSL, use the <tt>Sequel::Database</tt> schema
 modification methods such as +create_table+ and +alter_table+.
-See the {schema modification guide}[link:files/doc/schema_modification_rdoc.html] 
+See the {schema modification guide}[link:files/doc/schema_modification_rdoc.html]
 for details on the schema modification methods you can use.
 
 == A Basic Migration
@@ -182,6 +181,31 @@ need to update the database by hand.
 It's recommended to always run migrations on a test database and ensure they work
 before running them on any production database.
 
+== Transactions
+
+As mentioned above, Sequel attempts to run migrations inside of a transaction by default.
+However, you can disable this on a per migration basis by calling +no_transaction+ inside
+of the Sequel.migration block:
+
+  Sequel.migration do
+    no_transaction
+    change do
+      # ...
+    end
+  end
+
+This is necessary in some cases, such as when attempting to use CREATE INDEX CONCURRENTLY
+on PostgreSQL.
+
+You can also override the transactions setting at the migrator level, either by forcing
+transactions even if no_transaction is set, or by disabling transactions all together:
+
+  # Force transaction use
+  Sequel::Migrator.run(DB, '/path/to/migrations/dir', :use_transactions=>true)
+
+  # Disable use of transactions
+  Sequel::Migrator.run(DB, '/path/to/migrations/dir', :use_transactions=>false)
+
 == Migration files
 
 While you can create migration objects yourself and apply them manually, most of the
@@ -267,7 +291,7 @@ is important though, as it is used to mean that all migrations should be unappli
 === +TimestampMigrator+ Filenames
 
 With the +TimestampMigrator+, the version integer should represent a timestamp, though this isn't strictly
-required.  
+required.
 
 For example, for <tt>5/10/2010 12:00:00pm</tt>, you could use any of the following formats:
 
@@ -287,6 +311,9 @@ be difficult to unapply some the affected migrations correctly.
 The +TimestampMigrator+ will be used if any filename in the migrations directory has a version
 greater than 20000101.  Otherwise, the +IntegerMigrator+ will be used.
 
+You can force the use of the +TimestampMigrator+ in the API by calling TimestampMigrator.apply
+instead of Migrator.apply.
+
 === How to choose
 
 Basically, unless you need the features provided by the +TimestampMigrator+, stick with the
@@ -322,8 +349,8 @@ you should give it some thought before using it.
 
 Just don't do it.
 
-In general, you should not modify any migration that has been run on the database and been committed
-the source control repository, unless the migration contains a error that causes data loss.  As long
+In general, you should not modify any migration that has been run on the database and been committed to
+the source control repository, unless the migration contains an error that causes data loss.  As long
 as it is possible to undo the migration without losing data, you should just add another migration
 that undoes the actions of the previous bad migration, and maybe does the correct action afterward.
 
@@ -358,7 +385,7 @@ or they should use the reversible migrations feature with a +change+ block:
 It's usually easy to determine what you should put in your migration's +up+ block,
 as it's whatever change you want to make to the database.  The +down+ block is
 less obvious.  In general, it should reverse the changes made by the +up+ block, which means
-it should execute the opposite of what the +up+ block does in the reverse order in which 
+it should execute the opposite of what the +up+ block does in the reverse order in which
 the +up+ block does it.  Here's an example where you are switching from having a single
 artist per album to multiple artists per album:
 
@@ -402,12 +429,12 @@ artist per album to multiple artists per album:
     end
   end
 
-Note that the order in which things were done in the +down+ block is in
-reverse order to how they were done in the +up+ block.  Also note how it
-isn't always possible to reverse exactly what was done in the +up+ block.
-You should try to do so as much as possible, but if you can't, you may
-want to have your +down+ block raise a <tt>Sequel::Error</tt> exception
-saying why the migration cannot be reverted.
+Note that the operations performed in the +down+ block are performed in the
+reverse order of how they are performed in the +up+ block.  Also note how it
+isn't always possible to reverse exactly what was done in the +up+ block.  You
+should try to do so as much as possible, but if you can't, you may want to have
+your +down+ block raise a <tt>Sequel::Error</tt> exception saying why the
+migration cannot be reverted.
 
 == Running migrations
 

data/doc/opening_databases.rdoc

@@ -345,6 +345,8 @@ The following additional options are supported:
 :encoding :: Set the client_encoding to the given string
 :connect_timeout :: Set the number of seconds to wait for a connection (default 20, only respected
                     if using the pg library).
+:sslmode :: Set to 'disable', 'allow', 'prefer', 'require' to choose how to treat SSL (only
+            respected if using the pg library)
 
 === sqlite
 

data/doc/release_notes/3.29.0.txt

@@ -130,7 +130,7 @@
 * Database#<< and Dataset#<< now return self, which allow them
   to be used in chaining:
 
-    DB << "UPADTE foo SET bar_id = NULL" << "DROP TABLE bars"
+    DB << "UPDATE foo SET bar_id = NULL" << "DROP TABLE bars"
     DB[:foo] << {:bar_id=>0} << DB[:bars].select(:id)
 
 * A Database#timezone accessor has been added, allowing you to

data/doc/release_notes/3.33.0.txt

@@ -0,0 +1,157 @@
+= New Features
+
+* A server_block extension has been added that makes Sequel's
+  sharding support easier to use by scoping database access inside
+  the block to a given server/shard:
+
+    Sequel.extension :server_block
+    DB.extend Sequel::ServerBlock
+    DB.with_server(:shard_1) do
+      # All of these will execute against shard_1
+      DB.tables
+      DB[:table].all
+      DB.run 'SOME SQL'
+    end
+
+* An arbitrary_servers extension has been added that extends
+  Sequel's sharding support so that you can use arbitrary
+  connection options instead of referencing an existing, predefined
+  server/shard:
+
+    Sequel.extension :arbitrary_servers
+    DB.pool.extend Sequel::ArbitraryServers
+    DB[:table].server(:host=>'foo', :database=>'bar').all
+
+  You can use this extension in conjunction with the server_block
+  extension:
+
+    DB.with_server(:host=>'foo', :database=>'bar') do
+      DB.synchronize do
+        # All of these will execute on host foo, database bar
+        DB.tables
+        DB[:table].all
+        DB.run 'SOME SQL'
+      end
+    end
+
+  The combination of these two extensions makes it pretty easy to
+  write a thread-safe Rack middleware that scopes each request
+  to an arbitrary database.
+
+* The sqlite adapter now supports an integer_booleans setting
+  for using 1/0 for true/false values, instead of the the 't'/'f'
+  values used by default.  As SQLite recommends using integers to
+  store booleans, converting your existing database and enabling
+  this setting is recommended, but for backwards compatibility it
+  is set to false.  You can convert you existing database by doing
+  the following for each table/column that has booleans:
+
+    DB[:table].update(:boolean_column=>{'t'=>1}.
+                        case(0, :boolean_column))
+
+  The integer_booleans default setting may change in a future
+  version of Sequel, so you should set it manually to false if you
+  prefer the current default.
+
+* You can now disable transaction use in migrations, in one of two
+  ways.  You generally only need to do this if you are using an
+  SQL query inside a migration that is specifically not supported
+  inside a transaction, such as CREATE INDEX CONCURRENTLY on
+  PostgreSQL.
+  
+  The first way to disable transactions is on a per-migration basis
+  by calling the no_transaction method inside the Sequel.migration
+  block:
+
+    Sequel.migration do
+      no_transaction
+      change do
+        # ...
+      end
+    end
+
+  That will make it so that a transaction is not used for that
+  particular migration.  The second way is passing the
+  :use_tranctions=>false option when calling Migrator.run (using
+  the API), which will completely disable transactions for all
+  migrations during the migrator run.
+
+* The postgres adapter now respects an :sslmode option when using
+  pg as the underlying driver, you can set the value of this option to
+  disable, allow, prefer, or require.
+
+* Database#create_schema and #drop_schema are now defined when
+  connecting to PostgreSQL.
+
+* Database#supports_savepoints_in_prepared_transactions? has been
+  added for checking if savepoints are supported inside prepared
+  transactions.  This is true if both savepoints and prepared
+  transactions are both supported, except on MySQL > 5.5.12 (due to
+  MySQL bug 64374).
+
+= Other Improvements
+
+* The mysql and mysql2 adapters now both provide an accurate number
+  of rows matched, so Sequel::Model usage on those adapters will now
+  raise a NoExistingObject exception by default if you attempt to
+  delete or update an instance that no longer exists in the database.
+
+* Foreign key creation now works correctly without specifying the
+  :key option when using MySQL with the InnoDB table engine.  InnoDB
+  requires that you list the column explicitly, even if you are
+  referencing the primary key of the table, so if the :key option is
+  not given, the database schema is introspected to find the primary
+  key for the table.  If you are attempting to create a table with
+  a self-referential foreign key, it introspects the generator to
+  get the primary key for the table.
+
+* The sqlite adapter will now return 1/0 stored in boolean columns as
+  true/false.  It will convert dates stored as Integers/Floats to
+  Date objects by assuming they represent the julian date.  It will
+  convert times stored as Integers/Floats to Sequel::SQLTime objects
+  by assuming they represent a number of seconds. It will convert
+  datetimes stored as Integers by assuming they represent a unix
+  epoch time integer, and datetimes stored as Floats by assuming the
+  represent the julian date (with fractional part representing the
+  time of day).  These changes make Sequel handle SQLite's
+  recommendations for boolean/date/time storage.
+
+* The instance_hooks plugin's (before|after)_*_hook methods now return
+  self so they can be used in a method chain.
+
+* The list plugin now automatically adds new entries to the end of the
+  list when creating the entries, if the position field is not
+  specifically set.
+
+* An identifier_output_method is now respected in the mysql2 adapter.
+
+* NaN/Infinity Float values are now quoted correctly for input on
+  PostgreSQL, and the postgres adapter correctly handles them on
+  retrieval from the database.
+
+* The :collate column option is now respected when creating tables or
+  altering columns on MySQL.
+
+* You can now force use of the TimestampMigrator when the
+  IntegerMigrator would be used by default by calling
+  TimestampMigrator.apply or .run.
+
+* Mock adapter usage with a specific SQL dialect now uses the
+  appropriate defaults for quoting identifiers.
+
+* You can now disable the use of sudo in the rake install/uninstall
+  tasks using the SUDO='' environment variable.
+
+* A very misleading error message has been fixed when attempting
+  to constantize an invalid string in the model inflector.
+
+= Backwards Compatibility
+
+* The sqlite adapter now typecasts columns that SQLite stores as
+  INTEGER/REAL.  Previously, it only typecasted columns that
+  SQLite stored as TEXT/BLOB.  For details about SQLite storage, see
+  http://www.sqlite.org/datatype3.html.
+
+  Any custom type conversion procs used with the sqlite adapter should
+  be modified to work with Integer/Float objects in addition to String
+  objects.

data/doc/sharding.rdoc

@@ -8,13 +8,23 @@ that ship with Sequel.
 
 == The :servers Database option
 
-Both features use the :servers Database option.  The :servers option should
-be a hash with symbol keys and values that are either hashes or procs that
-return hashes. These hashes are merged into the Database object's default
-options hash to get the connection options for the shard, so you don't need
-to override all options, just the ones that need to be modified.  For example,
-if you are using the same user, password, and database name and just the
-host is changing, you only need a :host entry in each shard's hash.
+Sharding and read_only support are both enabled via the :servers database
+option.  Using the :servers database option makes Sequel use a connection pool
+class that supports sharding, and the minimum required to enable sharding
+support is to use the empty hash:
+
+  DB=Sequel.connect('postgres://master_server/database', :servers=>{})
+
+In most cases, you are probably not going to want to use an empty hash,
+so you'll want to have entries in the hash.  Keys in the server hash are
+not restricted to type, but the general recommendation is to use a symbol
+unless you have special requirements. Values in the server hash should be
+either hashes or procs that return hashes. These hashes are merged into
+the Database object's default options hash to get the connection options
+for the shard, so you don't need to override all options, just the ones
+that need to be modified.  For example, if you are using the same user,
+password, and database name and just the host is changing, you only need
+a :host entry in each shard's hash.
 
 Note that all servers should have the same schema for all
 tables you are accessing, unless you really know what you are doing.
@@ -159,3 +169,79 @@ work well with shards.  You just need to remember to set to model to use the plu
 If all of your models are sharded, you can set all models to use the plugin via:
 
   Sequel::Model.plugin :sharding
+
+=== server_block Extension
+
+By default, you must specify the server/shard you want to use for every dataset/action,
+or Sequel will use the default shard.  If you have a group of queries that should use the
+same shard, it can get a bit redundent to specify the same shard for all of them.
+
+The server_block extension adds a Database#with_server method that scopes all database
+access inside the block to the given shard by default:
+
+  Sequel.extension :server_block
+  DB.extend Sequel::ServerBlock
+  DB.with_server(:a) do
+    # this SELECT query uses the "a" shard
+    if r = Rainbow.first(:hash=>/31337/)
+      r.count += 1
+      # this UPDATE query also uses the "a" shard
+      r.save
+    end
+  end
+
+The server_block extension doesn't currently integrate with the sharding plugin, as it
+ties into the Dataset#server method.  This shouldn't present a problem in practice as
+long as you just access the models inside the with_server block, since they will use
+the shard set by with_server by default.  However, you will probably have issues if
+you retrieve the models inside the block and save them outside of the block.  If you
+need to do that, call the server method explicitly on the dataset used to retrieve the
+model objects.
+
+=== arbitrary_servers Extension
+
+By default, Sequel's sharding support is designed to work with predefined shards.  It ships
+with Database#add_servers and Database#remove_servers methods to modify these predefined
+shards on the fly, but it is a bit cumbersome to work with truly arbitrary servers
+(requiring you to call add_servers before use, then remove_servers after use).
+
+The arbitrary_servers extension allows you to pass a server/shard options hash as the
+server to use, and those options will be merged directly into the database's default options:
+
+  Sequel.extension :arbitrary_servers
+  DB.pool.extend Sequel::ArbitraryServers
+  DB[:rainbows].server(:host=>'hash_host_a').all
+  # or
+  DB[:rainbows].server(:host=>'hash_host_b', :database=>'backup').all
+
+arbitrary_servers is designed to work well in conjunction with the server_block extension:
+
+  DB.with_server(:host=>'hash_host_b', :database=>'backup') do
+    DB.synchronize do
+      # All queries here default to the backup database on hash_host_b
+    end
+  end
+
+If you are using arbitrary_servers with server_block, you may want to
+define the following method (or something similar) so that you don't
+need to call synchronize separately:
+
+  def DB.with_server(*)
+    super{synchronize{yield}}
+  end
+
+The reason for the synchronize method is that it checks out a connection
+and makes the same connection available for the duration of the block.
+If you don't do that, Sequel will probably disconnect from the database
+and reconnect to the database on each request, since connections to
+arbitrary servers are not cached.
+
+Note that this extension only works with the sharded threaded connection
+pool.  If you are using the sharded single connection pool, you need
+to switch to the sharded threaded connection pool before using this
+extension.  If you are passing the :single_threaded option to
+the Database, just remove that option.  If you are setting:
+
+  Sequel.single_threaded = true 
+
+just remove or comment out that code.