sequel 5.33.0 → 5.58.0

data/doc/dataset_filtering.rdoc

@@ -36,8 +36,8 @@ Ranges (both inclusive and exclusive) can also be used:
 
 If you need to select multiple items from a dataset, you can supply an array:
 
-	items.where(id: [1, 38, 47, 99]).sql
-	# "SELECT * FROM items WHERE (id IN (1, 38, 47, 99))"
+  items.where(id: [1, 38, 47, 99]).sql
+  # "SELECT * FROM items WHERE (id IN (1, 38, 47, 99))"
 
 == Filtering using expressions
 

data/doc/fork_safety.rdoc

@@ -0,0 +1,84 @@
+= Fork Safety
+
+If you are forking or using a library that forks after you have created a
+Sequel::Database instance, then you must disconnect database connections before forking. If you
+don't do this, you can end up with child processes sharing database connections
+and all sorts of weird behavior, including crashes.  Sequel will automatically create new
+connections on an as needed basis in the child processes, so you only need to do the following in
+the parent process:
+
+  DB.disconnect
+
+Or if you have connections to multiple databases:
+
+  Sequel::DATABASES.each(&:disconnect)
+
+== Puma
+
+When using the Puma web server in clustered mode (which is the default behavior in Puma 5+ when
+using multiple processes), you should disconnect inside the +before_fork+ hook in your
+Puma config:
+
+  before_fork do
+    Sequel::DATABASES.each(&:disconnect)
+  end
+
+== Unicorn
+
+When using the Unicorn web server and preloading the application (+preload_app true+ in the Unicorn
+config), you should disconnect inside the +before_fork+ hook in the Unicorn config:
+
+  before_fork do
+    Sequel::DATABASES.each(&:disconnect)
+  end
+
+== Passenger
+
+In Passenger web server, you should disconnect inside the
++starting_worker_process+ event hook:
+
+  if defined?(PhusionPassenger)
+    PhusionPassenger.on_event(:starting_worker_process) do |forked|
+      Sequel::DATABASES.each(&:disconnect) if forked
+    end
+  end
+
+Note that this disconnects after forking instead of before forking.  Passenger does not
+offer a before fork hook.
+
+== Spring
+
+In Spring application preloader, you should disconnect inside the +after_fork+ hook:
+
+  if defined?(Spring)
+    Spring.after_fork do
+      Sequel::DATABASES.each(&:disconnect)
+    end
+  end
+
+As the method indicates, this disconnects after forking instead of before forking.
+Spring does not offer a before fork hook.
+
+== Resque
+
+In Resque, you should disconnect inside the +before_fork+ hook:
+
+  Resque.before_fork do |job|
+    Sequel::DATABASES.each(&:disconnect)
+  end
+
+== Parallel
+
+If you're using the Parallel gem with processes, you should disconnect before
+calling it:
+
+  Sequel::DATABASES.each(&:disconnect)
+  Parallel.map(['a','b','c'], in_processes: 3) { |one_letter| }
+
+== Other Libraries Calling fork
+
+For any other library that calls fork, you should disconnect before calling
+a method that forks:
+
+  Sequel::DATABASES.each(&:disconnect)
+  SomeLibrary.method_that_forks

data/doc/migration.rdoc

@@ -352,7 +352,7 @@ you should give it some thought before using it.
 
 == Ignoring missing migrations
 
-In some cases, you may want to allow a migration in the database that does not exist in the filesystem (deploying to an older version of code without running a down migration when deploy auto-migrates, for example). If required, you can pass <tt>allow_missing_migration_files: true</tt> as an option. This will stop errors from being raised if there are migrations in the database that do not exist in the filesystem.
+In some cases, you may want to allow a migration in the database that does not exist in the filesystem (deploying to an older version of code without running a down migration when deploy auto-migrates, for example). If required, you can pass <tt>allow_missing_migration_files: true</tt> as an option. This will stop errors from being raised if there are migrations in the database that do not exist in the filesystem. Note that the migrations themselves can still raise an error when using this option, if the database schema isn't in the state the migrations expect it to be in.  In general, the <tt>allow_missing_migration_files: true</tt> option is very risky to use, and should only be used if it is absolutely necessary.
 
 == Modifying existing migrations
 
@@ -543,16 +543,22 @@ The main difference between the two is that <tt>-d</tt> will use the type method
 with the database independent ruby class types, while <tt>-D</tt> will use
 the +column+ method with string types.
 
-Note that Sequel cannot dump constraints other than primary key and possibly
-foreign key constraints.  If you are using database features such
-as constraints or triggers, you should use your database's dump and restore
-programs instead of Sequel's schema dumper.
-
 You can take the migration created by the schema dumper to another computer
 with an empty database, and attempt to recreate the schema using:
 
   sequel -m db/migrations postgres://host/database
 
+The schema_dumper extension is quite limited in what types of
+database objects it supports.  In general, it only supports
+dumping tables, columns, primary key and foreign key constraints,
+and some indexes.  It does not support most table options, CHECK
+constraints, partial indexes, database functions, triggers,
+security grants/revokes, and a wide variety of other useful
+database properties.  Be aware of the limitations when using the
+schema_dumper extension. If you are dumping the schema to restore
+to the same database type, it is recommended to use your database's
+dump and restore programs instead of the schema_dumper extension.
+
 == Checking for Current Migrations
 
 In your application code, you may want to check that you are up to date in

data/doc/model_plugins.rdoc

@@ -212,7 +212,7 @@ In general you should only require plugin arguments if you absolutely must have
 
 == Handling Subclasses
 
-Sequel::Model uses a copy-on-subclassing approach to model state.  So instead of model subclasses asking their parent class for a value if they don't have it defined, the value is automatically copied from the parent class to the subclass when the subclass is created.  While you can do this by overriding the inherited class method, there is a available shortcut that handles most cases:
+Sequel::Model uses a copy-on-subclassing approach to model state.  So instead of having a model subclass ask its superclass for a value if the subclass don't have the value defined, the value should be copied from the parent class to the subclass when the subclass is created.  While this can be implemented by overriding the +inherited+ class method, there is an available shortcut that handles most cases:
 
   module Sequel::Plugins::Foo
     module ClassMethods

data/doc/opening_databases.rdoc

@@ -102,6 +102,7 @@ The following options can be specified and are passed to the database's internal
 :after_connect :: A callable object called after each new connection is made, with the
                   connection object (and server argument if the callable accepts 2 arguments),
                   useful for customizations that you want to apply to all connections (nil by default).
+:connect_sqls :: An array of sql strings to execute on each new connection, after :after_connect runs.
 :max_connections :: The maximum size of the connection pool (4 connections by default on most databases)
 :pool_timeout :: The number of seconds to wait if a connection cannot be acquired before raising an error (5 seconds by default)
 :single_threaded :: Whether to use a single-threaded (non-thread safe) connection pool
@@ -208,7 +209,7 @@ Example connection strings:
 
   jdbc:sqlite::memory:
   jdbc:postgresql://localhost/database?user=username
-  jdbc:mysql://localhost/test?user=root&password=root
+  jdbc:mysql://localhost/test?user=root&password=root&serverTimezone=UTC
   jdbc:h2:mem:
   jdbc:hsqldb:mem:mymemdb
   jdbc:derby:memory:myDb;create=true
@@ -240,12 +241,16 @@ The following additional options are supported:
 There are a few issues with specific jdbc driver gems:
 
 jdbc-h2 :: jdbc-h2 versions greater than 1.3.175 have issues with ORDER BY not working correctly in some cases.
+jdbc-mysql :: Depending on the configuration of the MySQL server, jdbc-mysql versions greater 8 may complain
+              about the server time zone being unrecognized.  You can either use an older jdbc-mysql version,
+              or you can specify the +serverTimezone+ option in the connection string, as shown in the example
+              jdbc:mysql connection string above.
 
 === mysql 
 
 Requires: mysql
 
-The MySQL adapter does not support the pure-ruby mysql.rb driver, it requires the C-extension driver.
+This should work with the mysql gem (C extension) and the ruby-mysql gem (pure ruby).
 
 The following additional options are supported:
 
@@ -254,6 +259,8 @@ The following additional options are supported:
 :compress :: Whether to compress data sent/received via the socket connection.
 :config_default_group :: The default group to read from the in the MySQL config file, defaults to "client")
 :config_local_infile :: If provided, sets the Mysql::OPT_LOCAL_INFILE option on the connection with the given value.
+:disable_split_materialized :: Set split_materialized=off in the optimizer settings.  Necessary to pass the associations
+                               integration tests in MariaDB 10.5+, due to a unfixed bug in the optimizer.
 :encoding :: Specify the encoding/character set to use for the connection.
 :fractional_seconds :: On MySQL 5.6.5+, this option is recognized and will include fractional seconds in
                        time/timestamp values, as well as have the schema method create columns that can contain
@@ -274,7 +281,9 @@ if either the :sslca or :sslkey option is given.
 
 This is a newer MySQL adapter that does typecasting in C, so it is often faster than the
 mysql adapter.  The options given are passed to Mysql2::Client.new, see the mysql2 documentation
-for details on what options are supported.
+for details on what options are supported. The :timeout, :auto_is_null, :sql_mode, and :disable_split_materialized
+options supported by the mysql adapter are also supported for mysql2 adapter (and any other adapters connecting to
+mysql, such as the jdbc/mysql adapter).
 
 === odbc 
 
@@ -379,6 +388,9 @@ The following additional options are supported:
 
 :readonly :: open database in read-only mode
 :timeout :: the busy timeout to use in milliseconds (default: 5000).
+:setup_regexp_function :: Whether to setup a REGEXP function in the underlying SQLite3::Database object. Doing so
+                          allows you to use regexp support in dataset expressions.  Note that this creates a new
+                          Regexp object per call to the function, so it is not an efficient implementation.
 
 Note that SQLite memory databases are restricted to a single connection by
 default.  This is because SQLite does not allow multiple connections to

data/doc/postgresql.rdoc

@@ -213,7 +213,7 @@ other tables.  Support may be added in the future.
 === Creating Unlogged Tables
 
 PostgreSQL allows users to create unlogged tables, which are faster but not crash safe.  Sequel
-allows you do create an unlogged table by specifying the <tt>unlogged: true</tt> option to +create_table+:
+allows you to create an unlogged table by specifying the <tt>unlogged: true</tt> option to +create_table+:
 
   DB.create_table(:table, unlogged: true){Integer :i}
   # CREATE UNLOGGED TABLE "table" ("i" integer)
@@ -428,6 +428,14 @@ rows that are distinct on just those columns:
   DB[:table].distinct(:id).all
   # SELECT DISTINCT ON ("id") * FROM "table"
 
+=== JOIN USING table alias
+
+Sequel allows passing an SQL::AliasedExpression to join table methods to use a USING
+join with a table alias for the USING columns:
+
+  DB[:t1].join(:t2, Sequel.as([:c1, :c2], :alias))
+  # SELECT * FROM "t1" INNER JOIN "t2" USING ("c1", "c2") AS "alias"
+
 === Calling PostgreSQL 11+ Procedures <tt>postgres only</tt>
 
 PostgreSQL 11+ added support for procedures, which are different from the user defined

data/doc/querying.rdoc

@@ -357,7 +357,9 @@ For ranges, Sequel uses a pair of inequality statements:
   # SELECT * FROM artists WHERE ((id >= 1) AND (id < 5))
 
 Finally, for regexps, Sequel uses an SQL regular expression.  Note that this
-is probably only supported on PostgreSQL and MySQL.
+is only supported by default on PostgreSQL and MySQL.  It can also be supported
+on SQLite when using the sqlite adapter with the :setup_regexp_function
+Database option.
 
   Artist.where(name: /JM$/)
   # SELECT * FROM artists WHERE (name ~ 'JM$')
@@ -498,7 +500,7 @@ filters:
   # SELECT * FROM artists WHERE (id != 5)
   
   Artist.where(id: 5).exclude{name > 'A'}
-  # SELECT * FROM artists WHERE ((id = 5) OR (name <= 'A')
+  # SELECT * FROM artists WHERE ((id = 5) AND (name <= 'A')
 
 So to do a NOT IN with an array:
 
@@ -746,7 +748,7 @@ shortcuts for all common (and most uncommon) join types. For example
 
   Album.join(:artists, id: :artist_id)
   # SELECT * FROM albums
-  # INNER JOIN artists ON (artists.id = albums.artist_id))))
+  # INNER JOIN artists ON (artists.id = albums.artist_id)
 
 And +left_join+ does a LEFT JOIN:
 
@@ -870,14 +872,14 @@ In this case, you don't even need to specify any conditions.
 ==== Join Blocks
 
 You can provide a block to any of the join methods that accept
-conditions.  This block should accept 3 arguments, the table alias
+conditions.  This block should accept 3 arguments: the table alias
 for the table currently being joined, the table alias for the last
 table joined (or first table), and an array of previous
 <tt>Sequel::SQL::JoinClause</tt>s.
 
 This allows you to qualify columns similar to how the implicit
 qualification works, without worrying about the specific aliases
-being used.  For example, lets say you wanted to join the albums
+being used.  For example, let's say you wanted to join the albums
 and artists tables, but only want albums where the artist's name
 comes before the album's name.
 

data/doc/release_notes/5.34.0.txt

@@ -0,0 +1,40 @@
+= New Features
+
+* The association_pks plugin now creates *_pks_dataset methods for
+  each association.  These are similar to the existing *_pks getter
+  methods, but they return a dataset of the keys instead of the keys
+  themselves.
+
+* The association_pks plugin now supports a :cache_pks association
+  option, which will cache calls to the *_pks getter method. The
+  default behavior remains that the *_pks getter method only returns
+  cached values if the *_pks= setter method has been used to set the
+  values.
+
+* The *_pks getter methods supported by the association_pks plugin
+  now support a :refresh option to ignore any cached values, similar
+  to how the association getter methods work.
+
+= Other Improvements
+
+* If trying to disconnect a server that doesn't exist when using a
+  sharded connection pool, a Sequel::Error is now raised.  Previously,
+  the sharded threaded pool raised a NoMethodError and the sharded
+  single connection pool did not raise an error.
+
+* If using the :savepoint option when savepoints are not supported,
+  a Sequel::InvalidOperation exception is now raised, instead of a
+  NoMethodError.
+
+* Calling Dataset#eager_graph with no arguments now returns the
+  dataset.
+
+* If not connected to the database, the single connection pool will
+  not yield any connections to Database#pool.all_connections.
+
+* Forcing a :ruby eager limit strategy for an association without a
+  limit or offset now works correctly.
+
+* Multiple unnecessary conditionals have been removed.
+
+* Sequel core and model code now have 100% branch coverage.

data/doc/release_notes/5.35.0.txt

@@ -0,0 +1,56 @@
+= New Features
+
+* An instance_specific_default plugin has been added for setting the
+  default for the :instance_specific association option, or
+  warning/raises in cases where it is not specified.  This allows
+  you to easily find associations that would be considering instance
+  specific by default, and mark them as not instance specific for
+  better performance.
+
+= Other Improvements
+
+* Setting the :instance_specific association option to false now
+  works correctly if the association uses a block.  Associations
+  that set the :dataset option are now always considered instance
+  specific, even if the :instance_specific option is explicitly
+  passed.
+
+* The validation_class_methods plugin now considers all :if,
+  :allow_missing, :allow_nil, and :allow_blank options.  Previously,
+  it only considered the first of those options that was set.
+
+* Model.finalize_associations no longer breaks if you have
+  instance-specific associations.
+
+* Model.plugin now warns if you load the plugin with arguments or a
+  block if the plugin does not accept arguments or block.  This is
+  because a future change to Sequel could break the call.
+
+* When emulating unsupported alter table operations on SQLite, Sequel
+  now copies composite unique constraints unless the alter table
+  operation is the dropping of a unique constraint.
+
+* Sequel now recognizes an additional disconnect error in the oracle
+  adapter.
+
+* In the run_transaction_hooks extension, calling
+  run_after_{commit,rollback}_hooks now raises the correct exception
+  class.
+
+* In the pg_range extension, conversion procs for the tsrange[] and
+  tstzrange[] types are not added unless the Database uses the
+  pg_array extension.
+
+* Multiple unnecessary conditionals in plugins and extensions have
+  been removed.
+
+* Sequel plugin and extension code now have 100% branch coverage.
+
+* Sequel now avoids a statement not reached verbose warning in
+  Dataset#clone.
+
+= Backwards Compatibility
+
+* The output of Dataset#to_dot in the to_dot extension has changed
+  slightly, including hash entries with nil keys.  These entries
+  were previously ignored.

data/doc/release_notes/5.36.0.txt

@@ -0,0 +1,60 @@
+= New Features
+
+* Dataset#with_ties has been added on PostgreSQL 13+ and Microsoft
+  SQL Server, which will have a limited dataset also return all
+  rows with the same order as the final row.
+
+* In the pg_json_ops extension, the following methods have been
+  added to Postgres::JSONBOp, all of which require PostgreSQL 13+:
+  
+  * #set_lax
+  * #path_exists_tz!
+  * #path_match_tz!
+  * #path_query_tz
+  * #path_query_array_tz
+  * #path_query_first_tz
+
+* On Oracle, the Database#view_exists? method now accepts a
+  :current_schema option to limit the views returned to the
+  current schema, instead of all non-system schemas.
+
+= Other Improvements
+
+* Sequel will now pass keyword arguments through in the following
+  cases:
+
+  * When loading plugins (Model.plugin)
+
+  * Class methods automically defined for methods defined in a
+    Model.dataset_module block
+
+  * Methods defined by Plugins.def_dataset_method
+
+  * Database methods called inside migrations
+
+  * Methods called via an association proxy when using the
+    association_proxies plugin.
+
+  * Dataset methods called inside a Dataset#query block when using
+    the query extension.
+
+  Previously, keywords were not handled in these cases, which would
+  cause deprecation warnings in Ruby 2.7 and ArgumentErrors in Ruby
+  3.0.  Note that Sequel itself does not use keyword arguments at
+  all, so all of these changes only affect cases where external
+  methods are defined that accept keywords, and Sequel methods are
+  called with keywords that end up being delegated to the external
+  methods.
+
+* The odbc adapter will now stream result sets instead of loading
+  the entire result set in memory and then iterating over it.
+
+* Sequel now recognizes another disconnect error in the mysql and
+  mysql2 adapters.
+
+= Backwards Compatibility
+
+* Due to the odbc adapter change to use streaming, issuing queries
+  inside a Dataset#each block will no longer work unless a different
+  shard or thread is used.  The behavior of such code is considered
+  undefined on all Sequel adapters.

data/doc/release_notes/5.37.0.txt

@@ -0,0 +1,30 @@
+= New Features
+
+* Model#column_previously_was and #column_previously_changed? have
+  been added to the dirty plugin, for getting the previous values
+  of the column before saving and for whether there were changes
+  before saving.
+  
+  Model#column_previously_changed? accepts :from and :to options
+  to allow you to more easily determine if the value changed from
+  and/or to specific values.
+
+  This information was previously obtainable via
+  Model#previous_changes, but these new methods offer a friendlier
+  interface.
+
+* Postgres::PGRow::{Array,Hash}Row#op has been added to the
+  pg_row_ops extension if the pg_row extension is loaded.  This
+  is similar to how the pg_array_ops, pg_hstore_ops, and
+  pg_json_ops and #op method to their objects.  This makes it
+  easier to perform row operations on literal rows.
+
+= Other Improvements
+
+* The schema_dumper extension now supports more unsigned numeric
+  types, such as "decimal(7,2) unsigned" and "real unsigned".
+
+* IntegerMigrator now raises an Migrator::Error if attempting to
+  migrate down when there are migration files missing and needed for
+  the down migration.  Previously, IntegerMigrator would not raise an
+  exception and would make no database changes in this case.

data/doc/release_notes/5.38.0.txt

@@ -0,0 +1,28 @@
+= New Features
+
+* The jdbc/mysql adapter now supports the newer
+  com.mysql.cj.jdbc.Driver driver.  The adapter will still attempt to
+  load the older com.mysql.jdbc.Driver if the com.mysql.cj.jdbc.Driver
+  is not found.
+
+= Other Improvements
+
+* When testing a connection after creating a new Database instance
+  raises an exception, the Database instance is removed from
+  Sequel::DATABASES.
+
+* The single_table_inheritance and prepared_statements plugins now
+  work correctly if loaded into the same class.
+
+* Database connect and disconnect errors are no longer swallowed when
+  calling Database#create_or_replace_view, Database#server_version
+  on PostgreSQL, or Database#create_table* on Oracle.
+
+= Backwards Compatibility
+
+* Previously, instantiating a new Database instance directly using
+  Sequel::Database.new did not test the connection by default.  That
+  was instead handled by Sequel::Database.connect.  The test
+  connection now happens inside Database#initialize.  This should only
+  affect backwards compatibility for code that is calling
+  Sequel::Database.new directly.

data/doc/release_notes/5.39.0.txt

@@ -0,0 +1,19 @@
+= New Features
+
+* On Microsoft SQL Server, the :clustered option is now supported
+  for primary key and unique constraints.  You can use a true value
+  for CLUSTERED and a false value for NONCLUSTERED.
+
+= Other Improvements
+
+* Partitioned tables are now included in the result of
+  Database#tables on PostgreSQL.
+
+* alter_table set_column_allow_null no longer drops the size of
+  binary columns on Microsoft SQL Server.
+
+* In the tree plugin, the roots_dataset method now works correctly
+  with queries using joins by qualifying the parent column.
+
+* A fork safety guide has been added, discussing fork safety issues
+  when using Sequel.

data/doc/release_notes/5.40.0.txt

@@ -0,0 +1,40 @@
+= New Features
+
+* On SQLite 3.33.0+, the UPDATE FROM syntax is now supported. This
+  allows you to update one table based on a join to another table.
+  The SQLite syntax is based on the PostgreSQL syntax, and the
+  Sequel API is the same for both.  You need to pass multiple tables
+  to Dataset#from.  The first table is the table to update, and the
+  remaining tables are used to construct the UPDATE FROM clause:
+
+    DB[:a, :b].where{{a[:c]=>b[:d]}}.update(:e=>'f')
+    # UPDATE a SET e = 'f' FROM b WHERE (a.c = b.d)
+
+  Unlike PostgreSQL, SQLite does not support the deletion of joined
+  datasets.  Related to this, the following methods for testing
+  database support for modifying joined datasets have been added:
+
+  * supports_updating_joins?
+  * supports_deleting_joins?
+
+= Other Improvements
+
+* The pg_interval and date_arithmetic extensions now support
+  ActiveSupport 6.1.
+
+* Sequel no longer issues method redefinition warnings in verbose
+  mode.  As Ruby 3 has dropped uninitialized instance variable
+  warnings, Sequel is now verbose warning free on Ruby 3.
+
+= Backwards Compatibility
+
+* Trying to truncate or insert into a joined dataset now correctly
+  raises an exception even if the joined dataset supports updates.
+
+* The private Dataset#check_modification_allowed! method is now
+  deprecated, and users (custom adapters) should now switch to one
+  of the more specific methods introduced in this version:
+
+  * check_insert_allowed!
+  * check_update_allowed!
+  * check_delete_allowed!

data/doc/release_notes/5.41.0.txt

@@ -0,0 +1,25 @@
+= New Features
+
+* The validation methods added by the validation_helpers plugin now
+  support the :skip_invalid option, which will not add a validation
+  error on a column if it already has a validation error.  This can
+  be useful if you want to avoid having duplicate errors.
+
+* The auto_validations plugin now supports a :skip_invalid plugin
+  option, which will pass the :skip_invalid option when calling
+  validation methods.
+
+= Other Improvements
+
+* The :adder, :remover, and :clearer association options now
+  support keyword arguments in Ruby 2.7+.
+
+* In the pg_interval extension, Sequel now uses the same number of
+  seconds per month and seconds per year as active_support.  It
+  originally used the same number, but active_support changed the
+  values in active_support 5.1.  Sequel now uses the active_support
+  values if they are available.
+
+* When adding a String column on PostgreSQL, an explicit text: true
+  option now takes precedence over an explicit :size option, as it
+  does in Sequel's default behavior.