sequel 5.39.0 → 5.72.0

data/doc/release_notes/5.62.0.txt

@@ -0,0 +1,132 @@
+= New Features
+
+* The pg_auto_parameterize extension for automatically using bound
+  variables when using postgres adapter with the pg driver has been
+  added back to Sequel.  This extension was originally added in Sequel
+  3.34.0, but was removed in 4.0.0 due to the many corner cases it
+  had.  Almost all of the corner cases have now been fixed, and
+  the extension is now recommended for production use.  Compared
+  to the original version in Sequel 3, the reintroduced version
+  of the extension includes the following changes:
+
+  * Handles integers used in LIMIT/ORDER
+  * Respects explicit CASTs
+  * Tries to convert column IN (int, ...) into column = ANY($) with an
+    array parameter
+  * Uses the same parameter for the same object used more than once in
+    a query
+  * Uses parameters when inserting multiple rows via Dataset#import
+  * Supports automatically parameterizing all of the
+    PostgreSQL-specific types that Sequel ships support for in pg_*
+    extensions (though some values of those types may not support
+    automatic parameterization).
+  * Supports skipping auto parameterization for specific values.
+
+  Automatic parameterization is generally slower than Sequel's default
+  behavior, since some optimizations Sequel uses by default do not
+  currently support automatic parameterization.
+
+  Applications may need changes to work correctly with the
+  pg_auto_parameterize extension, such as the addition of explicit
+  casts.  Please read the extension documentation for more details.
+
+* Integer column schema entries now include :min_value and :max_value
+  entries on most databases, indicating the minimum and maximum values
+  supported for the column.
+
+  The validation_helpers plugin now has validates_max_value and
+  validates_min_value for testing the column value is not greater
+  than the given maximum value and not less than the given minimum
+  value, respectively.
+
+  The auto_validations plugin now automatically uses the :min_value
+  and :max_value column schema entries with the new validation_helpers
+  methods to validate that the column values for integer columns are
+  in the allowed range.
+
+* A primary_key_lookup_check_values plugin has been added for
+  automatically typecasting and checking the primary key values are
+  in the allowed range (given by :min_value and :max_value column
+  schema entries) during lookup. If typecasting fails or the value
+  is outside the allowed range, the primary key lookup will return
+  nil without issuing a query (or will raise a NoMatchingRow error
+  if using with_pk!).
+
+  Note that this can change behavior in some cases if you are
+  passing filter conditions during lookup instead of passing primary
+  key values. The plugin tries to support most common filter
+  conditions, but there are still cases that will break.
+
+* Sequel now supports shard-specific :after_connect and :connect_sqls
+  Database options, allowing you to customize behavior for specific
+  shards:
+
+    DB = Sequel.connect('url', servers: {
+        :shard1 => {host: '...', after_connect: proc{|conn|}},
+        :shard2 => {host: '...', connect_sqls: ['...']},
+      })
+
+  Note that these shard-specific options will not be respected if you
+  are calling after_connect= or connect_sqls= on the Database's
+  connection pool.
+
+= Other Improvements
+
+* A Sequel::Postgres::IntegerOutsideBigintRange exception will now be
+  raised if trying to literalize an integer outside PostgreSQL bigint
+  range, to avoid PostgreSQL treating the integer as a numeric type
+  and not respecting indexes on the related column.
+
+  A pg_extended_integer_support extension has been added for
+  customizing the behavior when literalizing an integer outside
+  PostgreSQL bigint range, either quoting it or getting the
+  historical behavior of using it directly in the query.
+
+* Dataset#import and #multi_insert no longer use transactions when
+  they only run a single query.
+
+* Fractional seconds in timestamps are now respected in the
+  named_timezones extension.
+
+* Using hstore[] types as bound variables now works on PostgreSQL.
+
+* Using BC dates and timestamps in bound variables now works on
+  PostgreSQL.
+
+* A corner case has been fixed in eager loading where the window
+  function eager limit strategy would be used without removing the
+  row_number entries from the result.
+
+* The shared postgres adapter now caches reflection datasets, speeding
+  up Database#indexes and similar methods.
+
+* The mock postgres adapter now assumes PostgreSQL 15 instead of
+  PostgreSQL 14 by default.
+
+= Backwards Compatibility
+
+* If you are using Ruby integers outside PostgreSQL bigint range when
+  dealing with PostgreSQL numeric column values, this version may
+  not be compatible.  It is recommended you explicitly convert the
+  Ruby integers to BigDecimal objects if you are using them for
+  numeric column values.  You can also use the
+  pg_extended_integer_support extension introduced in this version.
+
+= Workaround for Older Versions
+
+* If you cannot upgrade to Sequel 5.62.0, but still want to avoid the
+  problems that come from using literal large integers on PostgreSQL,
+  you can use the following code, where DB is your Sequel::Database
+  object:
+
+    DB.extend_datasets do
+      def literal_integer(v)
+        if v > 9223372036854775807 || v < -9223372036854775808
+          raise Sequel::InvalidValue, "PostgreSQL int too large: #{v}"
+        end
+        super
+      end
+    end
+
+  This workaround should work all the way back to Sequel 3.29.0,
+  released in November 2011.

data/doc/release_notes/5.63.0.txt

@@ -0,0 +1,33 @@
+= New Features
+
+* On Ruby 3.2, the pool_class: :timed_queue Database option can now
+  be used to use an alternative connection pool that stores
+  connections in a queue, and uses the new Queue#pop :timeout option
+  in Ruby 3.2 to implement the pool timeout.  This new connection
+  pool is simpler than the default connection pool.  It is not yet
+  the default connection pool on Ruby 3.2, but it may become the
+  default in a later version.  Users of Ruby 3.2 are encouraged to
+  try out the pool_class: :timed_queue Database option and provide
+  feedback on how it works in their application.
+
+= Other Improvements
+
+* The tactical_eager_loading plugin now works in combination with the
+  single_table_inheritance and class_table_inheritance plugins, when
+  loading an association only defined in a specific subclass.
+  Previously, eager loading would be skipped in such a case.  Now,
+  an eager load will be attempted for all instances supporting the
+  association.
+
+* The validate_associated plugin now avoids database type errors for
+  non-integer association keys.  In cases where the associated object
+  doesn't have a value for the associated key, and the current object
+  does not have a key value that can be set in the associated object,
+  validation errors in the associated object related to the associated
+  key will be ignored.
+
+* Thread-keyed connection pool hashes now use compare_by_identity for
+  better performance.
+
+* The JRuby workaround in the named_timezones extension is no longer
+  used on JRuby 9.3.9.0+, as JRuby fixed the related bug.

data/doc/release_notes/5.64.0.txt

@@ -0,0 +1,50 @@
+= New Features
+
+* A :graph_use_association_block association option has been added,
+  which makes eager_graph use the association block (as eager does),
+  generally resulting in a JOIN to a subquery:
+
+    Artist.one_to_many :tracks, graph_use_association_block: true do |ds|
+      ds.where(foo: 3)
+    end
+    Artist.eager_graph(:tracks)
+    # SELECT albums.id, tracks.id AS tracks_id, tracks.album_id
+    # FROM albums
+    # LEFT OUTER JOIN (SELECT * FROM tracks WHERE (foo = 3)) AS tracks
+    # ON (tracks.album_id = albums.id)
+
+  Assuming that the database can optimize the query correctly, using
+  the :graph_use_association_block option is probably simpler than
+  than using other :graph_* options to duplicate the conditions added
+  by the association block.
+
+* Numeric/Decimal column schema entries now include :min_value and
+  :max_value entries on most databases, indicating the minimum and
+  maximum values supported for the column. Similar to the support
+  for integer columns added in 5.62.0, this allows the
+  auto_validations plugin to automatically validate the values of
+  the columns are in the allowed range.
+
+= Other Improvements
+
+* many_through_{one,many} associations now support eager_graph
+  callbacks.
+
+* The :db_type column schema entries on SQLAnywhere now include
+  precision/scale information, to work with the numeric/decimal
+  column min_value/max_value support.
+
+* The oracle adapter now includes a :column_size column schema
+  entry containing the precision of the columns, to work with the
+  numeric/decimal column min_value/max_value support.
+
+= Backwards Compatibility
+
+* The private Database#column_schema_integer_min_max_values method
+  added in 5.62.0 now takes a column schema hash instead of a
+  database type string.
+
+* Code that previously looked at the :db_type column schema entry on
+  SQLAnywhere should be updated to look at the :domain_name entry, and
+  code that looked at the :domain_name_with_size entry should be
+  updated to look at the :db_type entry.

data/doc/release_notes/5.65.0.txt

@@ -0,0 +1,21 @@
+= Improvements
+
+* The pg_auto_parameterize extension now uses a modified placeholder
+  literalizer for speeding up the generation of SQL queries in the same
+  cases where a standard dataset would use a placeholder literalizer.
+  This can provide a 4% speedup for simple queries, with greater
+  speedups for more complex queries.
+
+* Database#indexes now returns indexes for partitioned tables on
+  PostgreSQL 11+.
+
+* MySQL versions not supporting CHECK constraints no longer include
+  :min_value/:max_value schema entries for decimal/numeric columns.
+
+= Backwards Compatibility
+
+* The Dataset::PlaceholderLiterlizer::Record.loader API has changed,
+  it now accepts the Dataset::PlaceholderLiterlizer class to use as
+  the first argument.  This makes it easier to create
+  Dataset::PlaceholderLiterlizer subclasses, such as the one now used
+  by the pg_auto_parameterize extension.

data/doc/release_notes/5.66.0.txt

@@ -0,0 +1,24 @@
+= Improvements
+
+* Dataset#empty? now correctly handles datasets using custom SQL or
+  Dataset#values where the first value in the first row is NULL.
+
+* Dataset#count without an argument or block now works correctly on
+  Microsoft SQL Server when using custom SQL that uses ORDER BY.
+
+* Dataset#count now works correctly for datasets using Dataset#values.
+
+* Sequel now recognizes an additional SQLite constraint violation
+  error that occurs with recent versions of amalgalite.
+
+* Dataset#values will now raise an exception when called with an empty
+  array. Previously, an exception would not be raised until the query
+  was sent to the database.
+
+= Backwards Compatibility
+
+* The changes to make Dataset#empty? and #count work with custom SQL
+  on Microsoft SQL Server now result in running the custom SQL, which
+  could result in worse performance than in previous versions. You can
+  wrap such datasets with Dataset#from_self manually to restore the
+  previous behavior.

data/doc/release_notes/5.67.0.txt

@@ -0,0 +1,32 @@
+= New Features
+
+* A set_literalizer extension has been added, for treating Set
+  instances in datasets similar to Array instances:
+
+    DB.extension :set_literalizer
+    DB[:table].where(column: Set.new([1, 2, 3]))
+    # SELECT FROM table WHERE (column IN (1, 2, 3))
+
+= Improvements
+
+* Sequel now avoids the use of singleton classes for datasets on Ruby
+  2.4+, instead creating a regular subclass whenever a dataset would
+  be extended via #extension or #with_extend.  This significantly
+  improves performance, up to 20-40% for common dataset usage,
+  because it avoids creating new singleton classes for every dataset
+  clone, and it allows for cached method lookup.
+
+* Database#tables and #views now support a :qualify option on Microsoft
+  SQL Server to returned qualified identifiers.
+
+* The schema_dumper extension can now dump tables in non-default schemas
+  when using Microsoft SQL Server.
+
+* The schema_dumper extension now correctly dumps string column sizes
+  when using Microsoft SQL Server.
+
+= Backwards Compatibility
+
+* Calling Sequel::Dataset.register_extension where the second argument
+  is not a module now issues a deprecation warning.  Support for this
+  will be removed in Sequel 6.

data/doc/release_notes/5.68.0.txt

@@ -0,0 +1,61 @@
+= New Features
+
+* On PostgreSQL 9.6+, date, timestamp, and timestamptz columns now
+  have min_value and max_value column schema entries, allowing the
+  auto_validations plugin to automatically enforce minimum and
+  maximum values for these column types, raising a validation error
+  before saving, instead of database error when the query is sent
+  to the database.
+
+* A validation_helpers_generic_type_messages plugin has been added,
+  which improves the default type validation error messages in
+  validation_helpers.  This change was not made directly to
+  validation_helpers for backwards compatibility reasons, but will
+  probably become the default behavior in Sequel 6.  Some examples
+  of the improvements:
+
+    # :blob type
+    # validation_helpers default: "value is not a valid sequel::sql::blob"
+    # with this plugin: "value is not a blob"
+
+    # :boolean type
+    # validation_helpers default: "value is not a valid trueclass or falseclass"
+    # with this plugin: "value is not true or false"
+
+    # :datetime type
+    # validation_helpers default: "value is not a valid time or datetime"
+    # with this plugin: "value is not a valid timestamp"
+
+    # custom/database-specific types
+    # validation_helpers default: "value is not a valid sequel::class_name"
+    # with this plugin: "value is not the expected type"
+
+= Improvements
+
+* The constraint_validations plugin no longer raises validation
+  errors for missing columns that have a default value. If a column
+  is missing but has a default value, we can assume the default
+  value is valid.  Additionally, the constraint validations are now
+  based on the underlying column value and not any deserialized
+  value, so that the validation matches the constraint.
+
+* On PostgreSQL, normal type name parsing is skipped for enum,
+  array, composite, range, and multirange types, ensuring that
+  such types will not be treated incorrectly based on their
+  type name.
+
+* The pg_extended_date_support extension now works correctly with
+  the jdbc/postgresql adapter if there is already an entry in the
+  oid_convertor_map for the timestamp and timestamptz types.
+
+= Backwards Compatibility
+
+* Database#schema_column_type is no longer called for all columns
+  on PostgreSQL when parsing schema. Enum, array, composite, range,
+  and multirange types each have their own separate type parsing
+  method.  So if you were overriding Database#schema_column_type to
+  implement custom column schema parsing, you may need to adjust
+  your code.
+
+* The Sequel::Postgres::ExtendedDateSupport::DATE_YEAR_1 constant
+  has been removed.

data/doc/release_notes/5.69.0.txt

@@ -0,0 +1,26 @@
+= New Features
+
+* An adapter has been added for the trilogy MySQL driver. One large
+  advantage over mysql2 is that trilogy does not require any MySQL
+  client libraries installed on the machine. The trilogy adapter
+  has basically the same issues/skipped specs as the mysql2 adapter,
+  but it also does not support an application_timezone different
+  than the database_timezone.
+
+* Model dataset modules now have a model accessor, allowing for
+  code such as:
+
+    class Foo < Sequel::Model
+      dataset_module do
+        where :kept, Sequel[model.table_name][:discarded_at] => nil
+      end
+    end
+
+= Improvements
+
+* The mysql adapter now works with ruby-mysql 4 (the pure-ruby
+  MySQL driver).  Note that multi-results support does not work
+  with ruby-mysql 4 (it doesn't work with mysql2, trilogy, or
+  other Sequel adapters in general).
+
+* Warnings for unsupported flags are now avoided on ruby-mysql 3.

data/doc/release_notes/5.70.0.txt

@@ -0,0 +1,35 @@
+= New Features
+
+* A sharded_timed_queue connection pool has been added. This offers
+  most of the same features as the sharded_threaded connection pool,
+  but uses the new Queue#pop :timeout features added in Ruby 3.2 to
+  allow for a simpler and possibly faster and more robust
+  implementation.
+
+* If a :pool_class option is not specified when creating a Database,
+  Sequel will now look at the SEQUEL_DEFAULT_CONNECTION_POOL
+  environment variable to determine the connection pool class to use.
+  This allows you to set SEQUEL_DEFAULT_CONNECTION_POOL=timed_queue
+  on Ruby 3.2 to test with the timed_queue connection pool without
+  making any code changes.  If the :servers Database option is given,
+  Sequel will automatically use the sharded version of the connection
+  pool specified by SEQUEL_DEFAULT_CONNECTION_POOL.
+
+= Other Improvements
+
+* The connection_validator, connection_expiration, and
+  async_thread_pool extensions now work with the timed_queue and
+  sharded_timed_queue connection pools.
+
+* The sharded_threaded connection pool now disconnects connections
+  for all specified servers instead of just the last specified server
+  when using remove_server.
+
+* The static_cache plugin now recognizes when the forbid_lazy_load
+  plugin is already loaded, and does not return instances that
+  forbid lazy load for methods that return a single object, such as
+  Database.{[],cache_get_pk,first}.
+
+* Sequel now displays an informative error message if attempting to
+  load the connection_validator or connection_expiration extensions
+  when using the single threaded connection pool.

data/doc/release_notes/5.71.0.txt

@@ -0,0 +1,21 @@
+= New Features
+
+* A pg_xmin_optimistic_locking plugin has been added.  This plugin
+  uses PostgreSQL's xmin system column to implement optimistic
+  locking.  The xmin system column is automatically updated whenever
+  the database row is updated.  You can load this plugin into a
+  base model and have all models that subclass from it use optimistic
+  locking, without needing any user-defined lock columns.
+
+= Other Improvements
+
+* set_column_allow_null is now a reversible migration method inside
+  alter_table blocks.
+
+* The use of ILIKE no longer forces the ESCAPE clause on PostgreSQL,
+  which allows the use of ILIKE ANY and other constructions.  There
+  is no need to use the ESCAPE clause with ILIKE, because the value
+  Sequel uses is PostgreSQL's default.
+
+* The xid PostgreSQL type is now recognized as an integer type in the
+  jdbc/postgresql adapter.

data/doc/release_notes/5.72.0.txt

@@ -0,0 +1,33 @@
+= New Features
+
+* A pg_auto_parameterize_in_array extension has been added, which
+  handles conversion of IN/NOT IN to = ANY or != ALL for more types.
+  The pg_auto_parameterize extension only handles integer types by
+  default, because other types require the pg_array extension. This
+  new extension adds handling for Float, BigDecimal, Date, Time,
+  DateTime, Sequel::SQLTime, and Sequel::SQL::Blob types.  It can
+  also handle String types if the :treat_string_list_as_text_array
+  Database option is present, using the text type for that. Handling
+  String values as text is not the default because that may cause
+  issues for some queries.
+
+= Other Improvements
+
+* The defaults_setter plugin now does a deep copy of database
+  default values that are hash/array or delegates to hash/array.
+  This fixes cases where the database default values are mutated.
+
+* Sequel now correctly handles infinite and NaN float values used
+  inside PostgreSQL array bound variables.
+
+* The data in the cache files used by the schema_caching and
+  index_caching extensions and static_cache_cache and
+  pg_auto_constraint_validations plugins are now sorted before the
+  cache file is saved, increasing consistency between runs.
+
+* bigdecimal has been added as a dependency.  bigdecimal is currently
+  a default gem in Ruby from 1.9 to 3.2, but it will move to a
+  bundled gem in Ruby 3.4, and there will be warnings in Ruby 3.3
+  for cases that will break in Ruby 3.4.  Adding bigdecimal as a
+  dependency should avoid warnings when using bundler in Ruby 3.3,
+  and should avoid errors in Ruby 3.4.

data/doc/schema_modification.rdoc

@@ -377,7 +377,7 @@ Sequel will not add a column, but will add a composite primary key constraint:
 It is possible to specify a name for the primary key constraint: via the :name option:
 
   alter_table(:albums_artists) do
-    add_primary_key [:album_id, :artist_id], :name=>:albums_artists_pkey
+    add_primary_key [:album_id, :artist_id], name: :albums_artists_pkey
   end
 
 If you just want to take an existing single column and make it a primary key, call

data/doc/security.rdoc

@@ -127,8 +127,8 @@ a ruby string as raw SQL.  For example:
   DB.literal(Date.today) # "'2013-03-22'"
   DB.literal('a') # "'a'"
   DB.literal(Sequel.lit('a')) # "a"
-  DB.literal(:a => 'a') # "(\"a\" = 'a')"
-  DB.literal(:a => Sequel.lit('a')) # "(\"a\" = a)"
+  DB.literal(a: 'a') # "(\"a\" = 'a')"
+  DB.literal(a: Sequel.lit('a')) # "(\"a\" = a)"
 
 ==== SQL Filter Fragments
 
@@ -178,7 +178,7 @@ user input for function names.
 For backwards compatibility, Sequel supports regular strings in the
 window function :frame option, which will be treated as a literal string:
 
-  DB[:table].select{fun(arg).over(:frame=>'SQL Here')}
+  DB[:table].select{fun(arg).over(frame: 'SQL Here')}
 
 You should make sure the frame argument is not derived from user input,
 or switch to using a hash as the :frame option value.
@@ -237,7 +237,7 @@ or:
 
 Instead, you should do:
 
-  DB[:table].update(:column => params[:value].to_s) # Safe
+  DB[:table].update(column: params[:value].to_s) # Safe
 
 Because using the auto_literal_strings extension makes SQL injection
 so much eaiser, it is recommended to not use it, and instead
@@ -402,29 +402,29 @@ This issue isn't necessarily specific to Sequel, but it is a good general practi
 If you are using values derived from user input, it is best to be explicit about
 their type.  For example:
 
-  Album.where(:id=>params[:id])
+  Album.where(id: params[:id])
 
 is probably a bad idea.  Assuming you are using a web framework, <tt>params[:id]</tt> could
 be a string, an array, a hash, nil, or potentially something else.
 
 Assuming that +id+ is an integer field, you probably want to do:
 
-  Album.where(:id=>params[:id].to_i)
+  Album.where(id: params[:id].to_i)
 
 If you are looking something up by name, you should try to enforce the value to be
 a string:
 
-  Album.where(:name=>params[:name].to_s)
+  Album.where(name: params[:name].to_s)
 
 If you are trying to use an IN clause with a list of id values based on input provided
 on a web form:
 
-  Album.where(:id=>params[:ids].to_a.map(&:to_i))
+  Album.where(id: params[:ids].to_a.map(&:to_i))
 
 Basically, be as explicit as possible. While there aren't any known security issues
 in Sequel when you do:
 
-  Album.where(:id=>params[:id])
+  Album.where(id: params[:id])
 
 It allows the attacker to choose to do any of the following queries:
 

data/doc/sharding.rdoc

@@ -39,7 +39,9 @@ is the simplest configuration:
     servers: {read_only: {host: 'replica_server'}})
 
 This will use the replica_server for SELECT queries and primary_server for
-other queries.
+other queries. The :read_only key in the :servers hash is special in that
+it sets the default database for Dataset methods that use SELECT queries
+(which are generally read queries that do not modify the database).
 
 If you want to ensure your queries are going to a specific database, you
 can force this for a given query by using the .server method and passing