sequel 5.40.0 → 5.45.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b3391bf9fe9e592892ad8ba6b39f7016fe94524908243e7c85fdf5b1cbf9a62a
-  data.tar.gz: 3b3c946fefa8c16b8f657c191debf49f8db84cb9cadb5a91ecdb01739c2ec1e9
+  metadata.gz: b37b49ac4a53cefbee674e57e1b64f3733ee7c330d548c59e370efe6b557cc5a
+  data.tar.gz: 0f7de640673c6dff3affc0c494609d96502f62d36d7c6b639c4e3dba2d90a89f
 SHA512:
-  metadata.gz: 1e7812b8851cdcc9e374bcd8439cc4e3dbbedfd550c72b54aa9d6aa16e3bad1e5f59eed824bdef026d784b67450d2f861be8776ddb762be58406cca75f7a0ea9
-  data.tar.gz: a60bcc72355a1916a0639c5a99ec806f157d60b515112ec24c1757f9a43efc587593a0b11a1e5312d3f40431b313f8f6fc241eee7e7a37c5362565db7c4c8d3c
+  metadata.gz: 342e7413e9a93cea694f1d1062803d4729b28f3d8adfc2f537ecd24dc988c448754c13046709d0a6505a78c0b21fb3383a1e5d55f73ee5e8364dbd0efb84fa7d
+  data.tar.gz: 9ea9cd35c75b670053804399f27475e38e28ea7b21062cf736e9bbea641c9d0862ef77edab6007d224a3a669a7469cab0c94658c4477e4d97a93c1c0355f031f

data/CHANGELOG

@@ -1,3 +1,55 @@
+=== 5.45.0 (2021-06-01)
+
+* Fix handling of NULL values in boolean columns in the ODBC adapter (jeremyevans) (#1765)
+
+* Add auto_validations_constraint_validations_presence_message plugin for auto_validations/constraint_validations presence message integration (jeremyevans)
+
+* Support Dataset#with :materialized option on SQLite 3.35+ for [NOT] MATERIALIZED (jeremyevans)
+
+* Use ALTER TABLE DROP COLUMN for dropping columns on SQLite 3.35+ (jeremyevans)
+
+=== 5.44.0 (2021-05-01)
+
+* Add concurrent_eager_loading plugin, for eager loading multiple associations concurrently using separate threads (jeremyevans)
+
+* Support :weeks as a interval unit in the date_arithmetic extension (jeremyevans) (#1759)
+
+* Raise an exception if an interval hash with an unsupported key is passed in the date_arithmetic extension (jeremyevans) (#1759)
+
+* Support dropping non-composite unique constraints on SQLite (jeremyevans) (#1755)
+
+=== 5.43.0 (2021-04-01)
+
+* Add column_encryption plugin, for encrypting column values (jeremyevans)
+
+=== 5.42.0 (2021-03-01)
+
+* Make the ado timestamp conversion proc a normal conversion proc that can be overridden similar to other conversion procs (jeremyevans)
+
+* Add :reject_nil option to the nested_attributes method, to ignore calls where nil is passed as the associated object data (jeremyevans)
+
+* Add async_thread_pool plugin for easier async usage with model classes and support for async destroy, with_pk, and with_pk! methods (jeremyevans)
+
+* Add async_thread_pool Database extension for executing queries asynchronously using a thread pool (jeremyevans)
+
+* Fix possible thread safety issue in Database#extension that could allow Module#extended to be called twice with the same Database instance (jeremyevans)
+
+* Support cases where validations make modifications beyond setting errors in Model#freeze (jeremyevans)
+
+* Add Model#to_json_data to the json_serializer plugin, returning a JSON data structure (jeremyevans)
+
+=== 5.41.0 (2021-02-01)
+
+* Have explicit :text option for a String column take priority over :size option on PostgreSQL (jeremyevans) (#1750)
+
+* Support a :skip_invalid option in auto_validations plugin for not adding errors to a column that already has an error (jeremyevans)
+
+* Support a :skip_invalid option in validation_helpers for not adding an error to a column that already has an error (jeremyevans)
+
+* Support :adder, :remover, and :clearer association options that use keyword arguments in Ruby 2.7+ (jeremyevans)
+
+* Make pg_interval use the same number of seconds per year and per month as ActiveSupport::Duration when using ActiveSupport 5.1+ (jeremyevans)
+
 === 5.40.0 (2021-01-01)
 
 * Support UPDATE FROM syntax in SQLite 3.33.0+ (jeremyevans)

data/MIT-LICENSE

@@ -1,5 +1,5 @@
 Copyright (c) 2007-2008 Sharon Rosner
-Copyright (c) 2008-2020 Jeremy Evans
+Copyright (c) 2008-2021 Jeremy Evans
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to

data/doc/release_notes/5.41.0.txt

@@ -0,0 +1,25 @@
+= New Features
+
+* The validation methods added by the validation_helpers plugin now
+  support the :skip_invalid option, which will not add a validation
+  error on a column if it already has a validation error.  This can
+  be useful if you want to avoid having duplicate errors.
+
+* The auto_validations plugin now supports a :skip_invalid plugin
+  option, which will pass the :skip_invalid option when calling
+  validation methods.
+
+= Other Improvements
+
+* The :adder, :remover, and :clearer association options now
+  support keyword arguments in Ruby 2.7+.
+
+* In the pg_interval extension, Sequel now uses the same number of
+  seconds per month and seconds per year as active_support.  It
+  originally used the same number, but active_support changed the
+  values in active_support 5.1.  Sequel now uses the active_support
+  values if they are available.
+
+* When adding a String column on PostgreSQL, an explicit text: true
+  option now takes precedence over an explicit :size option, as it
+  does in Sequel's default behavior.

data/doc/release_notes/5.42.0.txt

@@ -0,0 +1,136 @@
+= New Features
+
+* An async_thread_pool Database extension has been added, which
+  executes queries and processes results using a separate thread
+  pool.  This allows you do do things like:
+
+    foos = DB[:foos].async.all
+    bars = DB[:bars].async.select_map(:name)
+    foo_bars = DB[:foo_bars].async.each{|x| p x}
+
+  and have the three method calls (all, select_map, and each)
+  execute concurrently.  On Ruby implementations without a global
+  VM lock, such as JRuby, it will allow for parallel execution of
+  the method calls.  On CRuby, the main benefit will be for cases
+  where query execution takes a long time or there is significant
+  latency between the application and the database.
+  
+  When you call a method on foos, bars, or foo_bars, if the thread
+  pool hasn't finished processing the method, the calling code will
+  block until the method call has finished.
+
+  By default, for consistency, calling code will not preempt the
+  async thread pool.  For example, if you do:
+
+    DB[:foos].async.all.size
+
+  The calling code will always wait for the async thread pool to
+  run the all method, and then the calling code will call size on
+  the result.  This ensures that async queries will not use the
+  same connection as the the calling thread, even if calling thread
+  has a connection checked out.
+
+  In some cases, such as when the async thread pool is very busy,
+  preemption is desired for performance reasons.  If you set the
+  :preempt_async_thread Database option before loading the
+  async_thread_pool extension, preemption will be allowed.  With
+  preemption allowed, if the async thread pool has not started the
+  processing of the method at the time the calling code needs the
+  results of the method, the calling code will preempt the async
+  thread pool, and run the method on the current thread.
+
+  By default, the async thread pool uses the same number of threads as
+  the Database objects :max_connections attribute (the default for
+  that is 4).  You can modify the number of async threads by setting
+  the :num_async_threads Database option before loading the Database
+  async_thread_pool extension.
+
+  Most Dataset methods that execute queries on the database and return
+  results will operate asynchronously if the the dataset is set to be
+  asynchronous via the Dataset#async method.  This includes most
+  methods available due to the inclusion in Enumerable, even if not
+  defined by Dataset itself.
+
+  There are multiple caveats when using the async_thread_pool
+  extension:
+  
+  * Asynchronous behavior is harder to understand and harder to
+    debug.  It would be wise to only use this support in cases where
+    it provides is significant performance benefit.
+
+  * Dataset methods executed asynchronously will use a separate
+    database connection than the calling thread, so they will not
+    respect transactions in the calling thread, or other cases where
+    the calling thread checks out a connection directly using
+    Database#synchronize.  They will also not respect the use of
+    Database#with_server (from the server_block extension) in the
+    calling thread.
+
+  * Dataset methods executed asynchronously should never ignore their
+    return value.  Code such as:
+
+      DB[:table].async.insert(1)
+
+    is probablematic because without storing the return value, you
+    have no way to block until the insert has been completed.
+
+  * The returned object for Dataset methods executed asynchronously is
+    a proxy object (promise).  So you should never do:
+
+      row = DB[:table].async.first
+      # ...
+      if row
+      end
+
+      # or:
+
+      bool = DB[:table].async.get(:boolean_column)
+      # ...
+      if bool
+      end
+
+    because the if branches will always be taken as row and bool will
+    never be nil or false.  If you want to get the underlying value,
+    call itself on the proxy object (or __value if using Ruby <2.2).
+
+    For the same reason, you should not use the proxy objects directly
+    in case expressions or as arguments to Class#===.  Use itself or
+    __value in those cases.
+
+  * Dataset methods executed asynchronously that include blocks have the
+    block executed asynchronously as well, assuming that the method
+    calls the block.  Because these blocks are executed in a separate
+    thread, you cannot use control flow modifiers such as break or
+    return in them.
+
+* An async_thread_pool model plugin has been added.  This requires the
+  async_thread_pool extension has been loaded into the model's Database
+  object, and allows you to call Model.async instead of
+  Model.dataset.async.  It also adds async support to the destroy,
+  with_pk, and with_pk! model dataset methods.
+
+* Model#to_json_data has been added to the json_serializer plugin, for
+  returning a hash of data that can be converted to JSON, instead of
+  a JSON string.
+
+* A :reject_nil option has been added to the nested_attributes method
+  in the nested_attributes plugin.  This will ignore calls to the
+  nested attributes setter method where nil is passed as the setter
+  method argument.
+
+= Other Improvements
+
+* Model#freeze now works in case where model validation modifies the
+  object beyond adding errors.
+
+* Model#freeze in the composition, serialization, and
+  serialization_modification_detection plugins now works in cases
+  where validation would end up loading the composed or
+  serialized values.
+
+* Database#extension now avoids a possible thread safety issue that
+  could result in the extension being loaded into the Database twice.
+
+* The ado adapter now supports overriding the timestamp conversion
+  proc.  Previously, unlike other conversion procs, the timestamp
+  conversion proc was hard coded and could not be overridden.

data/doc/release_notes/5.43.0.txt

@@ -0,0 +1,98 @@
+= New Features
+
+* A column_encryption plugin has been added to support encrypting the
+  content of individual columns in a table.
+
+  Column values are encrypted with AES-256-GCM using a per-value
+  cipher key derived from a key provided in the configuration using
+  HMAC-SHA256.
+
+  If you would like to support encryption of columns in more than one
+  model, you should probably load the plugin into the parent class of
+  your models and specify the keys:
+ 
+    Sequel::Model.plugin :column_encryption do |enc|
+      enc.key 0, ENV["SEQUEL_COLUMN_ENCRYPTION_KEY"]
+    end
+ 
+  This specifies a single master encryption key.  Unless you are
+  actively rotating keys, it is best to use a single master key.
+ 
+  In the above call, 0 is the id of the key, and
+  ENV["SEQUEL_COLUMN_ENCRYPTION_KEY"] is the content of the key, which
+  must be a string with exactly 32 bytes. As indicated, this key
+  should not be hardcoded or otherwise committed to the source control
+  repository.
+ 
+  For models that need encrypted columns, you load the plugin again,
+  but specify the columns to encrypt:
+ 
+    ConfidentialModel.plugin :column_encryption do |enc|
+      enc.column :encrypted_column_name
+      enc.column :searchable_column_name, searchable: true
+      enc.column :ci_searchable_column_name, searchable: :case_insensitive
+    end
+ 
+  With this, all three specified columns (encrypted_column_name, 
+  searchable_column_name, and ci_searchable_column_name) will be
+  marked as encrypted columns.  When you run the following code:
+ 
+    ConfidentialModel.create(
+      encrypted_column_name: 'These',
+      searchable_column_name: 'will be',
+      ci_searchable_column_name: 'Encrypted'
+    )
+ 
+  It will save encrypted versions to the database.
+  encrypted_column_name will not be searchable, searchable_column_name
+  will be searchable with an exact match, and
+  ci_searchable_column_name will be searchable with a case insensitive
+  match.
+ 
+  To search searchable encrypted columns, use with_encrypted_value.
+  This example code will return the model instance created in the code
+  example in the previous section:
+ 
+    ConfidentialModel.
+      with_encrypted_value(:searchable_column_name, "will be")
+      with_encrypted_value(:ci_searchable_column_name, "encrypted").
+      first
+
+  To rotate encryption keys, add a new key above the existing key,
+  with a new key ID:
+ 
+    Sequel::Model.plugin :column_encryption do |enc|
+      enc.key 1, ENV["SEQUEL_COLUMN_ENCRYPTION_KEY"]
+      enc.key 0, ENV["SEQUEL_OLD_COLUMN_ENCRYPTION_KEY"]
+    end
+ 
+  Newly encrypted data will then use the new key.  Records encrypted
+  with the older key will still be decrypted correctly.
+ 
+  To force reencryption for existing records that are using the older
+  key, you can use the needing_reencryption dataset method and the
+  reencrypt instance method. For a small number of records, you can
+  probably do:
+ 
+    ConfidentialModel.needing_reencryption.all(&:reencrypt)
+ 
+  With more than a small number of records, you'll want to do this in
+  batches.  It's possible you could use an approach such as:
+ 
+    ds = ConfidentialModel.needing_reencryption.limit(100)
+    true until ds.all(&:reencrypt).empty?
+ 
+  After all values have been reencrypted for all models, and no models
+  use the older encryption key, you can remove it from the
+  configuration:
+ 
+    Sequel::Model.plugin :column_encryption do |enc|
+      enc.key 1, ENV["SEQUEL_COLUMN_ENCRYPTION_KEY"]
+    end
+
+  The column_encryption plugin supports encrypting serialized data,
+  as well as enforcing uniquenss of searchable encrypted columns
+  (in the absence of key rotation).  By design, it does not support
+  compression, mixing encrypted and unencrypted data in the same
+  column, or support arbitrary encryption ciphers.  See the plugin
+  documentation for more details.

data/doc/release_notes/5.44.0.txt

@@ -0,0 +1,32 @@
+= New Features
+
+* A concurrent_eager_loading plugin has been added.  This plugin
+  builds on top of the async_thread_pool Database extension and
+  allows eager loading multiple associations concurrently in
+  separate threads.  With this plugin, you can mark datasets for
+  concurrent eager loading using eager_load_concurrently:
+
+    Album.eager_load_concurrently.eager(:artist, :genre, :tracks).all
+
+  Datasets that are marked for concurrent eager loading will use
+  concurrent eager loading if they are eager loading more than one
+  association.  If you would like to make concurrent eager loading
+  the default, you can load the plugin with the :always option.
+
+  All of the association types that ship with Sequel now support
+  concurrent eager loading when using this plugin. For custom eager
+  loaders using the :eager_loader association option, please see the
+  documentation for the plugin for how to enable custom eager loading
+  for them.
+
+= Other Improvements
+
+* The date_arithmetic extension now handles ActiveSupport::Duration
+  values with weeks, as well as :weeks as a key in a hash value. Weeks
+  are converted into 7 days internally.
+
+* The shared SQLite adapter now emulates the dropping of non-composite
+  unique constraints.  Non-composite unique constraints are now
+  treated similarly to composite unique constraints, in that dropping
+  any unique constraints on a table will drop all unique constraints
+  on that table.

data/doc/release_notes/5.45.0.txt

@@ -0,0 +1,34 @@
+= New Features
+
+* A auto_validations_constraint_validations_presence_message plugin
+  has been added that provides integration for the auto_validations
+  and constraint_validations plugin in the following conditions:
+  
+  * The column has a NOT NULL constraint
+  * The column has a presence constraint validation with both
+    the :message and :allow_nil options used.
+
+  In this case, when saving a nil value in the column, the plugin
+  will make it so the more specific message from the presence
+  constraint validation is used, instead of the generic message
+  from auto_validations.
+
+= Other Improvements
+
+* On SQLite 3.35.0+, Sequel now uses ALTER TABLE DROP COLUMN for
+  dropping columns, instead of emulating the dropped column by
+  recreating the table.
+
+* The Dataset#with :materialized option is now supported on SQLite
+  3.35.0+ for specifying whether common table expressions should be
+  materialized.
+
+* The odbc adapter now correct handles boolean columns with NULL
+  values.  Previously, such values were returned as false instead
+  of nil.
+
+= Backwards Compatibility
+
+* The change to use ALTER TABLE DROP COLUMN on SQLite 3.35.0+ can
+  cause backwards compatibility issues if SQLite 3.35.0+ does
+  not allow dropping the column.

data/doc/sql.rdoc

@@ -544,7 +544,7 @@ On some databases, you can specify null ordering:
 
 === All Columns (.*)
 
-To select all columns in a table, Sequel supports the * method on identifiers and qualified without an argument:
+To select all columns in a table, Sequel supports the * method on identifiers and qualified identifiers without an argument:
 
   Sequel[:table].*          # "table".*
   Sequel[:schema][:table].* # "schema"."table".*

data/doc/testing.rdoc

@@ -157,9 +157,12 @@ The SEQUEL_INTEGRATION_URL environment variable specifies the Database connectio
 
 === Other
 
+SEQUEL_ASYNC_THREAD_POOL :: Use the async_thread_pool extension when running the specs
+SEQUEL_ASYNC_THREAD_POOL_PREEMPT :: Use the async_thread_pool extension when running the specs, with the :preempt_async_thread option
 SEQUEL_COLUMNS_INTROSPECTION :: Use the columns_introspection extension when running the specs
 SEQUEL_CONNECTION_VALIDATOR :: Use the connection validator extension when running the specs
 SEQUEL_DUPLICATE_COLUMNS_HANDLER :: Use the duplicate columns handler extension with value given when running the specs
+SEQUEL_CONCURRENT_EAGER_LOADING :: Use the async_thread_pool extension and concurrent_eager_loading plugin when running the specs
 SEQUEL_ERROR_SQL :: Use the error_sql extension when running the specs
 SEQUEL_INDEX_CACHING :: Use the index_caching extension when running the specs
 SEQUEL_FIBER_CONCURRENCY :: Use the fiber_concurrency extension when running the adapter and integration specs

data/doc/virtual_rows.rdoc

@@ -54,7 +54,7 @@ methods in the surrounding scope. For example:
   
   # Regular block
   ds.where{|o| o.c > a - b + @d}
-  # WHERE (c > 100)
+  # WHERE (c > 110)
   
   # Instance-evaled block
   ds.where{c > a - b + @d}

data/lib/sequel/adapters/ado.rb

@@ -195,10 +195,25 @@ module Sequel
         end
 
         @conversion_procs = CONVERSION_PROCS.dup
+        @conversion_procs[AdDBTimeStamp] = method(:adb_timestamp_to_application_timestamp)
 
         super
       end
 
+      def adb_timestamp_to_application_timestamp(v)
+        # This hard codes a timestamp_precision of 6 when converting.
+        # That is the default timestamp_precision, but the ado/mssql adapter uses a timestamp_precision
+        # of 3.  However, timestamps returned by ado/mssql have nsec values that end up rounding to a
+        # the same value as if a timestamp_precision of 3 was hard coded (either xxx999yzz, where y is
+        # 5-9 or xxx000yzz where y is 0-4).
+        #
+        # ADO subadapters should override this they would like a different timestamp precision and the
+        # this code does not work for them (for example, if they provide full nsec precision).
+        #
+        # Note that fractional second handling for WIN32OLE objects is not correct on ruby <2.2
+        to_application_timestamp([v.year, v.month, v.day, v.hour, v.min, v.sec, (v.nsec/1000.0).round * 1000])
+      end
+
       def dataset_class_default
         Dataset
       end
@@ -233,23 +248,8 @@ module Sequel
           cols = []
           conversion_procs = db.conversion_procs
 
-          ts_cp = nil
           recordset.Fields.each do |field|
-            type = field.Type
-            cp = if type == AdDBTimeStamp
-              ts_cp ||= begin
-                nsec_div = 1000000000.0/(10**(timestamp_precision))
-                nsec_mul = 10**(timestamp_precision+3)
-                meth = db.method(:to_application_timestamp)
-                lambda do |v|
-                  # Fractional second handling is not correct on ruby <2.2
-                  meth.call([v.year, v.month, v.day, v.hour, v.min, v.sec, (v.nsec/nsec_div).round * nsec_mul])
-                end
-              end
-            else
-              conversion_procs[type]
-            end
-            cols << [output_identifier(field.Name), cp]
+            cols << [output_identifier(field.Name), conversion_procs[field.Type]]
           end
 
           self.columns = cols.map(&:first)