sequel 5.11.0 → 5.12.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 935bcf2280ce67a62dfd6e705308c85324f9d82303f5144ccbede8883d0956e5
-  data.tar.gz: 91dc87ca6afcaa82372233829919bab98227e2f7a7555bc0229811cfc829b482
+  metadata.gz: 5d316ca35fb36d4b3777021b4e3d6c108036f0bb1197ab7c9efc4529012858d1
+  data.tar.gz: 7f0e338f35027945f84cfeb9d6d997114e098f46d74b7b2fc3e32db34c4f5925
 SHA512:
-  metadata.gz: f0540143323390f812a543e8a28fa2ec3184faad7ad5508e9d64cf49bf6642f09dd636f0356d8bd2a4ad632a21e5a4358eda096d86b08ddaae0f62cdf8bfc9fa
-  data.tar.gz: f85a4c4880b34f737b00affaab544473e02d70052e70ddc438597455974b411a65093f2aa511af84558a15121f3032192df191677bc7ea4b1c0b779cda1714ae
+  metadata.gz: 8d5f3c64cf5e6361ced90c90c0cfd8e3f35cfb50e2e5acfd7b25b79728dbf207bf673e03662ab72e9208cd283675e278f126d27d656a7e2f7ec0baec7330a0b4
+  data.tar.gz: c741e9fd139c5e8b13c0c434f66836bbecba900f03eecddf559f1992d4155bfcb2af1e846425d9f495fc442f09e79c82e1933dc3a362f81caced379ef608e248

data/CHANGELOG

@@ -1,3 +1,35 @@
+=== 5.12.0 (2018-08-31)
+
+* Make constraint_validations extension respect Database#constraint_validations_table setting (jeremyevans)
+
+* Make Sequel.extension load files from gems (jeremyevans)
+
+* Map clob prepared statement argument type to OCI8::CLOB in the oracle adapter (pipistrellka) (#1534)
+
+* Make Model.load_cache public in the static_cache plugin (AlexWayfer) (#1533)
+
+* Enable support for NOWAIT on MariaDB 10.3+ (jeremyevans)
+
+* Enable support for INTERSECT and EXCEPT on MariaDB 10.3+ (jeremyevans)
+
+* Make tactical_eager_loading plugin handle automatic eager loading for associated objects created by eager_graph (jeremyevans)
+
+* Cache eager_graph loader to speed up subsequent loads from the same dataset (jeremyevans)
+
+* Add caller_logging database extension to log callers before queries, useful during development (jeremyevans)
+
+* Add Database#call_procedure in the postgres adapter for calling PostgreSQL 11+ procedures (jeremyevans)
+
+* Add eager_graph_eager plugin for chaining eager association loads after eager_graph association loads (jeremyevans)
+
+* Support using Dataset#eager_graph in eager load callback for associations using join tables (jeremyevans)
+
+* Make Dataset#graph handle existing selections without determinable aliases by forcing a subselect (jeremyevans)
+
+* Freeze prepared statement arguments before returning the prepared statement (jeremyevans)
+
+* Refactor emulated prepared statement internals to use a placeholder literalizer (jeremyevans)
+
 === 5.11.0 (2018-08-01)
 
 * Fix using the jdbc/sqlserver adapter on JRuby 9.2+ (jeremyevans)

data/doc/advanced_associations.rdoc

@@ -1,29 +1,149 @@
 = Advanced Associations
 
-Sequel::Model's association support is very powerful and flexible, but using that power and flexibility
-often results in complexity.
+Sequel::Model's association support is powerful and flexible, but it can be difficult for
+new users to understand what the support enables.  This guide shows off some of the more
+advanced Sequel::Model association features.
 
 You should probably review the {Model Associations Basics and Options guide}[rdoc-ref:doc/association_basics.rdoc]
 before reviewing this guide.
 
-== Background: Sequel::Model association options
+== Sequel::Model Eager Loading
+
+Sequel::Model offers two different ways to perform eager loading, +eager+ and
++eager_graph+. +eager+ uses an SQL query per association, +eager_graph+ uses a single
+SQL query containing JOINs.
+
+Assuming the following associations:
+
+  Artist.one_to_many :albums
+  Album.one_to_many :tracks
+  Tracks.many_to_one :lyric
+
+Let's say you wanted to load all artists and eagerly load the related albums, tracks, and lyrics.
+
+  Artist.eager(albums: {tracks: :lyric})
+  # 4 Queries:
+  # SELECT * FROM artists;
+  # SELECT * FROM albums WHERE (artist_id IN (...));
+  # SELECT * FROM tracks WHERE (album_id IN (...));
+  # SELECT * FROM lyrics WHERE (id IN (...));
+
+  Artist.eager_graph(albums: {tracks: :lyric})
+  # 1 Query:
+  # SELECT artists.id, artists.name, ...
+  #        albums.id AS albums_id, albums.name AS albums_name, ...
+  #        tracks.id AS tracks_id, tracks.name AS tracks_name, ...
+  #        lyric.id AS lyric_id, ...
+  # FROM artists
+  # LEFT OUTER JOIN albums ON (albums.artist_id = artists.id)
+  # LEFT OUTER JOIN tracks ON (tracks.album_id = albums.id)
+  # LEFT OUTER JOIN lyrics AS lyric ON (lyric.id = tracks.lyric_id);
+
+In general, the recommendation is to use +eager+ unless you have a reason to use +eager_graph+.
++eager_graph+ is needed when you want to reference columns in an associated table.  For example,
+if you want to order the loading of returned artists based on the names of the albums, you cannot
+do:
+
+  Artist.eager(albums: {tracks: :lyric}).order{albums[:name]}
+
+because the initial query Sequel will use would be:
+
+  # SELECT * FROM artists ORDER BY albums.name;
+
+and +albums+ is not a valid qualifier in such a query.  In this situation, you must use +eager_graph+:
+
+  Artist.eager_graph(albums: {tracks: :lyric}).order{albums[:name]}
+
+Whether +eager+ or +eager_graph+ performs better is association and database dependent. If
+you are concerned about performance, you should try benchmarking both cases with appropriate
+data to see which performs better.
+
+=== Mixing eager and eager_graph
+
+Sequel offers the ability to mix +eager+ and +eager_graph+ when loading results.  This can
+be done at the main level by calling both +eager+ and +eager_graph+ on the same dataset:
+
+  Album.eager(:artist).eager_graph(:tracks)
+  # 2 Queries:
+  # SELECT albums.id, albums.name, ...
+  #        artist.id AS artist_id, artist.name AS artist_name, ...
+  # FROM albums
+  # LEFT OUTER JOIN artists AS artist ON (artist.id = albums.artist_id);
+  # SELECT * FROM artists WHERE (id IN (...));
+
+You can also use +eager+ to load initial associations, and +eager_graph+ to load
+remaining associations, by using +eager_graph+ in an eager load callback:
+
+  Artist.eager(albums: {tracks: proc{|ds| ds.eager_graph(:lyric)}})
+  # 3 Queries:
+  # SELECT * FROM artists;
+  # SELECT * FROM albums WHERE (artist_id IN (...));
+  # SELECT tracks.id, tracks.name, ...
+  #        lyric.id AS lyric_id, ...
+  # FROM tracks
+  # LEFT OUTER JOIN lyrics AS lyric ON (lyric.id = tracks.lyric_id)
+  # WHERE (tracks.album_id IN (...));
+
+Using the +eager_graph_eager+ plugin, you can use +eager_graph+ to load the
+initial associations, and +eager+ to load the remaining associations.  When
+you call +eager_graph_eager+, you must specify the dependency chain at
+which to start the eager loading via +eager+:
+
+  Artist.plugin :eager_graph_eager
+  Artist.eager_graph(albums: :tracks).eager_graph_eager([:albums, :tracks], :lyric)
+  # 2 Queries:
+  # SELECT artists.id, artists.name, ...
+  #        albums.id AS albums_id, albums.name AS albums_name, ...
+  #        tracks.id AS tracks_id, tracks.name AS tracks_name, ...
+  # FROM artists
+  # LEFT OUTER JOIN albums ON (albums.artist_id = artists.id)
+  # LEFT OUTER JOIN tracks ON (tracks.album_id= albums.id);
+  # SELECT * FROM lyrics WHERE (id IN (...));
+
+These two approaches can also be nested, with +eager+ -> +eager_graph+ -> +eager+:
+
+  Album.plugin :eager_graph_eager
+  Artist.eager(albums: proc{|ds| ds.eager_graph(:tracks).eager_graph_eager([:tracks], :lyric)})
+  # 3 Queries:
+  # SELECT * FROM artists;
+  # SELECT albums.id, albums.name, ...
+  #        tracks.id AS tracks_id, tracks.name AS tracks_name, ...
+  # FROM albums 
+  # LEFT OUTER JOIN tracks ON (tracks.album_id = albums.id)
+  # WHERE (albums.artist_id IN (...));
+  # SELECT * FROM lyrics WHERE (id IN (...));
+
+Or with 2 separate +eager_graph+ queries:
+
+  Artist.eager_graph(:albums).eager_graph_eager([:albums], :tracks=>proc{|ds| ds.eager_graph(:lyric)})
+  # 2 Queries:
+  # SELECT artists.id, artists.name, ...
+  #        albums.id AS albums_id, albums.name AS albums_name, ...
+  # FROM artists
+  # LEFT OUTER JOIN albums ON (albums.artist_id = artists.id);
+  # SELECT tracks.id, tracks.name, ...
+  #        lyric.id AS lyric_id, ...
+  # FROM tracks
+  # LEFT OUTER JOIN lyrics AS lyric ON (lyric.id = tracks.lyric_id)
+  # WHERE (tracks.album_id IN (...));
+
+== Sequel::Model Association Loading Options
 
 There are a bunch of advanced association options that are available to
 handle more complex cases.  First we'll go over some of the simpler ones:
 
 All associations take a block that can be used to further filter/modify the
-default dataset.  There's also an :eager_block option if you want to use
-a different block when eager loading via <tt>Dataset#eager</tt>. Association blocks are
-useful for things like:
+default dataset:
 
   Artist.one_to_many :gold_albums, class: :Album do |ds|
     ds.where{copies_sold > 500000}
   end
 
-There are a whole bunch of options for changing how the association is eagerly
-loaded via <tt>Dataset#eager_graph</tt>: <tt>:graph_block</tt>, <tt>:graph_conditions</tt>,
-<tt>:graph_only_conditions</tt>, <tt>:graph_join_type</tt> (and <tt>:graph_join_table_*</tt> ones for
-JOINing to the join table in a many_to_many association).
+There's also an :eager_block option if you want to use a different block when
+eager loading via <tt>Dataset#eager</tt>.
+
+There are many options for changing how the association is eagerly
+loaded via <tt>Dataset#eager_graph</tt>:
 
 :graph_join_type :: The type of join to do (<tt>:inner</tt>, <tt>:left</tt>, <tt>:right</tt>)
 :graph_conditions :: Additional conditions to put on join (needs to be a
@@ -62,10 +182,6 @@ These can be used like this:
   # Handles the case where both key columns have the name artist_name, and you want to use
   # a JOIN USING
   Artist.one_to_many :albums, key: :artist_name, graph_only_conditions: [:artist_name]
-  
-Remember, using +eager_graph+ is generally only necessary when you need to
-filter/order based on columns in an associated table, it is recommended to
-use +eager+ for eager loading if possible.
 
 One advantage of using +eager_graph+ is that you can easily filter/order
 on columns in an associated table on a per-query basis, using regular
@@ -83,6 +199,8 @@ For lazy loading (e.g. Model[1].association), the <tt>:dataset</tt> option can b
 to specify an arbitrary dataset (one that uses different keys, multiple keys,
 joins to other tables, etc.).
 
+== Custom Eager Loaders
+
 For eager loading via +eager+, the <tt>:eager_loader</tt> option can be used to specify
 how to eagerly load a complex association.  This is an extremely powerful
 option.  Though it can often be verbose (compared to other things in Sequel),

data/doc/postgresql.rdoc

@@ -337,6 +337,20 @@ rows that are distinct on just those columns:
   DB[:table].distinct(:id).all
   # SELECT DISTINCT ON ("id") * FROM "table"
 
+=== Calling PostgreSQL 11+ Procedures <tt>postgres only</tt>
+
+PostgreSQL 11+ added support for procedures, which are different from the user defined
+functions that PostgreSQL has historically supported.  These procedures are
+called via a special +CALL+ syntax, and Sequel supports them via
+<tt>Database#call_procedure</tt>:
+
+  DB.call_procedure(:foo, 1, "bar")
+  # CALL foo(1, 'bar')
+
+<tt>Database#call_procedure</tt> will return a hash of return values if
+the procedure returns a result, or +nil+ if the procedure does not return
+a result.
+
 === Using a Cursor to Process Large Datasets <tt>postgres only</tt>
 
 The postgres adapter offers a <tt>Dataset#use_cursor</tt> method to process large result sets

data/doc/release_notes/5.12.0.txt

@@ -0,0 +1,141 @@
+= New Features
+
+* An eager_graph_eager plugin has been added, which allows you to
+  chain eager loads using separate queries to an existing dataset that
+  uses eager_graph.  Given the following model associations:
+
+    Band.one_to_many :albums
+    Album.one_to_many :tracks
+
+  Let's say you wanted to return bands ordered by album name, and
+  eagerly load those albums, you can do that using:
+
+    Band.eager_graph(:albums).order{albums[:name]}
+
+  Let's say you also wanted to eagerly load the tracks for each album.
+  You could just add them to the eager_graph call:
+
+    Band.eager_graph(albums: :tracks).order{albums[:name]}
+
+  However, the bloats the result set, and you aren't ordering by the
+  track information, so a join is not required.  The eager_graph_eager
+  plugin allows you to specify that the tracks be eagerly loaded in a
+  separate query after the eager_graph load of albums:
+
+    Band.eager_graph(:albums).
+      eager_graph_eager([:albums], :tracks).
+      order{albums[:name]}
+
+  eager_graph_eager's first argument is a dependency chain, specified
+  as an array of symbols.  This specifies the point at which to
+  perform the eager load. The remaining arguments are arguments that
+  could be passed to Dataset#eager to specify what dependent
+  associations should be loaded at that point.
+
+* A caller_logging Database extension has been added, which logs
+  caller information before queries, filtering out the internal
+  Sequel callers.  Example:
+
+    DB.extension :caller_logging
+    DB[:table].first
+    # Logger:
+    # (0.000041s) (source: /path/to/app/foo/t.rb:12 in `get_first`)
+    # SELECT * FROM table LIMIT 1
+
+  You can further filter the caller lines by setting
+  Database#caller_logging_ignore to a regexp of additional caller
+  lines to ignore.  This is useful if you have specific methods or
+  internal extensions/plugins that you would also like to ignore as
+  they obscure the code actually making the request.
+ 
+    DB.caller_logging_ignore = %r{/path/to/app/lib/plugins}
+ 
+  You can also format the caller before it is placed in the logger,
+  using caller_logging_formatter:
+ 
+    DB.caller_logging_formatter = lambda do |caller|
+      "(#{caller.sub(/\A\/path\/to\/app\//, '')})"
+    end
+    DB[:table].first
+    # Logger:
+    # (0.000041s) (foo/t.rb:12 in `get_first`) SELECT * FROM table LIMIT 1
+ 
+* Database#call_procedure has been added to the postgres adapter, and
+  is usable on PostgreSQL 11+ for calling procedures created with
+  CREATE PROCEDURE.
+
+    DB.call_procedure(:foo, 1, "bar")
+    # CALL foo(1, 'bar')
+
+  This method will return a hash of results if the procedure returns
+  a result, or nil if it does not return a result.
+
+= Other Improvements
+
+* It is now possible to use Dataset#eager_graph in an eager load
+  callback for associations that use join tables.  This allows you
+  to eager load some associations using separate queries and other
+  associations using joins.  For example:
+
+    Band.eager(:albums=>proc{|ds| ds.eager_graph(:tracks)})
+
+  Will load the bands in one query, and load the albums and tracks
+  in a separate query using a join.  Previously, this construction
+  worked only for associations that did not use join tables.  It now
+  works for associations that use join tables, as long as existing
+  selected columns are not removed inside the callback.
+
+* The tactical_eager_loading plugin now handles automatic eager
+  loading for associated objects that were created during the
+  load of dataset that uses eager_graph.  When using the plugin,
+  the following code will now only execute 2 queries, instead of
+  issuing a separate query for each album to get the tracks for
+  the album.
+
+    artists = Artist.eager_graph(:albums).all
+    artists.each do |artist|
+      artist.albums.each do |album|
+        album.tracks
+      end
+    end
+
+* Calling Dataset#graph with a dataset with existing selections where
+  the column aliases cannot be determined automatically now works
+  correctly by using a subselect.  Previously, attempting to do this
+  would raise an exception.  This allows the following code to work:
+
+    DB[:table].select_all(:table).select_append(expr).graph(...)
+
+* Datasets now cache the EagerGraphLoader object that is generated to
+  convert arrays of hashes into an object graph, so that subsequent
+  eager loads on the same dataset do not need to recompute the same
+  information.  Most EagerGraphLoader internal state is now frozen to
+  prevent unintentional modification.
+
+* Sequel.extension now loads files from gems.  Previously, it used
+  Kernel.require, which does not load files from gems.
+
+* Adapters that emulate prepared statements using literalization now
+  use a placeholder literalizer and should execute significantly
+  faster.  More prepared statement internal metadata is now frozen
+  to prevent unintentional modification.
+
+* Dataset#intersect, #except, and #nowait are now supported on MariaDB
+  10.3+.
+
+* The constraint_validations extension now respects the
+  constraint_validations_table setting when adding metadata for the
+  constraint validations.
+
+* In the oracle adapter, the clob prepared statement argument type is
+  now mapped to the OCI8::CLOB class, allowing the use of Oracle
+  procedures with clob output parameters.
+
+* The Model.load_cache method in the static_cache plugin is now public.
+
+= Backwards Compatibility
+
+* The private Dataset#prepared_arg? method has been removed.  It is no
+  longer necessary after the refactoring to the prepared statement
+  code.  External adapters that currently call the method should be
+  updated to no longer call the method.

data/lib/sequel/adapters/ado/mssql.rb

@@ -50,7 +50,7 @@ module Sequel
         # is necessary as ADO's default :provider uses a separate native
         # connection for each query.
         def insert(*values)
-          return super if @opts[:sql] || @opts[:returning]
+          return super if (@opts[:sql] && !@opts[:prepared_sql]) || @opts[:returning]
           with_sql("SET NOCOUNT ON; #{insert_sql(*values)}; SELECT CAST(SCOPE_IDENTITY() AS INTEGER)").single_value
         end
         

data/lib/sequel/adapters/oracle.rb

@@ -115,7 +115,7 @@ module Sequel
 
       PS_TYPES = {'string'=>String, 'integer'=>Integer, 'float'=>Float,
         'decimal'=>Float, 'date'=>Time, 'datetime'=>Time,
-        'time'=>Time, 'boolean'=>String, 'blob'=>OCI8::BLOB}.freeze
+        'time'=>Time, 'boolean'=>String, 'blob'=>OCI8::BLOB, 'clob'=>OCI8::CLOB}.freeze
       def cursor_bind_params(conn, cursor, args)
         i = 0
         args.map do |arg, type|
@@ -129,6 +129,10 @@ module Sequel
             arg = arg.to_f
           when ::Sequel::SQL::Blob
             arg = ::OCI8::BLOB.new(conn, arg)
+          when String
+            if type == 'clob'
+              arg = ::OCI8::CLOB.new(conn, arg)
+            end
           end
           cursor.bind_param(i, arg, PS_TYPES[type] || arg.class)
           arg
@@ -347,11 +351,6 @@ module Sequel
           i = prepared_args.length
           LiteralString.new(":#{i}")
         end
-
-        # Always assume a prepared argument.
-        def prepared_arg?(k)
-          true
-        end
       end
       
       BindArgumentMethods = prepared_statements_module(:bind, ArgumentMapper)

data/lib/sequel/adapters/postgres.rb

@@ -173,6 +173,15 @@ module Sequel
         end
       end
 
+      # Call a procedure with the given name and arguments.  Returns a hash if the procedure
+      # returns a value, and nil otherwise.  Example:
+      #
+      #   DB.call_procedure(:foo, 1, 2)
+      #   # CALL foo(1, 2)
+      def call_procedure(name, *args)
+        dataset.send(:call_procedure, name, args)
+      end
+
       # Connects to the database.  In addition to the standard database
       # options, using the :encoding or :charset option changes the
       # client encoding for the connection, :connect_timeout is a
@@ -672,11 +681,6 @@ module Sequel
             end
             LiteralString.new("#{prepared_arg_placeholder}#{i}")
           end
-
-          # Always assume a prepared argument.
-          def prepared_arg?(k)
-           true
-          end
         end
 
         BindArgumentMethods = prepared_statements_module(:bind, [ArgumentMapper], %w'execute execute_dui')
@@ -701,6 +705,15 @@ module Sequel
       
       private
       
+      # Generate and execute a procedure call.
+      def call_procedure(name, args)
+        sql = String.new
+        sql << "CALL "
+        identifier_append(sql, name)
+        literal_append(sql, args)
+        with_sql_first(sql)
+      end
+
       # Use a cursor to fetch groups of records at a time, yielding them to the block.
       def cursor_fetch_rows(sql)
         server_opts = {:server=>@opts[:server] || :read_only}