sequel 4.7.0 → 4.8.0

data/doc/release_notes/4.8.0.txt

@@ -0,0 +1,175 @@
+= New Features
+
+* A one_through_one association type has been added.  This is similar
+  to the many_to_many association type in that it uses a join table,
+  but it returns a single record instead of an array of records.
+  This is designed for cases where the foreign key in the join table
+  that references the current table has a unique constraint, or where
+  you want to use an order to just pick the first matching record.
+
+  Similarly, the many_through_many plugin now also offers a
+  one_through_many association.
+
+* An association_join method has been added to model datasets, for
+  setting up joins based on associations.  This basically does the
+  same join that eager_graph would do, but does not make the other
+  changes that eager_graph makes.
+
+  Unlike eager_graph (which uses LEFT OUTER JOINs by default),
+  association_join uses INNER JOINs, but there are also
+  association_*_join methods (e.g. association_left_join) for
+  using different join types.
+
+  Similar to eager_graph, you can use cascading of associations or
+  multiple associations.
+
+    Album.association_join(:artist, :tracks)
+    Artist.association_left_join(:albums=>:tracks)
+
+* Dataset#eager_graph_with_options has been added for model
+  datasets.  It currently supports a :join_type option, for
+  overriding the type of join to use on a per-call basis, as well
+  as a :limit_strategy option.  The API is similar to eager_graph,
+  except that the associations to eagerly load are passed in as
+  a single argument, and it takes an options hash.
+
+  The :limit_strategy option works similarly to the
+  :eager_limit_strategy option when eagerly loading.  If set to
+  true and the database supports window functions, it will join
+  the current dataset to a subquery that uses a window function
+  to correctly restrict the join to only those objects that fall
+  within the association's limit/offset.
+
+  The :limit_strategy option is not on by default.  It is possible
+  for it to perform significantly worse than the default strategy
+  (which uses array slicing in ruby).  The :limit_strategy
+  significantly changes the SQL used, and can change the results
+  of the query if any filters/orders related to the association
+  are used.
+
+  It's recommended you only use the :limit_strategy option if you
+  are experiencing a bottleneck and you have benchmarked that it
+  is faster and still produces the desired results.
+
+    Artist.eager_graph_with_options(:first_10_albums,
+      :limit_strategy=>true)
+    # SELECT artists.id, artists.name,
+    #   first_10_albums.id AS first_10_albums_id,
+    #   first_10_albums.name AS first_10_albums_name,
+    #   first_10_albums.artist_id,
+    #   first_10_albums.release_date
+    # FROM artists
+    # LEFT OUTER JOIN (
+    #   SELECT id, name, artist_id, release_date
+    #   FROM (
+    #     SELECT *, row_number() OVER (PARTITION BY tracks.album_id)
+    #       AS x_sequel_row_number_x
+    #     FROM albums
+    #   ) AS t1 WHERE (x_sequel_row_number_x <= 10)
+    # ) AS first_10_albums ON (first_10_albums.artist_id = artists.id)
+
+* Dataset#full_text_search on PostgreSQL now supports :plain and
+  :phrase options.  :plain takes the search terms as a single
+  string, and searches for rows where all terms are used.
+  :phrase is similar to :plain, but also adds a substring search
+  to ensure that the string given appears verbatim in the text.
+
+* A :graph_order association option has been added, for using a
+  different order when using eager_graph.  This is mostly
+  designed for cases where :order should be qualified in other
+  cases, but using a qualification breaks eager_graph because the
+  correct qualifier is not known until runtime.
+
+* SQL::AliasedExpression#alias has been added as an alias for #aliaz.
+
+= Other Improvements
+
+* Sequel will now automatically use an eager limit strategy for
+  *_one associations that use an :order option.  For associations
+  that are truly one-to-one, an :order option is not needed, so it
+  only makes sense to have an :order option if the association
+  could theoretically return multiple results (in which case an
+  eager limit strategy is helpful).
+
+* The queries that Sequel uses to filter by associations when
+  those associations have conditions are now simpler and easier
+  for the database to execute.
+
+* The queries that Sequel uses for dataset associations now handle
+  cases where unqualified identifiers were used in the receiving
+  dataset that would be made ambiguous by a join.
+
+* A limit strategy is now used when filtering by associations if
+  the association has a limit and the database supports window
+  functions.  This allows Sequel to setup a correct filter in
+  such cases.
+
+    Artist.where(:first_10_albums=>Album[1]).all
+    # SELECT *
+    # FROM artists
+    # WHERE (artists.id IN (
+    #   SELECT albums.artist_id
+    #   FROM albums
+    #   WHERE ((albums.artist_id IS NOT NULL) AND (albums.id IN (
+    #     SELECT id FROM (
+    #       SELECT albums.id, row_number() OVER
+    #         (PARTITION BY albums.artist_id ORDER BY release_date)
+    #         AS x_sequel_row_number_x
+    #       FROM albums
+    #     ) AS t1
+    #     WHERE (x_sequel_row_number_x <= 10)
+    #   )) AND (albums.id = 1))))
+
+* A limit strategy is now used in the dataset_associations plugin
+  if the association has a limit and the database supports window
+  functions.  This makes the resulting datasets return correct
+  results.
+
+    Artist.first_10_albums
+    # SELECT *
+    # FROM albums
+    # WHERE ((albums.artist_id IN (
+    #   SELECT artists.id FROM artists)
+    # ) AND (albums.id IN (
+    #   SELECT id FROM (
+    #     SELECT albums.id, row_number() OVER
+    #       (PARTITION BY albums.artist_id ORDER BY release_date)
+    #       AS x_sequel_row_number_x
+    #     FROM albums
+    #   ) AS t1
+    #   WHERE (x_sequel_row_number_x <= 10)
+    # )))
+    # ORDER BY release_date
+
+* You can now pass symbols with embedded qualifiers or aliases,
+  as well as SQL::Identifier, SQL::QualifiedIdentifier, and
+  SQL::AliasedExpression objects as the first argument to
+  Dataset#graph.
+
+* The nested_attributes plugin now automatically handles presence
+  validations on foreign keys when creating associated objects.
+  It now sets the foreign key value (or a placeholder value)
+  before validating such objects.
+
+* Offsets on *_one associations are now respected when using
+  eager_graph.
+
+* eager graphing *_many associations with offsets no longer breaks
+  if there are no associated results.
+
+* Database#register_array_type in the pg_array extension now works
+  correctly if there is no existing scalar conversion proc for
+  the type.
+
+* Unique, foreign key, and not null constraint violations are now
+  recognized correctly on SQLite 3.8.2+.
+
+* The odbc adapter now returns fractional seconds in timestamps.
+
+* The obdc/mssql adapter now inputs timestamps with 3 decimal
+  places.
+
+= Backwards Compatibility
+
+* The private Model.apply_window_function_eager_limit_strategy
+  method has been removed.

data/lib/sequel/adapters/odbc.rb

@@ -127,7 +127,7 @@ module Sequel
         # ODBCColumn#mapSqlTypeToGenericType and Column#klass.
         case v
         when ::ODBC::TimeStamp
-          db.to_application_timestamp([v.year, v.month, v.day, v.hour, v.minute, v.second])
+          db.to_application_timestamp([v.year, v.month, v.day, v.hour, v.minute, v.second, v.fraction])
         when ::ODBC::Time
           Sequel::SQLTime.create(v.hour, v.minute, v.second)
         when ::ODBC::Date

data/lib/sequel/adapters/odbc/mssql.rb

@@ -32,10 +32,12 @@ module Sequel
       class Dataset < ODBC::Dataset
         include Sequel::MSSQL::DatasetMethods
 
+        # Use ODBC format, not Microsoft format, as the ODBC layer does
+        # some translation.  MSSQL version is over-ridden to allow 3 millisecond decimal places        
+        TIMESTAMP_FORMAT="{ts '%Y-%m-%d %H:%M:%S%N'}".freeze
+
         private
 
-        # Use ODBC format, not Microsoft format, as the ODBC layer does
-        # some translation.
         def default_timestamp_format
           TIMESTAMP_FORMAT
         end

data/lib/sequel/adapters/shared/postgres.rb

@@ -1193,12 +1193,28 @@ module Sequel
         lock_style(:share)
       end
 
-      # PostgreSQL specific full text search syntax, using tsearch2 (included
-      # in 8.3 by default, and available for earlier versions as an add-on).
+      # Run a full text search on PostgreSQL.  By default, searching for the inclusion
+      # of any of the terms in any of the cols.
+      #
+      # Options:
+      # :language :: The language to use for the search (default: 'simple')
+      # :plain :: Whether a plain search should be used (default: false).  In this case,
+      #           terms should be a single string, and it will do a search where cols
+      #           contains all of the words in terms.  This ignores search operators in terms.
+      # :phrase :: Similar to :plain, but also adding an ILIKE filter to ensure that
+      #            returned rows also include the exact phrase used.
       def full_text_search(cols, terms, opts = OPTS)
         lang = opts[:language] || 'simple'
         terms = terms.join(' | ') if terms.is_a?(Array)
-        filter("to_tsvector(?::regconfig, ?) @@ to_tsquery(?::regconfig, ?)", lang, full_text_string_join(cols), lang, terms)
+        to_tsquery = (opts[:phrase] || opts[:plain]) ? 'plainto_tsquery' : 'to_tsquery'
+
+        ds = where(Sequel.lit(["(to_tsvector(", "::regconfig, ", ") @@ #{to_tsquery}(", "::regconfig, ", "))"], lang, full_text_string_join(cols), lang, terms))
+
+        if opts[:phrase]
+          ds = ds.grep(cols, "%#{escape_like(terms)}%", :case_insensitive=>true)
+        end
+
+        ds
       end
 
       # Insert given values into the database.

data/lib/sequel/adapters/shared/sqlite.rb

@@ -332,10 +332,10 @@ module Sequel
       end
 
       DATABASE_ERROR_REGEXPS = {
-        /(is|are) not unique\z/ => UniqueConstraintViolation,
-        /foreign key constraint failed\z/ => ForeignKeyConstraintViolation,
+        /(is|are) not unique\z|UNIQUE constraint failed: .+\z/ => UniqueConstraintViolation,
+        /foreign key constraint failed\z|FOREIGN KEY constraint failed\z/ => ForeignKeyConstraintViolation,
         /\A(SQLITE ERROR 19 \(CONSTRAINT\) : )?constraint failed\z/ => ConstraintViolation,
-        /may not be NULL\z/ => NotNullConstraintViolation,
+        /may not be NULL\z|NOT NULL constraint failed: .+\z/ => NotNullConstraintViolation,
       }.freeze
       def database_error_regexps
         DATABASE_ERROR_REGEXPS

data/lib/sequel/ast_transformer.rb

@@ -33,7 +33,7 @@ module Sequel
       when SQL::OrderedExpression
         SQL::OrderedExpression.new(v(o.expression), o.descending, :nulls=>o.nulls)
       when SQL::AliasedExpression
-        SQL::AliasedExpression.new(v(o.expression), o.aliaz)
+        SQL::AliasedExpression.new(v(o.expression), o.alias)
       when SQL::CaseExpression
         args = [v(o.conditions), v(o.default)]
         args << v(o.expression) if o.expression?

data/lib/sequel/dataset/actions.rb

@@ -825,7 +825,7 @@ module Sequel
       when SQL::QualifiedIdentifier
         _hash_key_symbol(s.column, true)
       when SQL::AliasedExpression
-        _hash_key_symbol(s.aliaz, true)
+        _hash_key_symbol(s.alias, true)
       when String
         s.to_sym if recursing
       end

data/lib/sequel/dataset/graph.rb

@@ -26,7 +26,7 @@ module Sequel
     #
     # Arguments:
     # dataset :: Can be a symbol (specifying a table), another dataset,
-    #            or an object that responds to +dataset+ and returns a symbol or a dataset
+    #            or an SQL::Identifier, SQL::QualifiedIdentifier, or SQL::AliasedExpression.
     # join_conditions :: Any condition(s) allowed by +join_table+.
     # block :: A block that is passed to +join_table+.
     #
@@ -50,22 +50,36 @@ module Sequel
       # Allow the use of a dataset or symbol as the first argument
       # Find the table name/dataset based on the argument
       table_alias = options[:table_alias]
+      table = dataset
+      create_dataset = true
+
       case dataset
       when Symbol
-        table = dataset
-        dataset = @db[dataset]
-        table_alias ||= table
-      when ::Sequel::Dataset
+        # let alias be the same as the table name (sans any optional schema)
+        # unless alias explicitly given in the symbol using ___ notation
+        table_alias ||= split_symbol(table).compact.last
+      when Dataset
         if dataset.simple_select_all?
           table = dataset.opts[:from].first
           table_alias ||= table
         else
-          table = dataset
           table_alias ||= dataset_alias((@opts[:num_dataset_sources] || 0)+1)
         end
+        create_dataset = false
+      when SQL::Identifier
+        table_alias ||= table.value
+      when SQL::QualifiedIdentifier
+        table_alias ||= split_qualifiers(table).last
+      when SQL::AliasedExpression
+        return graph(table.expression, join_conditions, {:table_alias=>table.alias}.merge(options), &block)
       else
         raise Error, "The dataset argument should be a symbol or dataset"
       end
+      table_alias = table_alias.to_sym
+
+      if create_dataset
+        dataset = db.from(table)
+      end
 
       # Raise Sequel::Error with explanation that the table alias has been used
       raise_alias_error = lambda do
@@ -76,8 +90,8 @@ module Sequel
       # Only allow table aliases that haven't been used
       raise_alias_error.call if @opts[:graph] && @opts[:graph][:table_aliases] && @opts[:graph][:table_aliases].include?(table_alias)
       
-      # Use a from_self if this is already a joined table
-      ds = (!@opts[:graph] && (@opts[:from].length > 1 || @opts[:join])) ? from_self(:alias=>options[:from_self_alias] || first_source) : self
+      # Use a from_self if this is already a joined table (or from_self specifically disabled for graphs)
+      ds = (@opts[:graph_from_self] != false && !@opts[:graph] && (@opts[:from].length > 1 || @opts[:join])) ? from_self(:alias=>options[:from_self_alias] || first_source) : self
       
       # Join the table early in order to avoid cloning the dataset twice
       ds = ds.join_table(options[:join_type] || :left_outer, table, join_conditions, :table_alias=>table_alias, :implicit_qualifier=>options[:implicit_qualifier], :qualify=>options[:qualify], &block)
@@ -121,7 +135,7 @@ module Sequel
                 column = column.value if column.is_a?(SQL::Identifier)
                 column.to_sym
               when SQL::AliasedExpression
-                column = sel.aliaz
+                column = sel.alias
                 column = column.value if column.is_a?(SQL::Identifier)
                 column.to_sym
               else

data/lib/sequel/dataset/misc.rb

@@ -97,7 +97,7 @@ module Sequel
       end
       case s = source.first
       when SQL::AliasedExpression
-        s.aliaz
+        s.alias
       when Symbol
         _, _, aliaz = split_symbol(s)
         aliaz ? aliaz.to_sym : s
@@ -178,7 +178,7 @@ module Sequel
         c_table, column, aliaz = split_symbol(c)
         [c_table ? SQL::QualifiedIdentifier.new(c_table, column.to_sym) : column.to_sym, aliaz]
       when SQL::AliasedExpression
-        [c.expression, c.aliaz]
+        [c.expression, c.alias]
       when SQL::JoinClause
         [c.table, c.table_alias]
       else

data/lib/sequel/dataset/sql.rb

@@ -298,7 +298,7 @@ module Sequel
     # SQL fragment for AliasedExpression
     def aliased_expression_sql_append(sql, ae)
       literal_append(sql, ae.expression)
-      as_sql_append(sql, ae.aliaz)
+      as_sql_append(sql, ae.alias)
     end
 
     # SQL fragment for Array
@@ -801,7 +801,7 @@ module Sequel
       when SQL::QualifiedIdentifier
         alias_symbol(sym.column)
       when SQL::AliasedExpression
-        alias_alias_symbol(sym.aliaz)
+        alias_alias_symbol(sym.alias)
       else
         raise Error, "Invalid alias for alias_symbol: #{sym.inspect}"
       end
@@ -1183,7 +1183,7 @@ module Sequel
             schema, table, t_alias = split_symbol(table)
             t_alias ||= Sequel::SQL::QualifiedIdentifier.new(schema, table) if schema
           when Sequel::SQL::AliasedExpression
-            t_alias = table.aliaz
+            t_alias = table.alias
           end
           c_table = t_alias || table
         end

data/lib/sequel/extensions/columns_introspection.rb

@@ -71,7 +71,7 @@ module Sequel
         col = c.column
         col.is_a?(SQL::Identifier) ? col.value.to_sym : col.to_sym
       when SQL::AliasedExpression
-        a = c.aliaz
+        a = c.alias
         a.is_a?(SQL::Identifier) ? a.value.to_sym : a.to_sym
       end
     end

data/lib/sequel/extensions/mssql_emulate_lateral_with_apply.rb

@@ -58,7 +58,7 @@ module Sequel
         ds = from(*source)
         lateral.each do |l|
           l = if l.is_a?(Sequel::SQL::AliasedExpression)
-            l.expression.clone(:lateral=>nil).as(l.aliaz)
+            l.expression.clone(:lateral=>nil).as(l.alias)
           else
             l.clone(:lateral=>nil)
           end

data/lib/sequel/extensions/pg_array.rb

@@ -185,7 +185,7 @@ module Sequel
 
         if soid = opts[:scalar_oid]
           raise Error, "can't provide both a converter and :scalar_oid option to register" if converter 
-          raise Error, "no conversion proc for :scalar_oid=>#{soid.inspect}" unless converter = type_procs[soid]
+          converter = type_procs[soid]
         end
 
         array_type = (opts[:array_type] || db_type).to_s.dup.freeze

data/lib/sequel/extensions/pg_array_ops.rb

@@ -42,6 +42,8 @@
 #   ia.any             # ANY(int_array_column)
 #   ia.all             # ALL(int_array_column)
 #   ia.dims            # array_dims(int_array_column)
+#   ia.hstore          # hstore(int_array_column)
+#   ia.hstore(:a)      # hstore(int_array_column, a)
 #   ia.length          # array_length(int_array_column, 1)
 #   ia.length(2)       # array_length(int_array_column, 2)
 #   ia.lower           # array_lower(int_array_column, 1)
@@ -57,6 +59,10 @@
 # If you are also using the pg_array extension, you should load it before
 # loading this extension.  Doing so will allow you to use PGArray#op to get
 # an ArrayOp, allowing you to perform array operations on array literals.
+#
+# In order for #hstore to automatically wrap the returned value correctly in
+# an HStoreOp, you need to load the pg_hstore_ops extension.
+
 module Sequel
   module Postgres
     # The ArrayOp class is a simple container for a single object that

data/lib/sequel/extensions/pg_hstore_ops.rb

@@ -58,6 +58,13 @@
 # If you are also using the pg_hstore extension, you should load it before
 # loading this extension.  Doing so will allow you to use HStore#op to get
 # an HStoreOp, allowing you to perform hstore operations on hstore literals.
+#
+# Some of these methods will accept ruby arrays and convert them automatically to
+# PostgreSQL arrays if you have the pg_array extension loaded.  Some of these methods
+# will accept ruby hashes and convert them automatically to PostgreSQL hstores if the
+# pg_hstore extension is loaded.  Methods representing expressions that return
+# PostgreSQL arrays will have the returned expression automatically wrapped in a
+# Postgres::ArrayOp if the pg_array_ops extension is loaded.
 
 module Sequel
   module Postgres

data/lib/sequel/extensions/pg_json_ops.rb

@@ -49,6 +49,11 @@
 # loading this extension.  Doing so will allow you to use JSONHash#op and
 # JSONArray#op to get a JSONOp, allowing you to perform json operations
 # on json literals.
+#
+# In order to get the automatic conversion from a ruby array to a PostgreSQL array
+# (as shown in the #[] and #get_text examples above), you need to load the pg_array
+# extension.
+
 module Sequel
   module Postgres
     # The JSONOp class is a simple container for a single object that