sequelizer 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 348bea43e8571c35056619a31d45731f55b48a24cb4b93e94f23ecf7eba3a4af
-  data.tar.gz: 170ae6de8d06dc7f8030fd97151b1ff7b5666a6d94849a4fe3e056a863938a65
+  metadata.gz: 9cda95a3cd5386a1d2e0ce1402793044ba78a07f457a265efe7a45e930095cb0
+  data.tar.gz: 5cbfe2a32b9b65f9922d47fa3a9e894e2335874dfed53fd972bc9d9483ee9dd5
 SHA512:
-  metadata.gz: 497923374b34e892918aa86dea037c0b25323dc8d829398bb0e97d77386c13e248b68d47f4c7c31650d099fdf85d9cf62203703c2576f31310054f9937df9119
-  data.tar.gz: f6a8dd133701a962fab9b16a0c0df30a4897d9c08526db07ec7ddef9029eb7140adcfbbab44974be7da2d970f2203308aa419b7112baad046947a50ad4ad93f5
+  metadata.gz: 3657c7dfd5f9a040cf0b38caa910c1e5526ff874367fc261eaa2da3c3cebfaba9ffc9ca511e183466b10d019d822cfecd9e4b12e3fcbe07e59fc8075608a98de
+  data.tar.gz: 29dd4df1852d028fea492621110a08a65cf186fe672a18645c6f4832aa89c5803c514db68bcd125dc8341e892d24273e0d11eee9f16c0f154cc10c59a967a02a

data/CHANGELOG.md

@@ -2,6 +2,35 @@
 
 All notable changes to this project will be documented in this file.
 
+## [0.4.0] - 2026-10-25
+
+### Breaking changes
+
+- **options:** Passed schema wins, and a caller's after_connect still runs ([8656edc](https://github.com/outcomesinsights/sequelizer/commit/8656edc9bf9d7a5d8f3c8242139c9f779af9b3e0)): when a lower-precedence source (.env, ENV or YAML) sets search_path and the caller passes schema: or schema_search_path:, the passed value is now the search path. Before, the lower-precedence search_path was used. A source that set schema while a lower-precedence source set search_path changes the same way.
+- **funky:** Hash takes a named strategy, and md5 is the default ([5a8ef4e](https://github.com/outcomesinsights/sequelizer/commit/5a8ef4e6df604f40a943f15a30c1618fe08db27a)): funky.hash returns different ids by default. Pass strategy: :legacy to get the ids that earlier versions built.
+- **smart_select_remove:** Raise AllColumnsRemovedError when every column is removed ([d8eeca4](https://github.com/outcomesinsights/sequelizer/commit/d8eeca42bbb5e365290bc4ea911e150ba9799d1f)): code that removed every column with select_remove got SELECT * and now raises AllColumnsRemovedError.
+- **smart_select_remove:** Raise NoColumnsError instead of producing SELECT * ([810a0c7](https://github.com/outcomesinsights/sequelizer/commit/810a0c7fe8406683a185b129797d9818ba4c871e)): code that got SELECT * from select_remove on a database that returns no columns now raises NoColumnsError. conceptql loads this extension and builds SQL on Sequel.mock(host: :postgres) in its tests, so it may see this error when it moves to sequelizer 0.4.
+- Drop hashie and OptionsHash for symbol-keyed hashes ([490c924](https://github.com/outcomesinsights/sequelizer/commit/490c924cba0c87d331607a49ed7dcb5963741e36)): Sequelizer.options and Options#to_hash now return a plain Hash with Symbol keys; look keys up with symbols (h[:adapter]).
+- Remove the update_gemfile command and GemfileModifier ([4837ff4](https://github.com/outcomesinsights/sequelizer/commit/4837ff470477b8e092afaa9de612bf3cf7582580)): the update_gemfile command and Sequelizer::GemfileModifier are removed; use `bundle add <adapter gem>` instead.
+
+### Added
+
+- **settable:** Quote Sequel expression values and make set_sql public ([5882d41](https://github.com/outcomesinsights/sequelizer/commit/5882d416429d88a5cc08a391e249e51d76399770))
+
+### Changed
+
+- **deps:** Drop the activesupport runtime dependency ([3380d25](https://github.com/outcomesinsights/sequelizer/commit/3380d25805b5608fc87226eaf8a2521262aaa655))
+- **deps:** Drop the pp dependency and require ([8f186b3](https://github.com/outcomesinsights/sequelizer/commit/8f186b34c2011382ed12a67e166dab909e99d93f))
+- Delete empty ConnectionMaker#configure_adapter_specific_options ([33da14e](https://github.com/outcomesinsights/sequelizer/commit/33da14ed0ad8acd41026d4c9e27cba6c62fb1a2b))
+- Apply db_opts without patching ConnectionPool#make_new ([3507790](https://github.com/outcomesinsights/sequelizer/commit/350779069a72e7def64717dbb93eb2266e1cde00))
+
+### Fixed
+
+- **sequelizer:** A URL string at new_db and db means { url: string } ([a2e6091](https://github.com/outcomesinsights/sequelizer/commit/a2e6091f735d9f496f521430fb9942898e4bae50))
+- **connection_maker:** Raise MissingAdapterError when nothing is configured ([c531242](https://github.com/outcomesinsights/sequelizer/commit/c531242e84c66390effa145cea5e4412eabaaaf4))
+- **options:** Accept a database URL string as the url option ([73c5f61](https://github.com/outcomesinsights/sequelizer/commit/73c5f61df0c3c2037605d3b53d9c301e087b1c32))
+- **smart_select_remove:** Resolve an explicit select list without querying the database ([f23c851](https://github.com/outcomesinsights/sequelizer/commit/f23c851f1f3ef987bc41869d43e04bb776db44fc))
+
 ## [0.3.0] - 2026-10-01
 
 ### Breaking changes
@@ -142,3 +171,5 @@ All notable changes to this project will be documented in this file.
 ### Added
 
 - The project itself
+
+[0.4.0]: https://github.com/outcomesinsights/sequelizer/compare/v0.3.0...v0.4.0

data/README.md

@@ -4,7 +4,7 @@ I was tired of writing the code to bootstrap a connection to my databases.
 
 [Sequel](https://github.com/jeremyevans/sequel/) provides an easy mechanism for connecting to a database, but I didn't want to store my username/password/other sensitive information in the scripts I was writing.
 
-So I wrote this gem that lets me store my database configuration options in either config/database.yml or .env and then lets me call `db` to get a connection to my database.
+So I wrote this gem that lets me store my database configuration options in either config/sequelizer.yml or .env and then lets me call `db` to get a connection to my database.
 
 I normally use this gem when I'm writing a quick script or a Thor-based command line utility.
 
@@ -30,7 +30,7 @@ $ gem install sequelizer
 
 ## Usage
 
-To get the most out of Sequelizer, you'll need to create a config/database.yml file or a .env file and specify your database configuration options in that file.
+To get the most out of Sequelizer, you'll need to create a config/sequelizer.yml file or a .env file and specify your database configuration options in that file.
 
 Sequelizer comes with a handy command that will print out your connection parameters. Just run:
 
@@ -38,16 +38,12 @@ Sequelizer comes with a handy command that will print out your connection parame
 bundle exec sequelizer config
 ```
 
-You'll also need to make sure the gem for your database is installed. You can do this by adding the gem to your application's Gemfile.
-
-Sequelizer comes with a handy command that will update your Gemfile for you. Once you've specified your database configuration, run
+You'll also need to make sure the gem for your database is installed. You can do this by adding the gem to your application's Gemfile, e.g. for PostgreSQL:
 
 ```
-bundle exec sequelizer update_gemfile
+bundle add pg
 ```
 
-The command will look up the right gem to use with the adapter you've specified, add a line to your Gemfile specifying that gem, and run `bundle install` to install the gem for you.
-
 Once you've specified your options and made sure your database's gem is installed, simply include the Sequelizer module in any class that needs a database connection and you'll get two handy-dandy methods: `db` and `new_db`
 
 Observe:
@@ -70,15 +66,17 @@ end
 
 `new_db` will create a new connection to the database on each call.
 
-Both take a hash of database options if you don't want to create a config/database.yml or .env file, or simply wish to override those options. Options are merged together from all sources with the following precedence:
+Both take a hash of database options, or a database URL string such as `"postgres://localhost/my_db"`, if you don't want to create a config/sequelizer.yml or .env file, or simply wish to override those options. Options are merged together from all sources with the following precedence:
 
 ```
-passed_options > .env > manually defined environment variables > config/database.yml > ~/.config/sequelizer/database.yml
+passed_options > .env > manually defined environment variables > config/sequelizer.yml > ~/.config/sequelizer/database.yml
 ```
 
-So if config/database.yml specifies a connection, you can set an environment variable (either manually, or through .env) to override one of those options. Similarly, if you pass an option to the method directly, that option will override the YAML and ENV-based options. See #3 for further discussion.
+So if config/sequelizer.yml specifies a connection, you can set an environment variable (either manually, or through .env) to override one of those options. Similarly, if you pass an option to the method directly, that option will override the YAML and ENV-based options. See #3 for further discussion.
+
+For PostgreSQL, `search_path`, `schema_search_path` and `schema` are three names for one option. The highest-precedence source that sets any of them decides the search path, so a `schema:` you pass overrides a `search_path` from .env or YAML. If one source sets more than one name, `search_path` wins over `schema_search_path`, which wins over `schema`. Sequelizer creates the schemas and sets the search path on each new connection, and then runs any `after_connect` you pass.
 
-Take a look at the examples directory for a few ways you can specify your database configuration options.
+Take a look at the examples directory for a few ways you can specify your database configuration options. Copy a `sequelizer*.yml` example to `config/sequelizer.yml` to use it.
 
 ## Frustrations
 

data/lib/sequel/extensions/cold_col.rb

@@ -38,18 +38,12 @@
 #
 #   DB.extension :cold_col
 
-require 'active_support/core_ext/object/try'
-require 'active_support/core_ext/object/blank'
-
 module Sequel
-
   module ColdColDatabase
-
     # Internal schema registry for managing column information across different sources.
     # This class centralizes the storage and retrieval of table/view column metadata
     # from multiple sources including created tables, views, and manually loaded schemas.
     class SchemaRegistry
-
       # Initialize a new schema registry for the given database.
       #
       # @param db [Sequel::Database] the database instance this registry belongs to
@@ -99,7 +93,7 @@ module Sequel
         literal_name = @db.literal(name)
         table_name = name_to_string(name)
 
-        [@created_views, @created_tables, @schemas].each do |registry|
+        [ @created_views, @created_tables, @schemas ].each do |registry|
           next unless registry
 
           if (columns = Sequel.synchronize { registry[literal_name] })
@@ -145,7 +139,6 @@ module Sequel
           name.to_s
         end
       end
-
     end
 
     # Sets up the cold column tracking when the extension is loaded
@@ -165,8 +158,8 @@ module Sequel
     def load_schema(path)
       schema_data = Psych.load_file(path) || {}
       schemas = schema_data.to_h do |table, info|
-        columns = (info[:columns] || {}).map { |column_name, col_info| [column_name.to_sym, col_info] }
-        [table.to_s, columns]
+        columns = (info[:columns] || {}).map { |column_name, col_info| [ column_name.to_sym, col_info ] }
+        [ table.to_s, columns ]
       end
       cold_col_registry.merge_schemas(schemas)
     end
@@ -223,13 +216,11 @@ module Sequel
     end
 
     def columns_from_generator(generator)
-      generator.columns.map { |c| [c[:name], c] }
+      generator.columns.map { |c| [ c[:name], c ] }
     end
-
   end
 
   module ColdColDataset
-
     # Return the columns for the dataset without executing a query
     def columns
       columns_search
@@ -249,7 +240,7 @@ module Sequel
 
     protected
 
-    WILDCARD = Sequel.lit('*').freeze
+    WILDCARD = Sequel.lit("*").freeze
 
     # Determine the probable columns for a dataset based on its query options.
     # This is the main entry point for column determination logic.
@@ -259,7 +250,7 @@ module Sequel
     def probable_columns(opts_chain)
       cols = opts_chain[:select]
 
-      return columns_from_sources(opts_chain) if cols.blank?
+      return columns_from_sources(opts_chain) if cols.nil? || cols.empty?
 
       columns_from_select_list(cols, opts_chain)
     end
@@ -398,7 +389,7 @@ module Sequel
     def find_cte_columns(name, opts_chain)
       current_opts = opts_chain
 
-      while current_opts.present?
+      while current_opts && !current_opts.empty?
         with = (current_opts[:with] || []).detect { |wh| literal(wh[:name]) == literal(name) }
         return with[:dataset].columns_search(opts_chain) if with
 
@@ -415,7 +406,10 @@ module Sequel
     # @param opts_chain [Hash] the dataset's query options
     # @return [Array<Symbol>, nil] column names if found, nil otherwise
     def find_join_alias(name, opts_chain)
-      join = (opts_chain[:join] || []).detect { |jc| literal(jc.table_expr.try(:alias)) == literal(name) }
+      join = (opts_chain[:join] || []).detect do |jc|
+        table_alias = jc.table_expr.alias if jc.table_expr.respond_to?(:alias)
+        literal(table_alias) == literal(name)
+      end
       return nil unless join
 
       join_expr = join.table_expr.expression
@@ -452,9 +446,7 @@ module Sequel
         a.is_a?(SQL::Identifier) ? a.value.to_sym : a.to_sym
       end
     end
-
   end
 
   Database.register_extension(:cold_col, Sequel::ColdColDatabase)
-
 end

data/lib/sequel/extensions/db_opts.rb

@@ -1,5 +1,4 @@
 module Sequel
-
   # = DbOpts
   #
   # Sequel extension that provides database-specific options handling.
@@ -19,10 +18,8 @@ module Sequel
   #   SET work_mem='256MB'
   #   SET shared_preload_libraries='pg_stat_statements'
   module DbOpts
-
     # Handles extraction and application of database-specific options.
     class DbOptions
-
       # @!attribute [r] db
       #   @return [Sequel::Database] the database instance
       attr_reader :db
@@ -53,7 +50,7 @@ module Sequel
 
         db.opts.select do |k, _|
           k.to_s.match(opt_regexp)
-        end.to_h { |k, v| [k.to_s.gsub(opt_regexp, '').to_sym, prep_value(k, v)] }
+        end.to_h { |k, v| [ k.to_s.gsub(opt_regexp, "").to_sym, prep_value(k, v) ] }
       end
 
       # Applies the database options to the given connection.
@@ -83,9 +80,8 @@ module Sequel
       #
       # @return [Array<String>] array of SQL SET statements
       def sql_statements
-        db.send(:set_sql, to_hash)
+        db.set_sql(to_hash)
       end
-
     end
 
     # Returns a DbOptions instance for this database.
@@ -94,9 +90,7 @@ module Sequel
     def db_opts
       @db_opts ||= DbOptions.new(self)
     end
-
   end
 
   Database.register_extension(:db_opts, DbOpts)
-
 end

data/lib/sequel/extensions/funky.rb

@@ -1,43 +1,81 @@
 module Sequel
-
   module Funky
+    # Raised when funky.hash is given a strategy name it does not know.
+    class UnknownHashStrategyError < Sequel::Error
+      def initialize(strategy)
+        super("unknown funky.hash strategy #{strategy.inspect}; known strategies: #{FunkyBase::HASH_STRATEGIES.map(&:inspect).join(", ")}")
+      end
+    end
 
     class FunkyBase
+      # The names funky.hash accepts. A strategy's definition never changes,
+      # because a change would change every id built with it; a new
+      # definition gets a new name.
+      HASH_STRATEGIES = %i[md5 legacy].freeze
+
+      # ASCII unit separator: joins the parts in the md5 strategy.
+      HASH_SEPARATOR = 31
+
+      # ASCII record separator: stands in for a NULL part in the md5 strategy.
+      HASH_NULL_MARKER = 30
 
       def initialize(db)
         @db = db
       end
 
+      # Returns a signed 64-bit integer expression that hashes +parts+.
+      #
+      # strategy: :md5 (the default) casts each part to a string, writes a NULL
+      # part as chr(30), joins the parts with chr(31), and reads the first 16
+      # hex characters of their md5 as a signed integer. It gives the same id
+      # on DuckDB and on Spark.
+      #
+      # strategy: :legacy is each database's form before :md5 existed:
+      # hash(concat(...)) on DuckDB and xxhash64(...) on Spark. Use it only to
+      # reproduce ids built with it; on DuckDB it gives duplicate ids.
+      def hash(*parts, strategy: :md5)
+        case strategy
+        when :md5 then md5_hash(parts)
+        when :legacy then legacy_hash(parts)
+        else raise UnknownHashStrategyError, strategy
+        end
+      end
+
       def to_strptime(format)
         return format if format =~ /%/
 
-        format.gsub('yyyy', '%Y').gsub('MM', '%m').gsub('dd', '%d')
+        format.gsub("yyyy", "%Y").gsub("MM", "%m").gsub("dd", "%d")
       end
 
       def from_strptime(format)
         return format unless format =~ /%/
 
-        format.gsub('%Y', 'yyyy').gsub('%m', 'MM').gsub('%d', 'dd')
+        format.gsub("%Y", "yyyy").gsub("%m", "MM").gsub("%d", "dd")
       end
 
+      private
+
+      # The first 16 hex characters of the md5 of the joined parts.
+      def md5_hex16(parts)
+        strings = parts.map do |part|
+          Sequel.function(:coalesce, Sequel.cast_string(part), Sequel.function(:chr, HASH_NULL_MARKER))
+        end
+        joined = Sequel.function(:concat_ws, Sequel.function(:chr, HASH_SEPARATOR), *strings)
+        Sequel.function(:substr, Sequel.function(:md5, joined), 1, 16)
+      end
     end
 
     class FunkySpark < FunkyBase
-
       def str_to_date(value, format, try: false) # rubocop:disable Lint/UnusedMethodArgument
         Sequel.function(:to_date, Sequel.cast_string(value), format)
       end
 
-      def hash(*)
-        Sequel.function(:xxhash64, *)
-      end
-
       def make_json_column(ds, key_column, value_column)
         json_object_col = Sequel.function(
           :named_struct,
-          'key',
+          "key",
           key_column,
-          'value',
+          "value",
           value_column,
         ).then do |json_object_col|
           Sequel.function(
@@ -64,24 +102,24 @@ module Sequel
         Sequel.function(:collect_list, column)
       end
 
+      private
+
+      # conv with a negative target base reads the hex as a signed number.
+      def md5_hash(parts)
+        Sequel.function(:conv, md5_hex16(parts), 16, -10).cast(:bigint)
+      end
+
+      def legacy_hash(parts)
+        Sequel.function(:xxhash64, *parts)
+      end
     end
 
     class FunkyDuckDB < FunkyBase
-
       def str_to_date(value, format, try: false)
         strptime_func = try ? :try_strptime : :strptime
         Sequel.function(strptime_func, Sequel.cast_string(value), to_strptime(format))
       end
 
-      def hash(*)
-        Sequel.function(:hash, concat(*)).then do |hash_val|
-          Sequel.case(
-            { hash_val <= 9_223_372_036_854_775_807 => hash_val },
-            hash_val - 18_446_744_073_709_551_616,
-          ).cast(:bigint)
-        end
-      end
-
       def concat(*)
         Sequel.function(:concat, *)
       end
@@ -106,6 +144,23 @@ module Sequel
         Sequel.function(:list, column)
       end
 
+      private
+
+      def md5_hash(parts)
+        signed(Sequel.cast(Sequel.join([ "0x", md5_hex16(parts) ]), :ubigint))
+      end
+
+      def legacy_hash(parts)
+        signed(Sequel.function(:hash, concat(*parts)))
+      end
+
+      # Folds a UBIGINT into the BIGINT with the same 64 bits.
+      def signed(ubigint)
+        Sequel.case(
+          { ubigint <= 9_223_372_036_854_775_807 => ubigint },
+          ubigint - 18_446_744_073_709_551_616,
+        ).cast(:bigint)
+      end
     end
 
     def self.extended(db)
@@ -128,9 +183,7 @@ module Sequel
         raise "No known functions for #{database_type}"
       end
     end
-
   end
 
   Database.register_extension(:funky, Funky)
-
 end

data/lib/sequel/extensions/make_readyable.rb

@@ -1,5 +1,4 @@
 module Sequel
-
   # = MakeReadyable
   #
   # Sequel extension that provides database readiness functionality,
@@ -17,7 +16,6 @@ module Sequel
   # @example External file sources
   #   db.make_ready(search_path: [Pathname.new('data.parquet')])
   module MakeReadyable
-
     # Prepares the database by setting up schemas, views, and external data sources.
     #
     # This method is primarily geared towards Spark SQL-based databases.
@@ -48,7 +46,6 @@ module Sequel
     def make_ready(opts = {})
       ReadyMaker.new(self, opts).run
     end
-
   end
 
   # = ReadyMaker
@@ -57,7 +54,6 @@ module Sequel
   # This class processes the make_ready options and executes the necessary
   # SQL statements to set up schemas, views, and external data sources.
   class ReadyMaker
-
     # @!attribute [r] db
     #   @return [Sequel::Database] the database instance
     # @!attribute [r] opts
@@ -146,7 +142,6 @@ module Sequel
     # This class creates temporary views that read from external files
     # like Parquet, ORC, etc.
     class FileSourcerer
-
       # @!attribute [r] db
       #   @return [Sequel::Database] the database instance
       # @!attribute [r] schema
@@ -167,7 +162,7 @@ module Sequel
       # @param _opts [Hash] unused options parameter
       # @return [Array<Symbol>] array containing the table name
       def tables(_opts = {})
-        [schema.basename(schema.extname).to_s.to_sym]
+        [ schema.basename(schema.extname).to_s.to_sym ]
       end
 
       # Creates a temporary view that reads from the external file.
@@ -181,7 +176,7 @@ module Sequel
           db.create_view(table, {
             temp: true,
             using: format,
-            options: { path: schema.expand_path },
+            options: { path: schema.expand_path }
           }.merge(opts))
         when :duckdb
           # DuckDB uses direct file reading with read_* functions
@@ -199,20 +194,20 @@ module Sequel
       # @param _opts [Hash] additional options to merge (currently unused for DuckDB)
       def create_duckdb_view(table, _opts)
         file_path = if schema.directory?
-                      schema.expand_path.join('**').join("*.#{format}").to_s
-                    else
+                      schema.expand_path.join("**").join("*.#{format}").to_s
+        else
                       schema.expand_path.to_s
-                    end
+        end
         read_function = case format
-                        when 'parquet'
+        when "parquet"
                           :read_parquet
-                        when 'csv'
+        when "csv"
                           :read_csv_auto
-                        when 'json'
+        when "json"
                           :read_json_auto
-                        else
+        else
                           raise Sequel::Error, "Unsupported file format '#{format}' for DuckDB"
-                        end
+        end
 
         # DuckDB doesn't support TEMPORARY views, use regular CREATE VIEW
         db.create_view(table, db.from(Sequel.function(read_function, file_path)))
@@ -224,11 +219,8 @@ module Sequel
       def format
         schema.extname[1..]
       end
-
     end
-
   end
 
   Database.register_extension(:make_readyable, MakeReadyable)
-
 end

data/lib/sequel/extensions/more_sql.rb

@@ -1,7 +1,6 @@
 # frozen_string_literal: true
 
 module Sequel
-
   # Provides additional SQL helper methods for database operations.
   #
   # The more_sql extension adds convenience methods for SQL operations that
@@ -19,7 +18,6 @@ module Sequel
   #   DB.create_schema(:staging, if_not_exists: true)
   #   # Executes: CREATE SCHEMA IF NOT EXISTS "staging"
   module MoreSql
-
     # Creates a database schema.
     #
     # Generates and executes a CREATE SCHEMA statement with optional
@@ -63,14 +61,12 @@ module Sequel
     #   create_schema_sql(:test, if_not_exists: true)
     #   # => 'CREATE SCHEMA IF NOT EXISTS "test"'
     def create_schema_sql(schema_name, opts)
-      sql = 'CREATE SCHEMA '
-      sql += 'IF NOT EXISTS ' if opts[:if_not_exists]
+      sql = "CREATE SCHEMA "
+      sql += "IF NOT EXISTS " if opts[:if_not_exists]
       sql += literal(schema_name)
       sql
     end
-
   end
 
   Database.register_extension(:more_sql, MoreSql)
-
 end