sequel-duckdb 0.4.0 → 0.4.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f9ed53934b345068605d21e881ef98049be3dcbb07213c6d5238ce6566c15cdf
-  data.tar.gz: 92a1cdd312058eb4459084f5eb9069b7780cf54f56ccfbc15ae7f7f931e0fbfd
+  metadata.gz: 7d728d2d4748d8ace682454b06155cea6ae570a8fbfbf888efa7d2a314077e5c
+  data.tar.gz: 525588b5baceefd4e7b0e3e61f429efb1c505d03250be98ca109bb9567bfce63
 SHA512:
-  metadata.gz: c372ff00e47bb0680b7e2601fbc5ab61a281e0cd7c68c4764c6f2355d8701014b99d54ad364d77f6f8becba32ed9e9dc6b09d9ae73c9f53d179abbbc70bdbd96
-  data.tar.gz: 5dce4902a7eb0f5b25ec86e129afc8f0c9962cdd2f960dea8d352827401303d9cc5bb189e1879369891cbaf0764facc9f8c726ef5d3b3ecac51ec9b7a21f0d71
+  metadata.gz: 63164b64d30f71edcb66edb7e3a01ed07074de9312673b51b823e264268f702609fd6fcf98481535c7fc9165f49fa6eb38aa28adfd4311f129ef10d3b9a6bb8b
+  data.tar.gz: 76f6fb5c6e9d2684a013176f3ff7b9414ba6e0a9e67e88d36312e5cb57df5ddda4805ad84f4506ce7c60576461748194e24c2727596493b735eb434bec00cfc7

data/CHANGELOG.md

@@ -5,6 +5,26 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.4.1] - 2026-10-26
+
+### Added
+
+- Support for the DuckDB C library 2.x and the duckdb gem 2.x, alongside 1.5.x. CI
+  tests library 1.5.6 and 2.0.0, and the duckdb gem 1.5.x and 2.x. Support for library
+  1.5.x ends when DuckDB 1.5.x reaches end of life in March 2027. sequel-duckdb still
+  does not depend on the duckdb gem, so each project picks its own version.
+- **database:** Add attach and detach for DuckDB catalogs ([f82169b](https://github.com/outcomesinsights/sequel-duckdb/commit/f82169b0ad4cd66abe588dfdad431053535151e1))
+
+### Changed
+
+- **adapter:** Remove the unused primary key caches ([9a14254](https://github.com/outcomesinsights/sequel-duckdb/commit/9a14254abd8494e71af9e5478d081ff048b46701))
+
+### Fixed
+
+- **connection:** Honor the readonly option ([e8e94c5](https://github.com/outcomesinsights/sequel-duckdb/commit/e8e94c5a5a210aacac0a53cc8190f3dc76b3bbdf))
+- **schema:** Keep the transaction when a name lookup misses on DuckDB 2.x ([b3c02c6](https://github.com/outcomesinsights/sequel-duckdb/commit/b3c02c6029095e11fb048ef2b3c0e84a6c62e3ec))
+- **schema:** Let a mock Database drop and create schemas ([f9aad74](https://github.com/outcomesinsights/sequel-duckdb/commit/f9aad74f94fc9d4ed024accd9ccb4bc6ecdb628e))
+
 ## [0.4.0] - 2026-10-25
 
 ### Breaking changes
@@ -348,3 +368,4 @@ tag-based release workflow.
 [0.3.0]: https://github.com/outcomesinsights/sequel-duckdb/compare/v0.2.1...v0.3.0
 [0.3.1]: https://github.com/outcomesinsights/sequel-duckdb/compare/v0.3.0...v0.3.1
 [0.4.0]: https://github.com/outcomesinsights/sequel-duckdb/compare/v0.3.1...v0.4.0
+[0.4.1]: https://github.com/outcomesinsights/sequel-duckdb/compare/v0.4.0...v0.4.1

data/README.md

@@ -286,6 +286,12 @@ db = Sequel.connect('duckdb://relative/path/to/database.duckdb')
 db = Sequel.connect('duckdb:///path/to/database.duckdb?readonly=true')
 ```
 
+`readonly` opens the file with DuckDB's `access_mode` set to `READ_ONLY`, so writes
+raise `Sequel::DatabaseError`. All `Sequel::Database` objects on one file in one
+process share one DuckDB instance, so the first one to open the file sets its
+settings. Opening the same file again with other settings, such as read-write after
+read-only, raises `Sequel::DatabaseConnectionError`.
+
 ### Connection Hash
 
 ```ruby
@@ -539,6 +545,38 @@ db[:users].copy_to_sql("/data/users.csv", header: true, delimiter: "|")
 # => COPY (SELECT * FROM "users") TO '/data/users.csv' (FORMAT CSV, HEADER, DELIMITER '|')
 ```
 
+### Attaching database files
+
+`Database#attach` runs a DuckDB `ATTACH`, and `Database#detach` runs `DETACH DATABASE`.
+The catalog name `as:` is required. A name that DuckDB derives from the file name can
+equal a schema name inside the file, and then `schema.table` is ambiguous. Other DuckDB
+`ATTACH` options pass through, for example `type: :sqlite`.
+
+```ruby
+db.attach("/data/dataset.duckdb", as: :dataset, read_only: true)
+db[Sequel[:dataset][:people]].count
+db.detach(:dataset, if_exists: true)
+
+db.attach_sql("/data/dataset.duckdb", as: :dataset, read_only: true, if_not_exists: true)
+# => ATTACH IF NOT EXISTS '/data/dataset.duckdb' AS "dataset" (READ_ONLY)
+```
+
+An attached catalog belongs to the DuckDB database, which all pool connections of a
+`Sequel::Database` share. It lasts until `Database#disconnect` closes the last
+connection. After that, attach it again; `if_not_exists: true` makes a repeat safe.
+
+`search_path` is different: DuckDB keeps it per connection. Set it on every new
+connection with Sequel's `connect_sqls` option. Put the writable catalog first, because
+`CREATE TABLE` without a schema goes to the first entry, and a read-only catalog there
+makes it fail.
+
+```ruby
+db = Sequel.connect("duckdb:///exports/42/work.duckdb",
+                    connect_sqls: [ "SET search_path = 'work.main'" ])
+db.attach("/data/dataset.duckdb", as: :dataset, read_only: true)
+db.create_table(:summary) { Integer :n } # created in work.main
+```
+
 ### Lists
 
 Sequel writes a Ruby Array as a parenthesized list, such as `(1, 2, 3)`, for `IN`
@@ -765,10 +803,17 @@ db[:users].where(active: true).all
 
 ### Version Compatibility
 
-- **Ruby**: 3.1.0 or higher
+- **Ruby**: 3.3 or higher (CI tests 3.3, 3.4 and 4.0)
 - **Sequel**: 5.0 or higher
-- **DuckDB**: 0.8.0 or higher
-- **ruby-duckdb**: 1.0.0 or higher
+- **DuckDB C library**: 1.5.x and 2.x. Support for 1.5.x ends when DuckDB 1.5.x
+  reaches end of life in March 2027.
+- **ruby-duckdb (the `duckdb` gem)**: 1.5.x and 2.x. sequel-duckdb does not depend on
+  it, so add `gem "duckdb"` to your own Gemfile. ruby-duckdb 2.x builds against
+  library 1.4 or newer, so it works with library 1.5.x.
+
+DuckDB 2.x aborts an open transaction after any failed statement, including a query
+on a missing table; 1.5.x does not. `table_exists?` and `schema` keep the transaction
+on both versions.
 
 ### Getting Help
 

data/lib/sequel/adapters/duckdb.rb

@@ -69,12 +69,10 @@ module Sequel
       # (InstanceRegistry), and #disconnect gives it back once no connection is open.
       def adapter_initialize
         @conversion_procs = {}
-        @primary_keys = {}
-        @primary_key_sequences = {}
         register_default_conversion_procs
         @duckdb_mutex = Mutex.new
         @duckdb_file = duckdb_file_path(opts[:database])
-        @duckdb_settings = InstanceRegistry.normalize_settings(opts[:config])
+        @duckdb_settings = InstanceRegistry.normalize_settings(duckdb_config)
         @duckdb_state = ConnectionState.new(open_duckdb_database, 0)
       rescue Sequel::DatabaseConnectionError
         raise
@@ -96,6 +94,19 @@ module Sequel
       end
 
       # nil for an in-memory database, which is never shared.
+      # The DuckDB settings: the config: option, plus access_mode READ_ONLY when the
+      # readonly option (readonly: true, or ?readonly=true in the URL) is true.
+      def duckdb_config
+        config = opts[:config] || {}
+        return config unless typecast_value_boolean(opts[:readonly])
+
+        mode = config.find { |name, _| name.to_s == "access_mode" }&.last
+        return config.merge(access_mode: "READ_ONLY") if mode.nil?
+        return config if mode.to_s.casecmp?("READ_ONLY")
+
+        raise Sequel::DatabaseConnectionError, "readonly: true conflicts with config access_mode #{mode.inspect}"
+      end
+
       def duckdb_file_path(database_path)
         return nil if database_path.nil? || database_path == ":memory:"
         return "/#{database_path}" if database_path.match?(/^[a-zA-Z]/) && !database_path.start_with?(":")
@@ -199,6 +210,11 @@ module Sequel
 
       private
 
+      # DuckDB 2.0 made binder and catalog errors abort an open transaction.
+      def bind_errors_abort_transactions?
+        ::DuckDB::LIBRARY_VERSION.to_i >= 2
+      end
+
       def database_error_classes
         [ ::DuckDB::Error ]
       end

data/lib/sequel/adapters/shared/duckdb.rb

@@ -294,19 +294,21 @@ module Sequel
       #
       # DuckDB resolves the name itself, as it does for any query, so every Sequel
       # identifier form, the search_path, temporary tables and views all work. The query
-      # reads no rows. A missing table is a catalog error, which does not abort an open
-      # DuckDB transaction, so no savepoint is needed.
+      # reads no rows. A missing table is a catalog error. It must not abort an open
+      # transaction, and DuckDB has no savepoints, so see keep_transaction_on_bind_error.
       #
       # @param table_name [Symbol, String, Sequel::SQL::Identifier, Sequel::SQL::QualifiedIdentifier] Name of the table
       # @param opts [Hash] Options (:schema qualifies an unqualified name)
       # @return [Boolean] true if table exists
       def table_exists?(table_name, opts = {})
-        from(qualify_with_schema_option(table_name, opts)).where(false).select(Sequel::NULL).all
-        true
-      rescue Sequel::DatabaseConnectionError, Sequel::DatabaseDisconnectError
-        raise
-      rescue Sequel::DatabaseError
-        false
+        keep_transaction_on_bind_error do
+          from(qualify_with_schema_option(table_name, opts)).where(false).select(Sequel::NULL).all
+          true
+        rescue Sequel::DatabaseConnectionError, Sequel::DatabaseDisconnectError
+          raise
+        rescue Sequel::DatabaseError
+          false
+        end
       end
 
       # Get list of tables
@@ -474,16 +476,18 @@ module Sequel
       # @return [Hash, nil] :catalog, :schema and :table, or nil if the name does not resolve
       def resolve_table(table_name, opts = {})
         table_name = qualify_with_schema_option(table_name, opts)
-        target = explain_plan(table_name)
-        return unless target
-
-        candidates = fetch("SELECT table_catalog, table_schema, table_name FROM information_schema.tables " \
-                           "WHERE lower(table_name) = lower(?)", bare_table_name(table_name)).all
-        matches = candidates.select do |c|
-          explain_plan(qualified_name(c[:table_catalog], c[:table_schema], c[:table_name])) == target
+        keep_transaction_on_bind_error do
+          target = explain_plan(table_name)
+          next unless target
+
+          candidates = fetch("SELECT table_catalog, table_schema, table_name FROM information_schema.tables " \
+                             "WHERE lower(table_name) = lower(?)", bare_table_name(table_name)).all
+          matches = candidates.select do |c|
+            explain_plan(qualified_name(c[:table_catalog], c[:table_schema], c[:table_name])) == target
+          end
+          match = matches.size > 1 ? first_in_search_order(matches) : matches.first
+          match && { catalog: match[:table_catalog], schema: match[:table_schema], table: match[:table_name] }
         end
-        match = matches.size > 1 ? first_in_search_order(matches) : matches.first
-        match && { catalog: match[:table_catalog], schema: match[:table_schema], table: match[:table_name] }
       end
 
       # resolve_table, but a name that does not resolve raises.
@@ -514,7 +518,7 @@ module Sequel
       end
 
       # The EXPLAIN plan of SELECT * FROM the name, or nil if DuckDB cannot bind the name.
-      # A bind error does not abort an open DuckDB transaction.
+      # Call it inside keep_transaction_on_bind_error.
       def explain_plan(table_name)
         plan = []
         execute("EXPLAIN (FORMAT JSON) #{from(table_name).sql}") { |row| plan << row }
@@ -525,6 +529,32 @@ module Sequel
         nil
       end
 
+      # Runs the block so that a binder or catalog error in it, such as a missing table,
+      # does not abort an open transaction.
+      #
+      # DuckDB 1.x never aborts a transaction on those errors. DuckDB 2.x aborts it by
+      # default, and its current_transaction_invalidation_policy setting turns that off
+      # for the open transaction only. DuckDB 1.x does not know the value, and a failed
+      # SET aborts the transaction, so the setting is used only where
+      # bind_errors_abort_transactions? is true. The block must handle its own errors:
+      # if it raises, the policy is not restored, and the transaction ends anyway.
+      def keep_transaction_on_bind_error
+        return yield unless bind_errors_abort_transactions? && in_transaction?
+
+        setting = "current_transaction_invalidation_policy"
+        previous = get(Sequel.function(:current_setting, setting))
+        run("SET #{setting} = 'SYNTACTIC_ERRORS_DO_NOT_INVALIDATE'")
+        result = yield
+        run("SET #{setting} = #{literal(previous)}")
+        result
+      end
+
+      # True when a binder or catalog error aborts an open transaction. The native adapter
+      # answers from the loaded DuckDB library. Other adapters run no queries, so false.
+      def bind_errors_abort_transactions?
+        false
+      end
+
       # DuckDB looks in the temp catalog first, then in the search_path schemas, then in the
       # default catalog.
       def first_in_search_order(candidates)
@@ -758,15 +788,11 @@ module Sequel
       # Remove all cached schema information
       #
       # Called after schema modifications to ensure cache consistency. It empties the
-      # caches in place, because a frozen Database cannot replace them.
+      # cache in place, because a frozen Database cannot replace it.
       #
       # @return [void]
       def remove_all_cached_schemas
-        Sequel.synchronize do
-          @schemas&.clear
-          @primary_keys.clear
-          @primary_key_sequences.clear
-        end
+        Sequel.synchronize { @schemas&.clear }
       end
 
       # List all schemas in the database
@@ -864,6 +890,43 @@ module Sequel
         sql
       end
 
+      # Runs a DuckDB ATTACH, which opens the database file at +path+ as the catalog
+      # +as+. The catalog is part of the DuckDB database, so every pool connection sees
+      # it. It lasts until Database#disconnect closes the last connection; attach again
+      # after that. See attach_sql for the options.
+      def attach(path, as:, **options)
+        run(attach_sql(path, as:, **options))
+      end
+
+      # The ATTACH statement that #attach runs. The alias +as+ is required: an alias
+      # that DuckDB derives from the file name can equal a schema name in the file,
+      # which makes schema.table ambiguous.
+      #
+      # @param path [String] The database file; a string literal in the SQL
+      # @param as [Symbol, String] The catalog name
+      # @param if_not_exists [Boolean] Do nothing if the catalog is already attached
+      # @param options [Hash] DuckDB ATTACH options, for example read_only: true or
+      #   type: :sqlite. A true value renders the bare keyword (READ_ONLY).
+      # @return [String]
+      def attach_sql(path, as:, if_not_exists: false, **options)
+        sql = +"ATTACH "
+        sql << "IF NOT EXISTS " if if_not_exists
+        sql << "#{literal(path.to_s)} AS #{Sequel::DuckDB::Helpers::SqlOptions.quoted_identifier(as)}"
+        list = Sequel::DuckDB::Helpers::SqlOptions.list(options)
+        sql << " #{list}" unless list.empty?
+        sql
+      end
+
+      # Runs a DuckDB DETACH of the catalog +name+.
+      def detach(name, if_exists: false)
+        run(detach_sql(name, if_exists:))
+      end
+
+      # The DETACH statement that #detach runs.
+      def detach_sql(name, if_exists: false)
+        "DETACH DATABASE #{"IF EXISTS " if if_exists}#{Sequel::DuckDB::Helpers::SqlOptions.quoted_identifier(name)}"
+      end
+
       # Runs a DuckDB COPY that writes +source+ (a Dataset or a SQL string) to
       # +path+. For a Dataset this is the SQL from Dataset#copy_to_sql.
       def copy_to(source, path, options = Sequel::OPTS)

data/lib/sequel/duckdb/helpers/copier.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require_relative "sql_options"
+
 module Sequel
   module DuckDB
     module Helpers
@@ -9,9 +11,6 @@ module Sequel
       # destination's file extension. A destination with neither (for example a
       # directory for a partitioned COPY) raises Sequel::Error.
       class Copier
-        # Option values that DuckDB accepts as a bare keyword (FORMAT PARQUET).
-        BARE_WORD = /\A[A-Za-z_][A-Za-z0-9_]*\z/
-
         def initialize(src, dst, options = Sequel::OPTS)
           @src = src
           @dst = dst
@@ -31,21 +30,11 @@ module Sequel
         end
 
         def options_str
-          opts_str = { format: format }.merge(@options).map { |k, v| format_option(k, v) }.compact.join(", ").strip
-          opts_str.empty? ? "" : "(#{opts_str})"
+          SqlOptions.list({ format: format }.merge(@options))
         end
 
         def format_option(key, value)
-          return "PARTITION_BY (#{Array(value).map { |name| quoted_identifier(name) }.join(", ")})" if key.to_s == "partition_by"
-
-          key = key.to_s.upcase
-          case value
-          when true then key
-          when false then nil
-          when Integer, Float then "#{key} #{value}"
-          when BARE_WORD then "#{key} #{value.to_s.upcase}"
-          else "#{key} #{string_literal(value)}"
-          end
+          SqlOptions.option(key, value)
         end
 
         def format
@@ -61,11 +50,7 @@ module Sequel
         private
 
         def string_literal(value)
-          "'#{value.to_s.gsub("'", "''")}'"
-        end
-
-        def quoted_identifier(name)
-          "\"#{name.to_s.gsub('"', '""')}\""
+          SqlOptions.string_literal(value)
         end
       end
     end

data/lib/sequel/duckdb/helpers/sql_options.rb

@@ -0,0 +1,44 @@
+# frozen_string_literal: true
+
+module Sequel
+  module DuckDB
+    module Helpers
+      # Renders the option list of a DuckDB statement, as in COPY ... (FORMAT PARQUET)
+      # and ATTACH ... (READ_ONLY, TYPE SQLITE).
+      module SqlOptions
+        # Option values that DuckDB accepts as a bare keyword (FORMAT PARQUET).
+        BARE_WORD = /\A[A-Za-z_][A-Za-z0-9_]*\z/
+
+        module_function
+
+        # "(KEY value, ...)", or "" when no option renders. A true value renders the key
+        # alone, and a false value leaves the option out.
+        def list(options)
+          rendered = options.map { |k, v| option(k, v) }.compact
+          rendered.empty? ? "" : "(#{rendered.join(", ")})"
+        end
+
+        def option(key, value)
+          return "PARTITION_BY (#{Array(value).map { |name| quoted_identifier(name) }.join(", ")})" if key.to_s == "partition_by"
+
+          key = key.to_s.upcase
+          case value
+          when true then key
+          when false then nil
+          when Integer, Float then "#{key} #{value}"
+          when BARE_WORD then "#{key} #{value.to_s.upcase}"
+          else "#{key} #{string_literal(value)}"
+          end
+        end
+
+        def string_literal(value)
+          "'#{value.to_s.gsub("'", "''")}'"
+        end
+
+        def quoted_identifier(name)
+          "\"#{name.to_s.gsub('"', '""')}\""
+        end
+      end
+    end
+  end
+end

data/lib/sequel/duckdb/version.rb

@@ -11,6 +11,6 @@ module Sequel
     #   puts Sequel::DuckDB::VERSION
     #
     # @since 0.1.0
-    VERSION = "0.4.0"
+    VERSION = "0.4.1"
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: sequel-duckdb
 version: !ruby/object:Gem::Version
-  version: 0.4.0
+  version: 0.4.1
 platform: ruby
 authors:
 - Ryan Duryea
@@ -119,42 +119,42 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.40.0
+        version: '0.41'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.40.0
+        version: '0.41'
 - !ruby/object:Gem::Dependency
   name: rubocop-rake
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.7.1
+        version: '0.7'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.7.1
+        version: '0.7'
 - !ruby/object:Gem::Dependency
   name: rubocop-sequel
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.4.1
+        version: '0.4'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.4.1
+        version: '0.4'
 - !ruby/object:Gem::Dependency
   name: simplecov
   requirement: !ruby/object:Gem::Requirement
@@ -186,6 +186,7 @@ files:
 - lib/sequel/duckdb.rb
 - lib/sequel/duckdb/helpers/copier.rb
 - lib/sequel/duckdb/helpers/pathifier.rb
+- lib/sequel/duckdb/helpers/sql_options.rb
 - lib/sequel/duckdb/instance_registry.rb
 - lib/sequel/duckdb/sql_time_tz.rb
 - lib/sequel/duckdb/version.rb