sequel-duckdb 0.3.1 → 0.4.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d1a28241665b1678903381a6230815dc1d8bc03b54f0778dfa86ee1042f04edb
-  data.tar.gz: a855f1bcac746a4f495ac1d003f015da38afac13f0663675c65d2df704baef66
+  metadata.gz: 7d728d2d4748d8ace682454b06155cea6ae570a8fbfbf888efa7d2a314077e5c
+  data.tar.gz: 525588b5baceefd4e7b0e3e61f429efb1c505d03250be98ca109bb9567bfce63
 SHA512:
-  metadata.gz: c26aef724f16dd8f29b396f2e63338fbcfd72ab9d739614638038e46a41e2ea051dbca9bd6a6cbb2d935d1aec99853651019cc8b803958eedc63c0cc7ccdb984
-  data.tar.gz: fa3cc4c38ae41d572095bb09ccd16547519645f935beae228866591180997e7846cbd274817c486bc95d1c5edbbe333ce592e1926322e0366f18020d1db98a8a
+  metadata.gz: 63164b64d30f71edcb66edb7e3a01ed07074de9312673b51b823e264268f702609fd6fcf98481535c7fc9165f49fa6eb38aa28adfd4311f129ef10d3b9a6bb8b
+  data.tar.gz: 76f6fb5c6e9d2684a013176f3ff7b9414ba6e0a9e67e88d36312e5cb57df5ddda4805ad84f4506ce7c60576461748194e24c2727596493b735eb434bec00cfc7

data/CHANGELOG.md

@@ -5,6 +5,185 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.4.1] - 2026-10-26
+
+### Added
+
+- Support for the DuckDB C library 2.x and the duckdb gem 2.x, alongside 1.5.x. CI
+  tests library 1.5.6 and 2.0.0, and the duckdb gem 1.5.x and 2.x. Support for library
+  1.5.x ends when DuckDB 1.5.x reaches end of life in March 2027. sequel-duckdb still
+  does not depend on the duckdb gem, so each project picks its own version.
+- **database:** Add attach and detach for DuckDB catalogs ([f82169b](https://github.com/outcomesinsights/sequel-duckdb/commit/f82169b0ad4cd66abe588dfdad431053535151e1))
+
+### Changed
+
+- **adapter:** Remove the unused primary key caches ([9a14254](https://github.com/outcomesinsights/sequel-duckdb/commit/9a14254abd8494e71af9e5478d081ff048b46701))
+
+### Fixed
+
+- **connection:** Honor the readonly option ([e8e94c5](https://github.com/outcomesinsights/sequel-duckdb/commit/e8e94c5a5a210aacac0a53cc8190f3dc76b3bbdf))
+- **schema:** Keep the transaction when a name lookup misses on DuckDB 2.x ([b3c02c6](https://github.com/outcomesinsights/sequel-duckdb/commit/b3c02c6029095e11fb048ef2b3c0e84a6c62e3ec))
+- **schema:** Let a mock Database drop and create schemas ([f9aad74](https://github.com/outcomesinsights/sequel-duckdb/commit/f9aad74f94fc9d4ed024accd9ccb4bc6ecdb628e))
+
+## [0.4.0] - 2026-10-25
+
+### Breaking changes
+
+- **adapter:** TIME and TIME_NS result values are now `Sequel::SQLTime`, not
+  `Time`. Sequel's postgres and sqlite adapters use the same class for a time of day. The
+  value keeps the hour, minute, second and fraction, and TIME_NS keeps its nanoseconds. A
+  value assigned to a TIME model column is also a `Sequel::SQLTime` on today's date. It
+  was a `Time` on 1970-01-01 before. So a model that is assigned the time it already has
+  no longer reports the column as changed. A `Sequel::SQLTime` in a query is written as
+  `'HH:MM:SS.ffffff'`, so writes keep the fraction. `Sequel::SQLTime` is a subclass of
+  `Time`, so `is_a?(Time)` is still true. Code that compares the class exactly with
+  `Time`, or that expects the date 1970-01-01, must change.
+- **adapter:** TIMETZ columns keep their UTC offset. `db.schema` reports
+  them as `:time_timezone`, not `:time`. Result values are the new class
+  `Sequel::DuckDB::SQLTimeTZ`, a subclass of `Sequel::SQLTime` on today's date that
+  keeps the offset. A value assigned to a TIMETZ model column becomes the same class,
+  so assigning a row its own value no longer reports the column as changed. A
+  `SQLTimeTZ` in a query is written as `'HH:MM:SS.ffffff+HH:MM'`. Before, a model wrote
+  a TIMETZ value without its offset, so DuckDB applied the session time zone:
+  08:09:10.5+02:00 was stored as 08:09:10.5-07:00. Two values with the same instant
+  and different offsets are not equal. A plain `Time` in a dataset query is still
+  written as a timestamp without an offset.
+
+### Changed
+
+- **copy:** `Database#copy_to` and the new `Dataset#copy_to_sql` write partition columns
+  as quoted identifiers. `partition_by: "year"` gave `PARTITION_BY YEAR` and now gives
+  `PARTITION_BY ("year")`. DuckDB runs both forms, so the result does not change. Tests
+  that compare COPY strings must update their expected strings. Word values are
+  unchanged, for example `FORMAT PARQUET` and `COMPRESSION ZSTD`. Values that are not a
+  single word, such as `delimiter: "|"`, are now quoted strings. The old text for those
+  values was a DuckDB syntax error.
+- **adapter:** a `Time` dated 1970-01-01 in a query is now written as a timestamp, for
+  example `'1970-01-01 09:30:00'`, like any other `Time`. It was written as the time of
+  day `'09:30:00'` before. DuckDB casts both forms to the same TIME value, so a
+  comparison with a TIME column gives the same result. Use `Sequel::SQLTime` for a time
+  of day.
+- **adapter:** Remove the typecast_value override and its schema cache ([f75e08b](https://github.com/outcomesinsights/sequel-duckdb/commit/f75e08b4a507f9a933d60d72cdf80ae44be23d26))
+
+### Added
+
+- **capabilities:** the adapter now declares Sequel's flags for DISTINCT ON, GROUP BY
+  ROLLUP, CUBE and GROUPING SETS, LATERAL subqueries, MERGE, the WINDOW clause, regular
+  expressions, and `CREATE TABLE IF NOT EXISTS` / `DROP TABLE IF EXISTS`. Sequel refused
+  most of these features before, or left the WINDOW clause out of the SQL. Negated
+  regexps (`exclude(name: /x/)`) are now written as `NOT regexp_matches(...)`.
+  `create_table?` and `drop_table?` now emit `IF NOT EXISTS` / `IF EXISTS`. Other SQL is
+  unchanged.
+- **adapter:** `Database#add_conversion_proc(type, callable = nil, &block)` registers a
+  converter for result values of one DuckDB column type. Dataset fetches pass each
+  non-NULL value of a column of that type to the converter. The converter receives the
+  whole value, so a LIST of STRUCT arrives as one Array. `type` is the type name that
+  ruby-duckdb reports, for example `:time` or `:list`. A column with a type alias is
+  looked up by its alias first. JSON columns report the type `:varchar` with the alias
+  `JSON`, so you register them as `:json`. A column type without a converter keeps the
+  value that ruby-duckdb returns. The adapter registers converters for `:time`,
+  `:time_ns` and `:time_tz`, and a registration of your own replaces them.
+- **extensions:** the opt-in `duckdb_array` extension writes and reads DuckDB LIST and
+  ARRAY columns, as Sequel's `pg_array` does for PostgreSQL. `Sequel.duckdb_array([1, 2])`
+  returns a `Sequel::DuckDB::DuckDBArray`, which Sequel writes as the list `[1, 2]`. A
+  plain Ruby Array still means an IN list or a condition. Load the extension with
+  `DB.extension :duckdb_array`. LIST and ARRAY result values are then `DuckDBArray`
+  objects. A value nested inside another list, struct or map is not wrapped. `db.schema`
+  reports an `INTEGER[]` or `INTEGER[3]` column as `:integer_array`. A list of an element
+  type that has no Sequel type, for example a STRUCT, is reported as `:array`. A model
+  typecasts an assigned Array to a `DuckDBArray` and typecasts each element to the
+  element type. Without the extension, the SQL and the values do not change.
+- **extensions:** the opt-in `duckdb_struct` extension writes and reads DuckDB STRUCT
+  columns. Sequel's `pg_row` is the PostgreSQL counterpart. DuckDB structs have no type
+  name, so you do not register a type. `Sequel.duckdb_struct(a: 1, b: "x")` returns a
+  `Sequel::DuckDB::DuckDBStruct` with Symbol keys. Sequel writes it as the struct
+  `{'a': 1, 'b': 'x'}`. A plain Ruby Hash still means a condition. A struct or list
+  inside a struct is written only when it is a wrapper too. Load the extension with
+  `DB.extension :duckdb_struct`. STRUCT result values are then `DuckDBStruct` objects.
+  `db.schema` reports a STRUCT column as `:struct`. A model typecasts an assigned Hash to
+  a `DuckDBStruct`. With `duckdb_array` also loaded, a list of structs is reported as
+  `:struct_array`. Without the extension, the SQL and the values do not change.
+- **extensions:** the opt-in `duckdb_map` extension writes and reads DuckDB MAP columns.
+  Sequel's `pg_hstore` is the PostgreSQL counterpart, but a DuckDB MAP has typed keys
+  and values. `Sequel.duckdb_map("k" => 1)` returns a `Sequel::DuckDB::DuckDBMap`, which
+  Sequel writes as the map `MAP {'k': 1}`. An empty Hash gives `MAP {}`. The keys keep
+  their Ruby type, so an INTEGER-keyed map has Integer keys. A plain Ruby Hash still
+  means a condition. Load the extension with `DB.extension :duckdb_map`. MAP result
+  values are then `DuckDBMap` objects. `db.schema` reports a MAP column as `:map`. A
+  model typecasts an assigned Hash to a `DuckDBMap`. With `duckdb_array` also loaded, a
+  list of maps is reported as `:map_array`. Without the extension, the SQL and the
+  values do not change.
+- **extensions:** the opt-in `duckdb_json` extension writes and reads DuckDB JSON
+  columns, as Sequel's `pg_json` does for PostgreSQL. `Sequel.duckdb_json("a" => 1)`
+  returns a `Sequel::DuckDB::DuckDBJSONHash`, and an Array gives a `DuckDBJSONArray`.
+  Sequel writes each one as JSON text cast to JSON, for example `'{"a":1}'::JSON`. A
+  plain Ruby Hash or Array still means a condition or an IN list. Load the extension
+  with `DB.extension :duckdb_json`. JSON result values are then parsed. A JSON object
+  becomes a `DuckDBJSONHash` with String keys, and a JSON array becomes a
+  `DuckDBJSONArray`. A JSON string, number, boolean or null becomes the plain Ruby
+  value. A VARCHAR column is not parsed, even when it holds JSON text. `db.schema`
+  reports a JSON column as `:json`. A model typecasts an assigned Hash or Array to the
+  wrapper and parses an assigned String as JSON. With `duckdb_array` also loaded, a
+  JSON list is reported as `:json_array`. Without the extension, the SQL and the values
+  do not change.
+- **duckdb_ops:** Add opt-in duckdb_ops extension with Sequel.try_cast ([a8c0815](https://github.com/outcomesinsights/sequel-duckdb/commit/a8c081531c2bb071bb1a8d07752254346e39f32f))
+- **duckdb_ops:** Add named-argument builder and read\_\* helpers ([5409d10](https://github.com/outcomesinsights/sequel-duckdb/commit/5409d10f508452dd3b945b65bf0417767296a9f0))
+- **duckdb_ops:** Add Sequel.duckdb_lambda for DuckDB lambda expressions ([5de2362](https://github.com/outcomesinsights/sequel-duckdb/commit/5de2362580467f518de0c5ba3fe6253d655633a5))
+- **copy:** Add Dataset#copy_to_sql; Database#copy_to runs it ([a972d8f](https://github.com/outcomesinsights/sequel-duckdb/commit/a972d8f66ec6a3713f5c7fa612e0172f284d9332))
+- **dataset:** Declare supports_is_distinct_from? so IS DISTINCT FROM is native ([d8a1397](https://github.com/outcomesinsights/sequel-duckdb/commit/d8a13978a9c12e895d28bcf9fa2aea69006e7c2f))
+
+### Fixed
+
+- **dataset:** a CTE now works in INSERT, UPDATE and DELETE. `supports_cte?` took no
+  argument, so an INSERT from a dataset with a CTE raised `ArgumentError`. An UPDATE or a
+  DELETE on a dataset with a CTE left the WITH clause out of the SQL, so DuckDB could not
+  find the CTE's table. All three statements now start with the WITH clause.
+- **connection:** a frozen `Sequel::Database` can open and close connections and drop a
+  schema. Sequel recommends `Database#freeze` for production, but every disconnect and
+  every `drop_schema` raised `FrozenError` after it, and later queries failed.
+- **connection:** the `config:` option was documented but ignored. Its DuckDB settings,
+  for example `config: { threads: 4, memory_limit: "2GB" }`, now reach DuckDB. A file that
+  is already open in this process keeps the settings of the first `Sequel::Database` that
+  opened it. Opening it with different settings raises `Sequel::DatabaseConnectionError`,
+  as DuckDB's Python, Java and R clients do. No `config:` and some `config:` count as
+  different.
+- **schema:** `table_exists?` returned false for some tables that exist: a
+  `Sequel.identifier` name, an unqualified name on the `search_path`, a table in an
+  attached catalog, and a `Sequel.identifier` name with the `schema:` option. DuckDB now
+  resolves the name, as it does for a query. `schema` and `indexes` raised "does not
+  exist" for the same names. They now read the table or view that a query with that
+  name reads, including a temporary table that has the name of a `main` table.
+- **schema:** `db.schema` reported type `:string` for FLOAT and REAL, unsigned and
+  128-bit integer, VARINT, TIMESTAMPTZ, TIMESTAMP_S/MS/NS, TIME_NS and INTERVAL
+  columns. Sequel models then typecast an assigned value to a String, for example 1.5 to
+  "1.5" for a FLOAT column. These columns now report `:float`, `:integer`, `:bigint`,
+  `:datetime`, `:time` or `:interval`. List, struct, map, union, enum, JSON and BIT
+  columns still report `:string`.
+- **schema:** a schema, table or catalog name with a single quote in it broke
+  `tables`, `views`, `table_exists?`, `schema`, `indexes`, `schemas` and
+  `schema_exists?` with a syntax error. `set_pragma` had the same defect for a value that
+  is not a String, Boolean or Number.
+- **date_arithmetic:** a query that held both `Sequel.date_add(d, days: 30)` and
+  `Sequel.date_add(d, months: 1)` returned the same date for both. DuckDB judges
+  `INTERVAL 30 DAY` equal to `INTERVAL 1 MONTH`, and its optimizer computed only one of
+  the two expressions. The same happened with 360 days and 1 year, and inside aggregates.
+  Year and month intervals are now written as a subtraction of the negated value:
+  `months: 1` gives `"d" - INTERVAL (-1) MONTH`, and `months: -1` gives
+  `"d" - INTERVAL 1 MONTH`. The same merge happened between days and hours of equal
+  length, for example 1 day and 24 hours, which differ for a TIMESTAMPTZ across a
+  daylight-saving change. Hour, minute and second intervals are now written with the
+  interval first: `hours: 24` gives `INTERVAL 24 HOUR + "d"`. Day intervals are
+  unchanged. Tests that compare date arithmetic SQL strings must update their expected
+  strings for every unit except days.
+- **connection:** two `Sequel::Database` objects on one DuckDB file in one process opened
+  two DuckDB instances, and writes through one did not reliably reach the other. Every
+  `Sequel::Database` on a file now shares one DuckDB instance. `Database#disconnect` now
+  gives the file back once that database has no open connection. The file closes when no
+  `Sequel::Database` in the process uses it, and the next query opens it again.
+  In-memory databases are unchanged: each is separate and stays open after `disconnect`.
+- **dataset:** Declare supports_filtered_aggregates? so FILTER is native ([c7aad9c](https://github.com/outcomesinsights/sequel-duckdb/commit/c7aad9c9cfad8eaf28c0825004a23c2a6e52bc31))
+- **copy:** Escape paths, quote COPY options, infer-or-raise FORMAT ([13efc23](https://github.com/outcomesinsights/sequel-duckdb/commit/13efc239c0d491b6302af6207d19ab86a022f160))
+
 ## [0.3.1] - 2026-10-02
 
 ### Fixed
@@ -21,12 +200,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 - Ship only lib/ and three docs; stop tracking the beads key ([c45dfc4](https://github.com/outcomesinsights/sequel-duckdb/commit/c45dfc4636917269f6333da1be2b21ed8855a0c2))
 
-## [0.2.1](https://github.com/outcomesinsights/sequel-duckdb/compare/v0.2.0...v0.2.1) (2026-10-01)
+## [0.2.1] - 2026-10-01
 
 No changes to the gem's behaviour. The first release published by the
 tag-based release workflow.
 
-## [0.2.0](https://github.com/outcomesinsights/sequel-duckdb/compare/v0.1.0...v0.2.0) (2026-09-30)
+## [0.2.0] - 2026-09-30
 
 ### ⚠ BREAKING CHANGES
 
@@ -182,3 +361,11 @@ tag-based release workflow.
 - Connection string sanitization
 - File path validation for database files
 - Read-only database connection support
+
+[0.1.0]: https://github.com/outcomesinsights/sequel-duckdb/releases/tag/v0.1.0
+[0.2.0]: https://github.com/outcomesinsights/sequel-duckdb/compare/v0.1.0...v0.2.0
+[0.2.1]: https://github.com/outcomesinsights/sequel-duckdb/compare/v0.2.0...v0.2.1
+[0.3.0]: https://github.com/outcomesinsights/sequel-duckdb/compare/v0.2.1...v0.3.0
+[0.3.1]: https://github.com/outcomesinsights/sequel-duckdb/compare/v0.3.0...v0.3.1
+[0.4.0]: https://github.com/outcomesinsights/sequel-duckdb/compare/v0.3.1...v0.4.0
+[0.4.1]: https://github.com/outcomesinsights/sequel-duckdb/compare/v0.4.0...v0.4.1

data/README.md

@@ -286,6 +286,12 @@ db = Sequel.connect('duckdb://relative/path/to/database.duckdb')
 db = Sequel.connect('duckdb:///path/to/database.duckdb?readonly=true')
 ```
 
+`readonly` opens the file with DuckDB's `access_mode` set to `READ_ONLY`, so writes
+raise `Sequel::DatabaseError`. All `Sequel::Database` objects on one file in one
+process share one DuckDB instance, so the first one to open the file sets its
+settings. Opening the same file again with other settings, such as read-write after
+read-only, raises `Sequel::DatabaseConnectionError`.
+
 ### Connection Hash
 
 ```ruby
@@ -520,6 +526,159 @@ db[:users].multi_insert(users_data)
 db[:large_table].select(:id, :name, :created_at).where(active: true)
 ```
 
+### Exporting with COPY
+
+`Database#copy_to` runs a DuckDB `COPY` that writes a dataset (or a SQL string) to a file.
+FORMAT comes from the `format:` option, or else from the file extension. A path with
+neither, such as a directory for a partitioned export, raises `Sequel::Error`.
+
+```ruby
+db.copy_to(db[:users].where(active: true), "/data/users.parquet")
+db.copy_to(db[:events], "/data/events", format: :parquet, partition_by: [:year, :month])
+```
+
+`Dataset#copy_to_sql` returns the same `COPY` statement without running it, for example to
+write a plan file:
+
+```ruby
+db[:users].copy_to_sql("/data/users.csv", header: true, delimiter: "|")
+# => COPY (SELECT * FROM "users") TO '/data/users.csv' (FORMAT CSV, HEADER, DELIMITER '|')
+```
+
+### Attaching database files
+
+`Database#attach` runs a DuckDB `ATTACH`, and `Database#detach` runs `DETACH DATABASE`.
+The catalog name `as:` is required. A name that DuckDB derives from the file name can
+equal a schema name inside the file, and then `schema.table` is ambiguous. Other DuckDB
+`ATTACH` options pass through, for example `type: :sqlite`.
+
+```ruby
+db.attach("/data/dataset.duckdb", as: :dataset, read_only: true)
+db[Sequel[:dataset][:people]].count
+db.detach(:dataset, if_exists: true)
+
+db.attach_sql("/data/dataset.duckdb", as: :dataset, read_only: true, if_not_exists: true)
+# => ATTACH IF NOT EXISTS '/data/dataset.duckdb' AS "dataset" (READ_ONLY)
+```
+
+An attached catalog belongs to the DuckDB database, which all pool connections of a
+`Sequel::Database` share. It lasts until `Database#disconnect` closes the last
+connection. After that, attach it again; `if_not_exists: true` makes a repeat safe.
+
+`search_path` is different: DuckDB keeps it per connection. Set it on every new
+connection with Sequel's `connect_sqls` option. Put the writable catalog first, because
+`CREATE TABLE` without a schema goes to the first entry, and a read-only catalog there
+makes it fail.
+
+```ruby
+db = Sequel.connect("duckdb:///exports/42/work.duckdb",
+                    connect_sqls: [ "SET search_path = 'work.main'" ])
+db.attach("/data/dataset.duckdb", as: :dataset, read_only: true)
+db.create_table(:summary) { Integer :n } # created in work.main
+```
+
+### Lists
+
+Sequel writes a Ruby Array as a parenthesized list, such as `(1, 2, 3)`, for `IN`
+conditions. DuckDB does not read that as a list value. To build a DuckDB list from
+expressions, call DuckDB's `list_value` function:
+
+```ruby
+DB.literal([1, 2, 3])
+# => (1, 2, 3)
+DB.literal(Sequel.function(:list_value, :a, Sequel[:b] + 1))
+# => list_value("a", ("b" + 1))
+```
+
+### IS DISTINCT FROM
+
+The adapter declares that DuckDB supports `IS DISTINCT FROM`. Sequel's `is_distinct_from`
+extension then writes it natively. Load the extension on the Database. With only
+`Sequel.extension :is_distinct_from`, the builder exists, but the dataset cannot write it
+and raises `NoMethodError`. Write `IS NOT DISTINCT FROM` by negating the expression with
+`~`:
+
+```ruby
+DB.extension :is_distinct_from
+DB[:t].where(Sequel.is_distinct_from(:a, :b))
+# => SELECT * FROM "t" WHERE ("a" IS DISTINCT FROM "b")
+DB[:t].where(~Sequel.is_distinct_from(:a, :b))
+# => SELECT * FROM "t" WHERE NOT ("a" IS DISTINCT FROM "b")
+```
+
+### duckdb_ops extension
+
+The `duckdb_ops` extension adds DuckDB-specific expression builders to `Sequel`. It is
+opt-in. If you do not load it, Sequel generates the same SQL as before. Load it with
+either form. Both forms make the builders available globally.
+
+```ruby
+Sequel.extension :duckdb_ops
+# or
+DB.extension :duckdb_ops
+```
+
+`Sequel.try_cast(expr, type)` builds a DuckDB `TRY_CAST`. It returns NULL when the cast
+fails, where `CAST` raises an error. The type works as in `Sequel.cast`: a Ruby class
+maps to the adapter's type name, and a Symbol or String passes through unchanged.
+
+```ruby
+DB[:raw].select(Sequel.try_cast(:amount, :DOUBLE).as(:amount))
+# => SELECT TRY_CAST("amount" AS DOUBLE) AS "amount" FROM "raw"
+Sequel.try_cast(:amount, Float)
+# => TRY_CAST("amount" AS double precision)
+```
+
+DuckDB table functions take named parameters. `Sequel.duckdb_named(**params)` builds them
+for the last argument of `Sequel.function`. `Sequel.duckdb_read_parquet`,
+`Sequel.duckdb_read_csv` and `Sequel.duckdb_read_json` take a path or an array of paths,
+plus named parameters. An Array value becomes a DuckDB list. A Hash value becomes a
+struct, in which a Symbol is a bare type name and a String is a string literal.
+
+```ruby
+parts = Sequel.duckdb_read_parquet("/data/events/**/*.parquet",
+                                   hive_partitioning: true, hive_types: { year: :BIGINT })
+DB.from(parts.as(:events)).where(year: 2024)
+# => SELECT * FROM read_parquet(['/data/events/**/*.parquet'], hive_partitioning = TRUE,
+#      hive_types = {'year': BIGINT}) AS "events" WHERE ("year" = 2024)
+
+Sequel.function(:read_json, "/data/a.json", Sequel.duckdb_named(columns: { a: "INTEGER" }))
+# => read_json('/data/a.json', columns = {'a': 'INTEGER'})
+```
+
+#### Lambda expressions
+
+`Sequel.duckdb_lambda` builds a DuckDB lambda expression for list functions such as
+`list_transform`, `list_filter` and `list_reduce`. Call those functions through
+`Sequel.function`. The builder has two forms. In the block form, every argument is a
+parameter name, and the block gets one identifier for each parameter. In the body form,
+the first argument is a parameter name or an Array of names, and the second argument is
+the body. The body names the parameters as Symbols or with `Sequel[:x]`.
+
+```ruby
+Sequel.duckdb_lambda(:acc, :x) { |acc, x| acc + x }
+Sequel.duckdb_lambda([:acc, :x], Sequel[:acc] + :x)
+# => lambda "acc", "x": ("acc" + "x")
+
+Sequel.function(:list_reduce, :amounts,
+                Sequel.duckdb_lambda(:acc, :x) { |acc, x| acc + x },
+                Sequel.cast(0, :BIGINT))
+# => list_reduce("amounts", lambda "acc", "x": ("acc" + "x"), CAST(0 AS BIGINT))
+```
+
+DuckDB has no `let`. To compute a value once and use it more than once, wrap it in a
+one-element list, transform it, and take element 1 with `Sequel.subscript`:
+
+```ruby
+square_plus = Sequel.duckdb_lambda(:v) { |v| (v * v) + v }
+Sequel.subscript(Sequel.function(:list_transform, Sequel.function(:list_value, :n), square_plus), 1)
+# => (list_transform(list_value("n"), lambda "v": (("v" * "v") + "v")))[1]
+```
+
+The builder emits DuckDB's `lambda` keyword syntax, never the `x -> body` arrow, which
+DuckDB deprecated. It needs a DuckDB version that accepts the `lambda` keyword. Parameter
+names are quoted identifiers, the same as the references to them in the body.
+
 ## Model Integration
 
 ### Sequel::Model Usage
@@ -644,10 +803,17 @@ db[:users].where(active: true).all
 
 ### Version Compatibility
 
-- **Ruby**: 3.1.0 or higher
+- **Ruby**: 3.3 or higher (CI tests 3.3, 3.4 and 4.0)
 - **Sequel**: 5.0 or higher
-- **DuckDB**: 0.8.0 or higher
-- **ruby-duckdb**: 1.0.0 or higher
+- **DuckDB C library**: 1.5.x and 2.x. Support for 1.5.x ends when DuckDB 1.5.x
+  reaches end of life in March 2027.
+- **ruby-duckdb (the `duckdb` gem)**: 1.5.x and 2.x. sequel-duckdb does not depend on
+  it, so add `gem "duckdb"` to your own Gemfile. ruby-duckdb 2.x builds against
+  library 1.4 or newer, so it works with library 1.5.x.
+
+DuckDB 2.x aborts an open transaction after any failed statement, including a query
+on a missing table; 1.5.x does not. `table_exists?` and `schema` keep the transaction
+on both versions.
 
 ### Getting Help
 

data/lib/sequel/adapters/duckdb.rb

@@ -3,6 +3,7 @@
 require "sequel"
 require "duckdb"
 require_relative "shared/duckdb"
+require_relative "../duckdb/instance_registry"
 
 # Sequel is a database toolkit for Ruby that provides a powerful ORM and database abstraction layer.
 # This module extends Sequel to support DuckDB, a high-performance analytical database engine.
@@ -54,36 +55,122 @@ module Sequel
   # @since 0.1.0
   module DuckDB
     module DriverDatabaseMethods
+      # The open ::DuckDB::Database and the number of open connections to it. Connecting
+      # and disconnecting change both, so they live in this object and not in instance
+      # variables, which a frozen Sequel::Database cannot change.
+      ConnectionState = Struct.new(:database, :connections)
+
       private
 
       # Open the DuckDB::Database once during Sequel::Database initialization.
       # DuckDB's ATTACH/schema state is per-Database, not per-Connection, so all
-      # pool connections must share one Database instance.
+      # pool connections must share one Database instance. A database file is also
+      # shared with every other Sequel::Database on that file in this process
+      # (InstanceRegistry), and #disconnect gives it back once no connection is open.
       def adapter_initialize
-        database_path = opts[:database]
-
-        @duckdb_database = if database_path == ":memory:" || database_path.nil?
-                             ::DuckDB::Database.open(":memory:")
-        else
-                             if database_path.match?(/^[a-zA-Z]/) && !database_path.start_with?(":")
-                               database_path = "/#{database_path}"
-                             end
-                             ::DuckDB::Database.open(database_path)
-        end
+        @conversion_procs = {}
+        register_default_conversion_procs
+        @duckdb_mutex = Mutex.new
+        @duckdb_file = duckdb_file_path(opts[:database])
+        @duckdb_settings = InstanceRegistry.normalize_settings(duckdb_config)
+        @duckdb_state = ConnectionState.new(open_duckdb_database, 0)
+      rescue Sequel::DatabaseConnectionError
+        raise
       rescue ::DuckDB::Error => e
         raise Sequel::DatabaseConnectionError, "Failed to connect to DuckDB database: #{e.message}"
       rescue StandardError => e
         raise Sequel::DatabaseConnectionError, "Unexpected error connecting to DuckDB: #{e.message}"
       end
 
+      # ruby-duckdb returns TIME, TIME_NS and TIMETZ values as a Time on today's date.
+      # Sequel's convention for a time of day is Sequel::SQLTime, which is also what
+      # Sequel typecasts a value assigned to a TIME model column to. TIMETZ values become
+      # Sequel::DuckDB::SQLTimeTZ, which keeps the value's UTC offset.
+      def register_default_conversion_procs
+        to_sqltime = ->(value) { Sequel::SQLTime.create(value.hour, value.min, value.sec, Rational(value.nsec, 1000)) }
+        add_conversion_proc(:time, to_sqltime)
+        add_conversion_proc(:time_ns, to_sqltime)
+        add_conversion_proc(:time_tz, SQLTimeTZ.method(:from_time))
+      end
+
+      # nil for an in-memory database, which is never shared.
+      # The DuckDB settings: the config: option, plus access_mode READ_ONLY when the
+      # readonly option (readonly: true, or ?readonly=true in the URL) is true.
+      def duckdb_config
+        config = opts[:config] || {}
+        return config unless typecast_value_boolean(opts[:readonly])
+
+        mode = config.find { |name, _| name.to_s == "access_mode" }&.last
+        return config.merge(access_mode: "READ_ONLY") if mode.nil?
+        return config if mode.to_s.casecmp?("READ_ONLY")
+
+        raise Sequel::DatabaseConnectionError, "readonly: true conflicts with config access_mode #{mode.inspect}"
+      end
+
+      def duckdb_file_path(database_path)
+        return nil if database_path.nil? || database_path == ":memory:"
+        return "/#{database_path}" if database_path.match?(/^[a-zA-Z]/) && !database_path.start_with?(":")
+
+        database_path
+      end
+
+      # opts[:config] holds DuckDB settings, for example { threads: 4, memory_limit: "2GB" }.
+      def open_duckdb_database
+        if @duckdb_file
+          InstanceRegistry.acquire(@duckdb_file, @duckdb_settings)
+        else
+          InstanceRegistry.open(":memory:", @duckdb_settings)
+        end
+      end
+
       public
 
+      # Registers a converter for result values of one DuckDB column type. Dataset
+      # fetches pass each non-NULL value of a column of that type to the converter and
+      # return what it gives back. A converter receives the whole column value, so a
+      # LIST of STRUCT gives it the complete Array.
+      #
+      # +type+ is the type name that ruby-duckdb reports, for example :time or :list.
+      # A column with a type alias is looked up by its alias first: JSON columns report
+      # the type :varchar with the alias "JSON", so register them as :json. Register
+      # converters before you run queries. A later registration for the same type
+      # replaces the earlier one, including the built-in converters for :time and
+      # :time_ns, which return Sequel::SQLTime, and for :time_tz, which returns
+      # Sequel::DuckDB::SQLTimeTZ.
+      #
+      # @example
+      #   db.add_conversion_proc(:json) { |value| JSON.parse(value) }
+      def add_conversion_proc(type, callable = nil, &block)
+        @conversion_procs[conversion_proc_key(type)] = callable || block
+      end
+
       def connect(_server)
-        @duckdb_database.connect
+        @duckdb_mutex.synchronize do
+          state = @duckdb_state
+          state.database ||= open_duckdb_database
+          conn = state.database.connect
+          state.connections += 1
+          conn
+        end
       rescue ::DuckDB::Error => e
         raise Sequel::DatabaseConnectionError, "Failed to connect to DuckDB database: #{e.message}"
       end
 
+      # Closes the pool's idle connections. When no connection of this Sequel::Database
+      # is still open, it also gives back its reference to the shared database file, so
+      # the file closes once no Sequel::Database in the process uses it. The next query
+      # opens it again. An in-memory database stays open, so its data survives.
+      def disconnect(opts = OPTS)
+        super
+        @duckdb_mutex.synchronize do
+          state = @duckdb_state
+          next unless @duckdb_file && state.database && state.connections.zero?
+
+          InstanceRegistry.release(@duckdb_file)
+          state.database = nil
+        end
+      end
+
       def disconnect_connection(conn)
         return unless conn
 
@@ -91,6 +178,7 @@ module Sequel
           conn.close
         rescue ::DuckDB::Error # rubocop:disable Lint/SuppressedException -- best-effort cleanup on disconnect
         end
+        @duckdb_mutex.synchronize { @duckdb_state.connections -= 1 }
       end
 
       def valid_connection?(conn)
@@ -122,6 +210,11 @@ module Sequel
 
       private
 
+      # DuckDB 2.0 made binder and catalog errors abort an open transaction.
+      def bind_errors_abort_transactions?
+        ::DuckDB::LIBRARY_VERSION.to_i >= 2
+      end
+
       def database_error_classes
         [ ::DuckDB::Error ]
       end
@@ -164,6 +257,22 @@ module Sequel
           column.respond_to?(:name) ? column.name.to_s : column.to_s
         end
       end
+
+      def conversion_proc_key(type)
+        type.to_s.downcase.to_sym
+      end
+
+      # The converter for each result column, in column order, or nil for a column
+      # with none. Empty when nothing is registered, so rows stay exactly as
+      # ruby-duckdb returns them.
+      def result_conversion_procs(result)
+        return [] if @conversion_procs.empty?
+
+        result.columns.map do |column|
+          type_alias = column.logical_type.alias
+          (type_alias && @conversion_procs[conversion_proc_key(type_alias)]) || @conversion_procs[column.type]
+        end
+      end
     end
 
     module DriverDatasetMethods
@@ -173,11 +282,15 @@ module Sequel
 
           col_names = db.send(:result_column_names, result)
           self.columns = col_names.map { |c| output_identifier(c) }
+          converters = db.send(:result_conversion_procs, result)
 
           result.each do |row_array|
             row_hash = {}
             col_names.each_with_index do |col_name, i|
-              row_hash[output_identifier(col_name)] = row_array[i]
+              value = row_array[i]
+              converter = converters[i]
+              value = converter.call(value) if converter && !value.nil?
+              row_hash[output_identifier(col_name)] = value
             end
             yield row_hash
           end