sqlite3 2.9.5-x86-linux-gnu → 2.10.0-x86-linux-gnu

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e5083532df0ddbc68c040d18e416081700f0d82b34b148f01e99d2bedd7b6414
-  data.tar.gz: c4a460cb15d2f8f55879ef69a3664c1446035a3d195d07768181b42320d1a20c
+  metadata.gz: dcc1773a27e9a4a7868b50b5c466c6825efc1aef50630e1ffe048f3d00543645
+  data.tar.gz: d14d21b19660313063a229b22c1b14346efab6f7fb5824e53e6df59cb0e7a1ff
 SHA512:
-  metadata.gz: f36b1487d76089338cbe6ab46421b016080f5ac5c2eeed1c34c0e0622f4fe1c6c9e5a2632e7349af1320192cd54faf0e0f2c78845f810c38bf2a30814ddf8f9f
-  data.tar.gz: 5676d163011a6705b9b15369c4cf9227e624a71f049c7b35a0ad79eae5eb5c6ed967f9f36199fcc7a4c028040bf22572c7b553ae7e8d0e37999aa81e23df608b
+  metadata.gz: f6bd211b4d308eda12ba18a18d6ad5948545aa9e4722d4153fec3693ef85b7ddd37c14c741fd6eee519e23f4d268398469ec91a87404b9e134651ef4ec59c655
+  data.tar.gz: 97a661cc557f1be02a71fa942fd61e567401dc82bb6738311d4dee1b3dbc6a970811d662cefa901fc5651a0f9310b45909ecae3237c03b33fdec98bb1cd1a29a

data/CHANGELOG.md

@@ -1,10 +1,62 @@
 # sqlite3-ruby Changelog
 
+## 2.10.0 / 2026-10-13
+
+### Dependencies
+
+- Vendored sqlite is updated to [v3.54.0](https://www.sqlite.org/releaselog/3_54_0.html) (from v3.53.2). #750 @flavorjones
+
+  Notable changes in this version include:
+  - `ALTER TABLE` now raises an error when adding, renaming, or dropping a column named `ROWID`, `_ROWID_`, or `OID`.
+  - The authorizer callback is now invoked for SQL functions in the `DEFAULT` clauses of `CREATE TABLE` statements.
+  - SQLite no longer supports Windows XP or earlier. On Windows, the precompiled gem and any installation that compiles the vendored sqlite now require Windows Vista or later.
+
+### Added
+
+- `ResultSet#each` and `#each_hash` return an `Enumerator` when called without a block. Previously they raised `LocalJumpError`. #656 @kch
+- Introduce `Database#serialize`, which returns a database as a binary string, and `Database#deserialize`, which replaces a database with the contents of such a string. These wrap [`sqlite3_serialize()`](https://www.sqlite.org/c3ref/serialize.html) and [`sqlite3_deserialize()`](https://www.sqlite.org/c3ref/deserialize.html). Both methods are defined only when SQLite was compiled without `SQLITE_OMIT_DESERIALIZE`, and `Database#deserialize` also requires that the loaded SQLite is v3.54.0 or later. An image of a database in WAL mode cannot be queried after deserializing; see the `Database#deserialize` documentation. #685 @Maaarcocr
+
+### Fixed
+
+- Fix `Database#statement_timeout=` interrupting statements almost immediately instead of after the timeout elapsed, including re-executed prepared statements and calls to `Database#execute_batch2`. #737 @palkan @flavorjones
+
+### Improved
+
+- On CRuby, `Statement#types` allocates fewer strings. Each declared type is downcased and interned in C, instead of allocating two strings per column. #743 @jhawthorn
+
+### Changed
+
+- `ResultSet#each` and `#each_hash` return the result set when called with a block. As a consequence, `Database#execute` and `#execute2` called with a block return the closed `ResultSet` instead of `nil`. #656 @kch
+- `Statement#columns` and `Statement#types` return frozen arrays, and the strings in `Statement#types` are frozen. Code that mutates them now raises `FrozenError`. #743 @jhawthorn
+- `Statement#column_decltype` and `Statement#types` return UTF-8 strings. Previously they returned ASCII-8BIT strings. #743 @jhawthorn
+
+
+## 2.9.6 / 2026-08-11
+
+### Security / Stability
+
+- Fix a garbage collection bug where the argument array passed to a custom aggregate function's `step` was not visible to the GC, so arguments could be collected mid-conversion when the aggregate takes two or more arguments, corrupting the values passed to `step` or crashing the process. See [GHSA-mwm8-39rw-8826](https://github.com/sparklemotion/sqlite3-ruby/security/advisories/GHSA-mwm8-39rw-8826) for more information. #733 @jeremy
+
+### Fixed
+
+- Fix a leak where custom aggregate handler instances were never released, so a connection accumulated one instance per `GROUP BY` group per query for its lifetime. #722 @djmb
+- Fix GC compaction issues with custom functions, aggregates, collations, `#trace` and `#authorizer=`. These callbacks were registered with sqlite by passing a raw Ruby object pointer as user data; keeping the object reachable prevented collection but not relocation, after which sqlite held a stale address and the next call could raise `NoMethodError`, return a wrong result, or segfault. Affects applications that call `GC.compact` or run with `GC.auto_compact = true`. The equivalent issue in `#busy_handler` was fixed in #466. #723 @djmb
+- Fix the private methods `Database#open_v2` and `#open16` silently replacing a live connection and leaking the previous connection handle when invoked via `send` on an open database. They now raise `SQLite3::Exception`. #729 @flavorjones
+- Fix TEXT values containing an embedded NUL byte being truncated at the first NUL when passed as arguments to functions created with `Database#define_function`. #730 @flavorjones
+- Fix an exception raised inside a `Database#define_function` block leaving the connection's sqlite mutex held, which deadlocked any other thread that later used the connection. The exception now propagates to the caller and the connection remains usable. #731 @flavorjones
+- `Database.new` now raises `ArgumentError` when the filename or VFS name contains an embedded NUL byte (or an embedded 0x0000 code unit in a UTF-16 filename), instead of silently opening a path truncated at the NUL. #732 @flavorjones
+
+
+### Improved
+
+- When `Database.new` fails to open the database file, the underlying sqlite3 connection handle is now closed immediately instead of waiting for the garbage collector to clean it up. #719 @katafrakt
+
+
 ## 2.9.5 / 2026-06-07
 
 ### Dependencies
 
-- Vendored sqlite is updated to [v3.53.2](https://www.sqlite.org/releaselog/3_53_3.html) (from v3.53.1). #709 @flavorjones
+- Vendored sqlite is updated to [v3.53.2](https://www.sqlite.org/releaselog/3_53_2.html) (from v3.53.1). #709 @flavorjones
 
 ### Security / Stability
 

data/CONTRIBUTING.md

@@ -27,6 +27,71 @@ explicitly closed, it will be closed when it is GCed.
 will be closed/finalized when it is GCed.
 
 
+## Debugging memory issues
+
+Please install `valgrind` and `gdb` before you use the tools in this section.
+
+Run the test suite under valgrind and [`ruby_memcheck`](https://github.com/Shopify/ruby_memcheck) to
+look for memory leaks and other memory errors:
+
+``` sh
+bundle exec rake compile test:valgrind
+```
+
+If you can't install valgrind on your system, use the `sqlite3-dev` docker image, which contains
+valgrind:
+
+``` sh
+# build the image
+bundle exec rake docker:dev:build
+
+# run the test suite in a container
+bundle exec rake docker:dev:test
+
+# run the test suite under valgrind in a container
+bundle exec rake docker:dev:test:valgrind
+```
+
+Each `docker:dev:test` task builds the image first, then mounts your working copy at `/sqlite3` in
+the container. Note that the container compiles into the mounted working copy, so please re-run
+`rake compile` on your machine afterwards.
+
+Run the test suite in the debugger:
+
+``` sh
+bundle exec rake compile test:gdb
+```
+
+You can also run the test suite with a variety of GC behaviors, which is useful to localize some
+classes of memory bugs. Set the `SQLITE3_TEST_GC_LEVEL` environment variable (see `test/helper.rb`
+for more info). A more stressful level finds more bugs, but makes the suite slower:
+
+``` sh
+# ordinary GC behavior (the default)
+SQLITE3_TEST_GC_LEVEL=normal bundle exec rake compile test
+
+# minor GC after each test
+SQLITE3_TEST_GC_LEVEL=minor bundle exec rake compile test
+
+# major GC after each test
+SQLITE3_TEST_GC_LEVEL=major bundle exec rake compile test
+
+# major GC after each test, and GC compaction after every 20 tests
+SQLITE3_TEST_GC_LEVEL=compact bundle exec rake compile test
+
+# verify references after compaction, after every 20 tests
+# (see https://alanwu.space/post/check-compaction/)
+SQLITE3_TEST_GC_LEVEL=verify bundle exec rake compile test
+
+# run each test with GC "stress mode" on
+SQLITE3_TEST_GC_LEVEL=stress bundle exec rake compile test
+```
+
+The `compact` and `verify` levels fall back to `normal` on a platform that does not support GC
+compaction. The `stress` level makes the suite about 150 times slower, and it makes
+timing-sensitive tests unreliable.
+
+
 ## Building gems
 
 As a prerequisite please make sure you have `docker` correctly installed, so that you're able to cross-compile the native gems.

data/dependencies.yml

@@ -1,13 +1,13 @@
 sqlite3:
   # checksum verified by first checking the published sha3(256) checksum against https://sqlite.org/download.html:
-  # 025328da165109f48abccc6e7478508060804412bed2bd81d47e98ba1b72983b
+  # f5e3e724e9c9b3b5d512c76b02082091a59eeea9becd169a9fbf4d376403acce
   #
-  # $ sha3sum -a 256 ports/archives/sqlite-autoconf-3530200.tar.gz
-  # 025328da165109f48abccc6e7478508060804412bed2bd81d47e98ba1b72983b  ports/archives/sqlite-autoconf-3530200.tar.gz
+  # $ sha3sum -a 256 ports/archives/sqlite-autoconf-3540000.tar.gz
+  # f5e3e724e9c9b3b5d512c76b02082091a59eeea9becd169a9fbf4d376403acce  ports/archives/sqlite-autoconf-3540000.tar.gz
   #
-  # $ sha256sum ports/archives/sqlite-autoconf-3530200.tar.gz
-  # 588ad51949419a56ebe81fe56193d510c559eb94c9a57748387860b5d3069316  ports/archives/sqlite-autoconf-3530200.tar.gz
-  version: "3.53.2"
+  # $ sha256sum ports/archives/sqlite-autoconf-3540000.tar.gz
+  # 134ec0802dda5795816e25d25872d20b312cb3973438c49b30bc40b7705ea9ed  ports/archives/sqlite-autoconf-3540000.tar.gz
+  version: "3.54.0"
   files:
-    - url: "https://sqlite.org/2026/sqlite-autoconf-3530200.tar.gz"
-      sha256: "588ad51949419a56ebe81fe56193d510c559eb94c9a57748387860b5d3069316"
+    - url: "https://sqlite.org/2026/sqlite-autoconf-3540000.tar.gz"
+      sha256: "134ec0802dda5795816e25d25872d20b312cb3973438c49b30bc40b7705ea9ed"

data/ext/sqlite3/aggregator.c

@@ -97,7 +97,7 @@ rb_sqlite3_aggregate_instance_destroy(sqlite3_context *ctx)
     VALUE *inst_ptr = sqlite3_aggregate_context(ctx, 0);
     VALUE inst;
 
-    if (!inst_ptr || (inst = *inst_ptr)) {
+    if (!inst_ptr || !(inst = *inst_ptr)) {
         return;
     }
 
@@ -119,6 +119,7 @@ rb_sqlite3_aggregator_step(sqlite3_context *ctx, int argc, sqlite3_value **argv)
     VALUE inst = rb_sqlite3_aggregate_instance(ctx);
     VALUE handler_instance = rb_iv_get(inst, "-handler_instance");
     VALUE *params = NULL;
+    VALUE params_handle = 0;
     VALUE one_param;
     int exc_status = NUM2INT(rb_iv_get(inst, "-exc_status"));
     int i;
@@ -132,7 +133,9 @@ rb_sqlite3_aggregator_step(sqlite3_context *ctx, int argc, sqlite3_value **argv)
         params = &one_param;
     }
     if (argc > 1) {
-        params = xcalloc((size_t)argc, sizeof(VALUE));
+        /* ALLOCV memory is conservatively marked, so stored VALUEs survive a
+         * GC raised by a later sqlite3val2rb call. */
+        params = ALLOCV_N(VALUE, params_handle, argc);
         for (i = 0; i < argc; i++) {
             params[i] = sqlite3val2rb(argv[i]);
         }
@@ -140,7 +143,7 @@ rb_sqlite3_aggregator_step(sqlite3_context *ctx, int argc, sqlite3_value **argv)
     rb_sqlite3_protected_funcall(
         handler_instance, rb_intern("step"), argc, params, &exc_status);
     if (argc > 1) {
-        xfree(params);
+        ALLOCV_END(params_handle);
     }
 
     rb_iv_set(inst, "-exc_status", INT2NUM(exc_status));
@@ -254,10 +257,17 @@ rb_sqlite3_define_aggregator2(VALUE self, VALUE aggregator, VALUE ruby_name)
     CHECK(ctx->db, status);
 
     rb_ary_push(aggregators, aw);
+    RB_OBJ_WRITE(self, &ctx->aggregators, aggregators);
 
     return self;
 }
 
+void
+rb_sqlite3_aggregator_pin_instances(VALUE aw)
+{
+    rb_sqlite3_pin_array_and_contents(rb_iv_get(aw, "-instances"));
+}
+
 void
 rb_sqlite3_aggregator_init(void)
 {

data/ext/sqlite3/aggregator.h

@@ -7,4 +7,8 @@ VALUE rb_sqlite3_define_aggregator2(VALUE self, VALUE aggregator, VALUE ruby_nam
 
 void rb_sqlite3_aggregator_init(void);
 
+/* sqlite stores each live instance's VALUE in its aggregate context, so those
+ * must not move either. */
+void rb_sqlite3_aggregator_pin_instances(VALUE aw);
+
 #endif

data/ext/sqlite3/database.c

@@ -10,6 +10,10 @@
   if(!_ctxt->db) \
     rb_raise(rb_path2class("SQLite3::Exception"), "cannot use a closed database");
 
+#define REQUIRE_CLOSED_DB(_ctxt) \
+  if(_ctxt->db) \
+    rb_raise(rb_path2class("SQLite3::Exception"), "cannot open a database that is already open");
+
 VALUE cSqlite3Database;
 
 /* See adr/2024-09-fork-safety.md */
@@ -70,11 +74,64 @@ close_or_discard_db(sqlite3RubyPtr ctx)
 }
 
 
+void
+rb_sqlite3_pin_array_and_contents(VALUE ary)
+{
+    long i;
+
+    if (NIL_P(ary) || !ary) { return; }
+
+    rb_gc_mark(ary);
+    for (i = 0; i < RARRAY_LEN(ary); i++) {
+        rb_gc_mark(RARRAY_AREF(ary, i));
+    }
+}
+
+static int
+pin_hash_value(VALUE UNUSED(key), VALUE value, VALUE UNUSED(arg))
+{
+    rb_gc_mark(value);
+    return ST_CONTINUE;
+}
+
+static void
+pin_hash_and_contents(VALUE hash)
+{
+    if (NIL_P(hash) || !hash) { return; }
+
+    rb_gc_mark(hash);
+    rb_hash_foreach(hash, pin_hash_value, 0);
+}
+
+/* Each wrapper also owns live aggregate instances, whose VALUEs sqlite keeps in
+ * its own aggregate contexts. */
+static void
+pin_aggregators(VALUE aggregators)
+{
+    long i;
+
+    rb_sqlite3_pin_array_and_contents(aggregators);
+
+    if (NIL_P(aggregators) || !aggregators) { return; }
+
+    for (i = 0; i < RARRAY_LEN(aggregators); i++) {
+        rb_sqlite3_aggregator_pin_instances(RARRAY_AREF(aggregators, i));
+    }
+}
+
 static void
 database_mark(void *ctx)
 {
     sqlite3RubyPtr c = (sqlite3RubyPtr)ctx;
+
+    /* sqlite holds raw pointers to these, so they must not move. */
     rb_gc_mark(c->busy_handler);
+    rb_gc_mark(c->trace_handler);
+    rb_gc_mark(c->authorizer);
+
+    rb_sqlite3_pin_array_and_contents(c->functions);
+    pin_hash_and_contents(c->collations);
+    pin_aggregators(c->aggregators);
 }
 
 static void
@@ -114,7 +171,21 @@ allocate(VALUE klass)
 static char *
 utf16_string_value_ptr(VALUE str)
 {
+    VALUE utf16str, codepoints;
+
     StringValue(str);
+
+    utf16str = rb_str_dup(str);
+    if (!UTF16_LE_P(utf16str) && !UTF16_BE_P(utf16str)) {
+        /* an untagged string (the utf16 option) holds native-byte-order UTF-16 */
+        const union { uint16_t u16; uint8_t u8[2]; } native = { 1 };
+        rb_enc_associate_index(utf16str, rb_enc_find_index(native.u8[0] ? "UTF-16LE" : "UTF-16BE"));
+    }
+    codepoints = rb_funcall(utf16str, rb_intern("codepoints"), 0);
+    if (RTEST(rb_funcall(codepoints, rb_intern("include?"), 1, INT2FIX(0)))) {
+        rb_raise(rb_eArgError, "string contains null char");
+    }
+
     rb_str_buf_cat(str, "\x00\x00", 2L);
     return RSTRING_PTR(str);
 }
@@ -135,6 +206,7 @@ rb_sqlite3_open_v2(VALUE self, VALUE file, VALUE mode, VALUE zvfs)
     int flags;
 
     TypedData_Get_Struct(self, sqlite3Ruby, &database_type, ctx);
+    REQUIRE_CLOSED_DB(ctx);
 
 #if defined TAINTING_SUPPORT
 #  if defined StringValueCStr
@@ -147,13 +219,19 @@ rb_sqlite3_open_v2(VALUE self, VALUE file, VALUE mode, VALUE zvfs)
 
     flags = NUM2INT(mode);
     status = sqlite3_open_v2(
-                 StringValuePtr(file),
+                 StringValueCStr(file),
                  &ctx->db,
                  flags,
-                 NIL_P(zvfs) ? NULL : StringValuePtr(zvfs)
+                 NIL_P(zvfs) ? NULL : StringValueCStr(zvfs)
              );
 
-    CHECK(ctx->db, status);
+    if (status != SQLITE_OK) {
+        char *msg = sqlite3_mprintf("%s", sqlite3_errmsg(ctx->db));
+        sqlite3_close_v2(ctx->db);
+        ctx->db = NULL;
+        CHECK_MSG(ctx->db, status, msg);
+    }
+
     if (flags & SQLITE_OPEN_READONLY) {
         ctx->flags |= SQLITE3_RB_DATABASE_READONLY;
     }
@@ -246,9 +324,8 @@ total_changes(VALUE self)
 static void
 tracefunc(void *data, const char *sql)
 {
-    VALUE self = (VALUE)data;
-    VALUE thing = rb_iv_get(self, "@tracefunc");
-    rb_funcall(thing, rb_intern("call"), 1, rb_str_new2(sql));
+    sqlite3RubyPtr ctx = (sqlite3RubyPtr)data;
+    rb_funcall(ctx->trace_handler, rb_intern("call"), 1, rb_str_new2(sql));
 }
 
 /* call-seq:
@@ -272,9 +349,9 @@ trace(int argc, VALUE *argv, VALUE self)
 
     if (NIL_P(block) && rb_block_given_p()) { block = rb_block_proc(); }
 
-    rb_iv_set(self, "@tracefunc", block);
+    RB_OBJ_WRITE(self, &ctx->trace_handler, block);
 
-    sqlite3_trace(ctx->db, NIL_P(block) ? NULL : tracefunc, (void *)self);
+    sqlite3_trace(ctx->db, NIL_P(block) ? NULL : tracefunc, (void *)ctx);
 
     return self;
 }
@@ -340,8 +417,10 @@ rb_sqlite3_statement_timeout(void *context)
     clock_gettime(CLOCK_MONOTONIC, &currentTime);
 
     if (!timespecisset(&ctx->stmt_deadline)) {
-        // Set stmt_deadline if not already set
-        ctx->stmt_deadline = currentTime;
+        struct timespec timeout;
+        timeout.tv_sec = ctx->stmt_timeout / 1000;
+        timeout.tv_nsec = (ctx->stmt_timeout % 1000) * 1000000L;
+        timespecadd(&currentTime, &timeout, &ctx->stmt_deadline);
     } else if (timespecafter(&currentTime, &ctx->stmt_deadline)) {
         return 1;
     }
@@ -398,7 +477,9 @@ sqlite3val2rb(sqlite3_value *val)
             rb_val = rb_float_new(sqlite3_value_double(val));
             break;
         case SQLITE_TEXT: {
-            rb_val = rb_utf8_str_new_cstr((const char *)sqlite3_value_text(val));
+            const char *text = (const char *)sqlite3_value_text(val);
+            int len = sqlite3_value_bytes(val);
+            rb_val = rb_utf8_str_new(text, len);
             rb_obj_freeze(rb_val);
             break;
         }
@@ -466,24 +547,46 @@ set_sqlite3_func_result(sqlite3_context *ctx, VALUE result)
     }
 }
 
-static void
-rb_sqlite3_func(sqlite3_context *ctx, int argc, sqlite3_value **argv)
+typedef struct {
+    sqlite3_context *ctx;
+    int argc;
+    sqlite3_value **argv;
+} rb_sqlite3_func_args_t;
+
+static VALUE
+rb_sqlite3_func_protected(VALUE func_args_value)
 {
-    VALUE callable = (VALUE)sqlite3_user_data(ctx);
-    VALUE params = rb_ary_new2(argc);
+    rb_sqlite3_func_args_t *args = (rb_sqlite3_func_args_t *)func_args_value;
+    VALUE callable = (VALUE)sqlite3_user_data(args->ctx);
+    VALUE params = rb_ary_new2(args->argc);
     VALUE result;
     int i;
 
-    if (argc > 0) {
-        for (i = 0; i < argc; i++) {
-            VALUE param = sqlite3val2rb(argv[i]);
-            rb_ary_push(params, param);
-        }
+    for (i = 0; i < args->argc; i++) {
+        VALUE param = sqlite3val2rb(args->argv[i]);
+        rb_ary_push(params, param);
     }
 
     result = rb_apply(callable, rb_intern("call"), params);
 
-    set_sqlite3_func_result(ctx, result);
+    set_sqlite3_func_result(args->ctx, result);
+
+    return Qnil;
+}
+
+static void
+rb_sqlite3_func(sqlite3_context *ctx, int argc, sqlite3_value **argv)
+{
+    rb_sqlite3_func_args_t args = { .ctx = ctx, .argc = argc, .argv = argv };
+    int exc_status;
+
+    rb_protect(rb_sqlite3_func_protected, (VALUE)&args, &exc_status);
+
+    if (exc_status) {
+        /* the user should never see this message, because Statement#step will
+         * re-raise the exception still in rb_errinfo */
+        sqlite3_result_error(ctx, "Ruby Exception occurred", -1);
+    }
 }
 
 #ifndef HAVE_RB_PROC_ARITY
@@ -503,7 +606,7 @@ static VALUE
 define_function_with_flags(VALUE self, VALUE name, VALUE flags)
 {
     sqlite3RubyPtr ctx;
-    VALUE block;
+    VALUE block, functions;
     int status;
 
     TypedData_Get_Struct(self, sqlite3Ruby, &database_type, ctx);
@@ -524,7 +627,9 @@ define_function_with_flags(VALUE self, VALUE name, VALUE flags)
 
     CHECK(ctx->db, status);
 
-    rb_ary_push(rb_iv_get(self, "@functions"), block);
+    functions = rb_iv_get(self, "@functions");
+    rb_ary_push(functions, block);
+    RB_OBJ_WRITE(self, &ctx->functions, functions);
 
     return self;
 }
@@ -626,14 +731,13 @@ rb_sqlite3_auth(
     const char *_c,
     const char *_d)
 {
-    VALUE self   = (VALUE)ctx;
+    sqlite3RubyPtr db_ctx = (sqlite3RubyPtr)ctx;
     VALUE action = INT2NUM(_action);
     VALUE a      = _a ? rb_str_new2(_a) : Qnil;
     VALUE b      = _b ? rb_str_new2(_b) : Qnil;
     VALUE c      = _c ? rb_str_new2(_c) : Qnil;
     VALUE d      = _d ? rb_str_new2(_d) : Qnil;
-    VALUE callback = rb_iv_get(self, "@authorizer");
-    VALUE result = rb_funcall(callback, rb_intern("call"), 5, action, a, b, c, d);
+    VALUE result = rb_funcall(db_ctx->authorizer, rb_intern("call"), 5, action, a, b, c, d);
 
     if (T_FIXNUM == TYPE(result)) { return (int)NUM2INT(result); }
     if (Qtrue == result) { return SQLITE_OK; }
@@ -662,12 +766,12 @@ set_authorizer(VALUE self, VALUE authorizer)
     REQUIRE_OPEN_DB(ctx);
 
     status = sqlite3_set_authorizer(
-                 ctx->db, NIL_P(authorizer) ? NULL : rb_sqlite3_auth, (void *)self
+                 ctx->db, NIL_P(authorizer) ? NULL : rb_sqlite3_auth, (void *)ctx
              );
 
     CHECK(ctx->db, status);
 
-    rb_iv_set(self, "@authorizer", authorizer);
+    RB_OBJ_WRITE(self, &ctx->authorizer, authorizer);
 
     return self;
 }
@@ -750,6 +854,7 @@ static VALUE
 collation(VALUE self, VALUE name, VALUE comparator)
 {
     sqlite3RubyPtr ctx;
+    VALUE collations;
     TypedData_Get_Struct(self, sqlite3Ruby, &database_type, ctx);
     REQUIRE_OPEN_DB(ctx);
 
@@ -760,8 +865,10 @@ collation(VALUE self, VALUE name, VALUE comparator)
               (void *)comparator,
               NIL_P(comparator) ? NULL : rb_comparator_func));
 
-    /* Make sure our comparator doesn't get garbage collected. */
-    rb_hash_aset(rb_iv_get(self, "@collations"), name, comparator);
+    /* sqlite holds a raw pointer to the comparator, so keep it alive and unmoved. */
+    collations = rb_iv_get(self, "@collations");
+    rb_hash_aset(collations, name, comparator);
+    RB_OBJ_WRITE(self, &ctx->collations, collations);
 
     return self;
 }
@@ -885,6 +992,8 @@ exec_batch(VALUE self, VALUE sql, VALUE results_as_hash)
     TypedData_Get_Struct(self, sqlite3Ruby, &database_type, ctx);
     REQUIRE_OPEN_DB(ctx);
 
+    timespecclear(&ctx->stmt_deadline);
+
     if (results_as_hash == Qtrue) {
         status = sqlite3_exec(ctx->db, StringValuePtr(sql), (sqlite3_callback)hash_callback_function,
                               (void *)callback_ary,
@@ -919,6 +1028,137 @@ db_filename(VALUE self, VALUE db_name)
     return Qnil;
 }
 
+#ifdef HAVE_SQLITE3_SERIALIZE
+/* call-seq: db.serialize(database_name = "main")
+ *
+ * Serialize the database into a binary string.
+ *
+ * [database_name]  The name of the database to serialize. The default value
+ *                  is +main+ which is the main database file. For ATTACHed
+ *                  databases, use the name given in the ATTACH statement.
+ *
+ * Returns the contents of the database as a binary string. For an ordinary
+ * on-disk database file, this is just a copy of the disk file. For an
+ * in-memory database or a "TEMP" database, this is the same sequence of bytes
+ * which would be written to disk if that database were backed up to disk.
+ *
+ * Note: This method will only be defined if SQLite was compiled without
+ * +SQLITE_OMIT_DESERIALIZE+.
+ *
+ * See also: https://www.sqlite.org/c3ref/serialize.html
+ */
+static VALUE
+serialize(int argc, VALUE *argv, VALUE self)
+{
+    sqlite3RubyPtr ctx;
+    VALUE db_name;
+    const char *zSchema;
+    sqlite3_int64 size;
+    unsigned char *data;
+    VALUE result;
+
+    TypedData_Get_Struct(self, sqlite3Ruby, &database_type, ctx);
+    REQUIRE_OPEN_DB(ctx);
+
+    rb_scan_args(argc, argv, "01", &db_name);
+
+    zSchema = NIL_P(db_name) ? NULL : StringValueCStr(db_name);
+
+    data = sqlite3_serialize(ctx->db, zSchema, &size, 0);
+
+    if (data == NULL && size < 0) {
+        rb_raise(rb_path2class("SQLite3::Exception"), "unable to serialize database \"%s\"",
+                 zSchema ? zSchema : "main");
+    }
+    if (data == NULL && size > 0) {
+        rb_raise(rb_eNoMemError, "failed to allocate %lld bytes for serialization", (long long)size);
+    }
+
+    result = rb_str_new((const char *)data, (long)size);
+    sqlite3_free(data);
+
+    return result;
+}
+#endif
+
+#ifdef HAVE_SQLITE3_DESERIALIZE
+/* call-seq: db.deserialize(data, database_name = "main")
+ *
+ * Deserialize a binary string into the database. The database is replaced with
+ * the contents of the serialized data.
+ *
+ * [data]           A binary string containing the serialized database content.
+ *                  This is typically obtained from a prior call to #serialize.
+ *
+ * [database_name]  The name of the database to replace. The default value is
+ *                  +main+ which is the main database file. For ATTACHed
+ *                  databases, use the name given in the ATTACH statement.
+ *
+ * Raises SQLite3::BusyException if the database has an open read or write
+ * transaction, including one held by an unfinished statement, or is involved in
+ * a backup.
+ *
+ * Deserializing into a database backed by a file detaches the connection from
+ * that file. The file is left unchanged, and #filename(database_name) returns an
+ * empty string afterwards.
+ *
+ * ⚠ An image of a database in WAL mode deserializes without error, but later
+ * statements, even <tt>SELECT 1</tt>, raise SQLite3::CantOpenException. Rails
+ * enables WAL mode by default.
+ *
+ * ⚠ Deserializing untrusted data carries the same risks as opening an untrusted
+ * database file. See https://www.sqlite.org/security.html
+ *
+ * Note: This method will only be defined if the loaded SQLite is version 3.54.0
+ * or later and was compiled without +SQLITE_OMIT_DESERIALIZE+. Older versions do
+ * not raise SQLite3::BusyException; they free memory that an unfinished statement
+ * still uses.
+ *
+ * See also: https://www.sqlite.org/c3ref/deserialize.html
+ */
+static VALUE
+deserialize(int argc, VALUE *argv, VALUE self)
+{
+    sqlite3RubyPtr ctx;
+    VALUE data, db_name;
+    const char *zSchema;
+    unsigned char *pData;
+    sqlite3_int64 szDb, szBuf;
+    int status;
+
+    TypedData_Get_Struct(self, sqlite3Ruby, &database_type, ctx);
+    REQUIRE_OPEN_DB(ctx);
+
+    rb_scan_args(argc, argv, "11", &data, &db_name);
+
+    StringValue(data);
+    zSchema = NIL_P(db_name) ? NULL : StringValueCStr(db_name);
+    szDb = RSTRING_LEN(data);
+
+    /* SQLite's docs advise padding the buffer with about 20 extra bytes, because a maliciously
+     * malformed image may make SQLite read past its end. See https://www.sqlite.org/c3ref/deserialize.html
+     *
+     * With SQLITE_DESERIALIZE_RESIZEABLE set, SQLite copies pages out of the buffer with bounds
+     * checks instead of reading it in place, so the padding is defensive. It also keeps the
+     * allocation nonzero for an empty string, for which sqlite3_malloc64(0) returns NULL.
+     */
+    szBuf = szDb + 20;
+    pData = sqlite3_malloc64(szBuf);
+    if (pData == NULL) {
+        rb_raise(rb_eNoMemError, "failed to allocate %lld bytes for deserialization", (long long)szBuf);
+    }
+    memcpy(pData, RSTRING_PTR(data), (size_t)szDb);
+    memset(pData + szDb, 0, (size_t)(szBuf - szDb));
+
+    status = sqlite3_deserialize(ctx->db, zSchema, pData, szDb, szDb,
+                                 SQLITE_DESERIALIZE_FREEONCLOSE | SQLITE_DESERIALIZE_RESIZEABLE);
+
+    CHECK(ctx->db, status);
+
+    return self;
+}
+#endif
+
 static VALUE
 rb_sqlite3_open16(VALUE self, VALUE file)
 {
@@ -926,6 +1166,7 @@ rb_sqlite3_open16(VALUE self, VALUE file)
     sqlite3RubyPtr ctx;
 
     TypedData_Get_Struct(self, sqlite3Ruby, &database_type, ctx);
+    REQUIRE_CLOSED_DB(ctx);
 
 #if defined TAINTING_SUPPORT
 #if defined StringValueCStr
@@ -941,7 +1182,12 @@ rb_sqlite3_open16(VALUE self, VALUE file)
     // so we do not ever set SQLITE3_RB_DATABASE_READONLY in ctx->flags
     status = sqlite3_open16(utf16_string_value_ptr(file), &ctx->db);
 
-    CHECK(ctx->db, status)
+    if (status != SQLITE_OK) {
+        char *msg = sqlite3_mprintf("%s", sqlite3_errmsg(ctx->db));
+        sqlite3_close_v2(ctx->db);
+        ctx->db = NULL;
+        CHECK_MSG(ctx->db, status, msg);
+    }
 
     return INT2NUM(status);
 }
@@ -993,6 +1239,16 @@ init_sqlite3_database(void)
 #endif
 #endif
 
+#ifdef HAVE_SQLITE3_SERIALIZE
+    rb_define_method(cSqlite3Database, "serialize", serialize, -1);
+#endif
+
+#ifdef HAVE_SQLITE3_DESERIALIZE
+    /* Before 3.54.0, sqlite3_deserialize frees memory that an unfinished statement still uses. */
+    if (sqlite3_libversion_number() >= 3054000) {
+        rb_define_method(cSqlite3Database, "deserialize", deserialize, -1);
+    }
+#endif
 
     rb_sqlite3_aggregator_init();
 }

data/ext/sqlite3/database.h

@@ -10,6 +10,11 @@
 struct _sqlite3Ruby {
     sqlite3 *db;
     VALUE busy_handler;
+    VALUE functions;
+    VALUE collations;
+    VALUE aggregators;
+    VALUE trace_handler;
+    VALUE authorizer;
     int stmt_timeout;
     struct timespec stmt_deadline;
     rb_pid_t owner;
@@ -19,6 +24,9 @@ struct _sqlite3Ruby {
 typedef struct _sqlite3Ruby sqlite3Ruby;
 typedef sqlite3Ruby *sqlite3RubyPtr;
 
+/* Pinning a collection doesn't pin what's in it, hence both. */
+void rb_sqlite3_pin_array_and_contents(VALUE ary);
+
 void init_sqlite3_database();
 void set_sqlite3_func_result(sqlite3_context *ctx, VALUE result);
 

data/ext/sqlite3/extconf.rb

@@ -65,8 +65,11 @@ module Sqlite3
               "-DSQLITE_ENABLE_DBPAGE_VTAB=1",
               "-DSQLITE_ENABLE_DBSTAT_VTAB=1"
             ]
-            env["CFLAGS"] = [user_cflags, env["CFLAGS"], more_cflags].flatten.join(" ")
-            recipe.configure_options += env.select { |k, v| ENV_ALLOWLIST.include?(k) }
+            # sqlite >= 3.54 requires Vista APIs, same as ruby's default target. Placed first so a
+            # user-defined _WIN32_WINNT takes precedence.
+            windows_cflags = windows? ? ["-D_WIN32_WINNT=0x0600"] : []
+            env["CFLAGS"] = [windows_cflags, user_cflags, env["CFLAGS"], more_cflags].flatten.join(" ")
+            recipe.configure_options += env.slice(*ENV_ALLOWLIST)
               .map { |key, value| "#{key}=#{value.strip}" }
           end
 
@@ -115,9 +118,10 @@ module Sqlite3
 
         abort_could_not_find(libname) unless find_library(libname, "sqlite3_libversion_number", "sqlite3.h")
 
-        # Truffle Ruby doesn't support this yet:
+        # Ruby 3.0+, but not on every implementation:
         # https://github.com/oracle/truffleruby/issues/3408
         have_func("rb_enc_interned_str_cstr")
+        have_func("rb_enc_interned_str")
 
         # Functions defined in 1.9 but not 1.8
         have_func("rb_proc_arity")
@@ -139,6 +143,8 @@ module Sqlite3
         have_func("sqlite3_prepare_v2")
         have_func("sqlite3_db_name", "sqlite3.h") # v3.39.0
         have_func("sqlite3_error_offset", "sqlite3.h") # v3.38.0
+        have_func("sqlite3_serialize", "sqlite3.h") # v3.23.0
+        have_func("sqlite3_deserialize", "sqlite3.h") # v3.23.0
 
         have_type("sqlite3_int64", "sqlite3.h")
         have_type("sqlite3_uint64", "sqlite3.h")

data/ext/sqlite3/statement.c

@@ -66,6 +66,8 @@ prepare(VALUE self, VALUE db, VALUE sql)
      * variable will keep it from being GCed. */
     ctx->db = db_ctx;
 
+    timespecclear(&db_ctx->stmt_deadline);
+
 #ifdef HAVE_SQLITE3_PREPARE_V2
     status = sqlite3_prepare_v2(
 #else
@@ -79,7 +81,6 @@ prepare(VALUE self, VALUE db, VALUE sql)
              );
 
     CHECK_PREPARE(db_ctx->db, status, StringValuePtr(sql));
-    timespecclear(&db_ctx->stmt_deadline);
 
     return rb_utf8_str_new_cstr(tail);
 }
@@ -139,6 +140,10 @@ step(VALUE self)
 
     stmt = ctx->st;
 
+    if (!sqlite3_stmt_busy(stmt)) {
+        timespecclear(&ctx->db->stmt_deadline);
+    }
+
     value = sqlite3_step(stmt);
     if (rb_errinfo() != Qnil) {
         /* some user defined function was invoked as a callback during step and
@@ -216,6 +221,21 @@ step(VALUE self)
  * Fixnum, it is treated as an index for a positional placeholder.
  * Otherwise it is used as the name of the placeholder to bind to.
  *
+ * Type mapping from Ruby to SQLite3:
+ * - +nil+        → NULL
+ * - Integer      → INTEGER (or REAL when outside the signed int64 range)
+ * - Float        → REAL
+ * - SQLite3::Blob → BLOB
+ * - String with Encoding::ASCII_8BIT (a.k.a. BINARY) → BLOB
+ * - String with Encoding::UTF_16LE or Encoding::UTF_16BE → TEXT (bound as UTF-16)
+ * - String (all other encodings) → TEXT (re-encoded to UTF-8 if necessary)
+ * - Any other type → raises RuntimeError
+ *
+ * Note: if you have a string with only ASCII characters but ASCII-8BIT
+ * encoding and want it bound as TEXT rather than BLOB, re-encode it first:
+ *
+ *   stmt.bind_param(1, my_binary_str.encode(Encoding::UTF_8))
+ *
  * See also #bind_params.
  */
 static VALUE
@@ -398,6 +418,33 @@ interned_utf8_cstr(const char *str)
 }
 #endif
 
+/* Like interned_utf8_cstr, but ASCII-downcases as it copies. */
+static VALUE
+interned_downcased_utf8_cstr(const char *str)
+{
+#if HAVE_RB_ENC_INTERNED_STR
+    VALUE tmp, ret;
+    char *buf;
+    size_t i, len;
+
+    len = strlen(str);
+    buf = ALLOCV_N(char, tmp, len);
+
+    for (i = 0; i < len; i++) {
+        unsigned char c = (unsigned char)str[i];
+        buf[i] = (char)((c >= 'A' && c <= 'Z') ? c + ('a' - 'A') : c);
+    }
+
+    ret = rb_enc_interned_str(buf, (long)len, rb_utf8_encoding());
+    ALLOCV_END(tmp);
+    return ret;
+#else
+    VALUE rb_str = rb_funcall(rb_utf8_str_new_cstr(str), rb_intern("downcase"), 1,
+                              ID2SYM(rb_intern("ascii")));
+    return rb_funcall(rb_str, rb_intern("-@"), 0);
+#endif
+}
+
 /* call-seq: stmt.column_name(index)
  *
  * Get the column name at +index+.  0 based.
@@ -425,7 +472,8 @@ column_name(VALUE self, VALUE index)
 
 /* call-seq: stmt.column_decltype(index)
  *
- * Get the column type at +index+.  0 based.
+ * Get the declared type of the column at +index+, as a UTF-8 string.  0 based.
+ * Returns +nil+ if the column has no declared type, such as an expression.
  */
 static VALUE
 column_decltype(VALUE self, VALUE index)
@@ -440,7 +488,24 @@ column_decltype(VALUE self, VALUE index)
 
     name = sqlite3_column_decltype(ctx->st, (int)NUM2INT(index));
 
-    if (name) { return rb_str_new2(name); }
+    if (name) { return rb_utf8_str_new_cstr(name); }
+    return Qnil;
+}
+
+static VALUE
+column_decltype_downcased(VALUE self, VALUE index)
+{
+    sqlite3StmtRubyPtr ctx;
+    const char *name;
+
+    TypedData_Get_Struct(self, sqlite3StmtRuby, &statement_type, ctx);
+
+    REQUIRE_LIVE_DB(ctx);
+    REQUIRE_OPEN_STMT(ctx);
+
+    name = sqlite3_column_decltype(ctx->st, (int)NUM2INT(index));
+
+    if (name) { return interned_downcased_utf8_cstr(name); }
     return Qnil;
 }
 
@@ -733,6 +798,7 @@ init_sqlite3_statement(void)
     rb_define_method(cSqlite3Statement, "memused", memused, 0);
 #endif
 
+    rb_define_private_method(cSqlite3Statement, "column_decltype_downcased", column_decltype_downcased, 1);
     rb_define_private_method(cSqlite3Statement, "prepare", prepare, 2);
     rb_define_private_method(cSqlite3Statement, "stats_as_hash", stats_as_hash, 0);
     rb_define_private_method(cSqlite3Statement, "stat_for", stat_for, 1);

data/ext/sqlite3/timespec.h

@@ -15,6 +15,15 @@
 			(vsp)->tv_nsec += 1000000000L;			\
 		}							\
 	} while (0)
+#define	timespecadd(tsp, usp, vsp)					\
+	do {								\
+		(vsp)->tv_sec = (tsp)->tv_sec + (usp)->tv_sec;		\
+		(vsp)->tv_nsec = (tsp)->tv_nsec + (usp)->tv_nsec;	\
+		if ((vsp)->tv_nsec >= 1000000000L) {			\
+			(vsp)->tv_sec++;				\
+			(vsp)->tv_nsec -= 1000000000L;			\
+		}							\
+	} while (0)
 #define timespecafter(tsp, usp)						\
 	(((tsp)->tv_sec > (usp)->tv_sec) ||				\
 	 ((tsp)->tv_sec == (usp)->tv_sec && (tsp)->tv_nsec > (usp)->tv_nsec))

data/lib/sqlite3/database.rb

@@ -171,8 +171,6 @@ module SQLite3
         end
       end
 
-      @tracefunc = nil
-      @authorizer = nil
       @progress_handler = nil
       @collations = {}
       @functions = []
@@ -458,9 +456,7 @@ module SQLite3
     #
     # See also #create_aggregate_handler for a more object-oriented approach to
     # aggregate functions.
-    def create_aggregate(name, arity, step = nil, finalize = nil,
-      text_rep = Constants::TextRep::ANY, &block)
-
+    def create_aggregate(name, arity, step = nil, finalize = nil, text_rep = Constants::TextRep::ANY, &block)
       proxy = Class.new do
         def self.step(&block)
           define_method(:step_with_ctx, &block)

data/lib/sqlite3/resultset.rb

@@ -43,20 +43,26 @@ module SQLite3
       @stmt.step
     end
 
-    # Required by the Enumerable mixin. Provides an internal iterator over the
-    # rows of the result set.
+    # With a block given, iterates over the rows of the result set, passing each
+    # to the block. Returns self.
+    # With no block given, returns a new Enumerator.
     def each
+      return enum_for(__method__) unless block_given?
       while (node = self.next)
         yield node
       end
+      self
     end
 
-    # Provides an internal iterator over the rows of the result set where
-    # each row is yielded as a hash.
+    # With a block given, iterates over the rows of the result set, passing each
+    # to the block as a hash. Returns self.
+    # With no block given, returns a new Enumerator.
     def each_hash
+      return enum_for(__method__) unless block_given?
       while (node = next_hash)
         yield node
       end
+      self
     end
 
     # Closes the statement that spawned this result set.

data/lib/sqlite3/statement.rb

@@ -42,6 +42,24 @@ module SQLite3
     # See Database#execute for a description of the valid placeholder
     # syntaxes.
     #
+    # Type mapping from Ruby to SQLite3:
+    # - +nil+                            → NULL
+    # - Integer                          → INTEGER (or REAL when outside the signed int64 range)
+    # - Float                            → REAL
+    # - SQLite3::Blob                    → BLOB
+    # - String with Encoding::ASCII_8BIT
+    #   (a.k.a. BINARY)                  → BLOB
+    # - String with Encoding::UTF_16LE
+    #   or Encoding::UTF_16BE            → TEXT (bound as UTF-16)
+    # - String (all other encodings)     → TEXT (re-encoded to UTF-8 if needed)
+    # - Any other type                   → raises RuntimeError
+    #
+    # Note: a String with Encoding::ASCII_8BIT (BINARY) is always bound as a
+    # BLOB, even when its bytes are all valid ASCII. If you want such a string
+    # compared or stored as TEXT, re-encode it before binding:
+    #
+    #   stmt.bind_params(my_binary_str.encode(Encoding::UTF_8))
+    #
     # Example:
     #
     #   stmt = db.prepare( "select * from table where a=? and b=?" )
@@ -112,9 +130,9 @@ module SQLite3
       !done?
     end
 
-    # Return an array of the column names for this statement. Note that this
-    # may execute the statement in order to obtain the metadata; this makes it
-    # a (potentially) expensive operation.
+    # Return a frozen array of the column names for this statement, as frozen
+    # UTF-8 strings. Note that this may execute the statement in order to
+    # obtain the metadata; this makes it a (potentially) expensive operation.
     def columns
       get_metadata unless @columns
       @columns
@@ -128,9 +146,11 @@ module SQLite3
       end
     end
 
-    # Return an array of the data types for each column in this statement. Note
-    # that this may execute the statement in order to obtain the metadata; this
-    # makes it a (potentially) expensive operation.
+    # Return a frozen array of the declared types of the columns in this
+    # statement, as frozen UTF-8 strings with ASCII letters downcased. A column
+    # with no declared type, such as an expression, has a +nil+ type. Note that
+    # this may execute the statement in order to obtain the metadata; this makes
+    # it a (potentially) expensive operation.
     def types
       must_be_open!
       get_metadata unless @types
@@ -180,11 +200,10 @@ module SQLite3
     def get_metadata
       @columns = Array.new(column_count) do |column|
         column_name column
-      end
+      end.freeze
       @types = Array.new(column_count) do |column|
-        val = column_decltype(column)
-        val&.downcase
-      end
+        column_decltype_downcased(column)
+      end.freeze
     end
   end
 end

data/lib/sqlite3/version.rb

@@ -1,4 +1,4 @@
 module SQLite3
   # (String) the version of the sqlite3 gem, e.g. "2.1.1"
-  VERSION = "2.9.5"
+  VERSION = "2.10.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: sqlite3
 version: !ruby/object:Gem::Version
-  version: 2.9.5
+  version: 2.10.0
 platform: x86-linux-gnu
 authors:
 - Jamis Buck