extralite 2.15 → 3.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 4394001511dda6eca48bbcf1cc6904e8bb0fc8b9951915bc0cf7192b2ea31c9a
-  data.tar.gz: f0466fc3d8cda8fd367aa159cf89a98fc667c9a55e9bdd589afc7cb8446c58a9
+  metadata.gz: eb66a5b2636ccad51ab5a86ff0b7802d25574a564a5c4395c838a2d26f2599b9
+  data.tar.gz: d00921f287ba1c602c8669ff85740108a7bafb4f8111722f14055612db7afc3f
 SHA512:
-  metadata.gz: 2051da94afe02e5facb094de6a03727c6215f1fb115d34e7541f52b5999e8750972fd697e05ed3e61c6f49103a43d0be1f73fe06d638415cf494acae99be839e
-  data.tar.gz: 3ed1815f50ff89b5e1d37f06afd17dc17737c68e99cb0a9a415d3f319c4a9fc926abab0452e9a4b27222e04440564c100f791a4ab668587954aedba73d44d78b
+  metadata.gz: be12c6a8629f83db24107fc6a6d06f1b94beff46622b9d6c0c47ed55575069cee2529389ba709a7f8dec4f31f11cb3738e3df25f17667264031014fc5e3dc7c8
+  data.tar.gz: 43be64a94f7e29cbafb25a2b32271e1b260d2a7f5b6f69eb214c0e069faaca1973e0bb92fb6d62f1bcba05e55136369c6841d28dc22c06b3290188da1a84bba2

data/.github/workflows/test-bundle.yml

@@ -12,7 +12,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-latest, macos-latest]
-        ruby: ['3.2', '3.3', '3.4', 'head']
+        ruby: ['3.4', '4.0', 'head']
 
     name: ${{matrix.os}}, ${{matrix.ruby}}
 

data/.github/workflows/test.yml

@@ -12,7 +12,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-latest, macos-latest]
-        ruby: ['3.2', '3.3', '3.4', 'head']
+        ruby: ['3.4', '4.0', 'head']
 
     name: ${{matrix.os}}, ${{matrix.ruby}}
 

data/CHANGELOG.md

@@ -1,3 +1,13 @@
+## 3.0.0 2026-07-02
+
+- Update benchmark results in README
+- Prevent extension loading using `load_extension` SQL function
+- Remove `wal: true` option, add `legacy: true` option for
+- Change default behaviour to set WAL journal mode + synchronous + foreign keys
+- Add transform DSL
+- Add support for type coercion in transforms
+- Add `Extralite::Transform` class for transforming rows into an object graph
+
 ## 2.15 2026-06-28
 
 - Update bundled SQLite to 3.53.3

data/README.md

@@ -37,12 +37,14 @@ latest features and enhancements.
 
 ## Features
 
-- Best-in-class [performance](#performance) (up to 4.5X the performance of the
+- Best-in-class [performance](#performance) (up to 2.35X the performance of the
   [sqlite3](https://github.com/sparklemotion/sqlite3-ruby) gem).
-- Support for [concurrency](#concurrency) out of the box for multi-threaded
-  and multi-fibered apps.
+- Support for [concurrency](#concurrency) out of the box for multi-threaded and
+  multi-fibered apps.
 - A variety of ways to [retrieve data](#query-modes) - hashes, arrays, single
-  columns, single rows, [transforms](#value-transforms).
+  columns, single rows.
+- Support for retrieving records as structured objects using
+  [transforms](#transforms).
 - Support for [external iteration](#iterating-over-records-in-a-prepared-query),
   allowing iterating through single records or batches of records.
 - [Prepared queries](#prepared-queries).
@@ -53,7 +55,7 @@ latest features and enhancements.
   [backups](#creating-backups), retrieve [status
   information](#retrieving-status-information), work with
   [changesets](#working-with-changesets), interrogate [database
-  limits](#working-with-database-limits),  [trace](#tracing-sql-statements)
+  limits](#working-with-database-limits), [trace](#tracing-sql-statements)
   queries.
 - [Sequel](#usage-with-sequel) adapter.
 
@@ -63,7 +65,7 @@ latest features and enhancements.
 - [Getting Started](#getting-started)
 - [Query Modes](#query-modes)
 - [Parameter binding](#parameter-binding)
-- [Value Transforms](#value-transforms)
+- [Transforms](#transforms)
 - [Data Types](#data-types)
 - [Prepared Queries](#prepared-queries)
 - [Batch Execution of Queries](#batch-execution-of-queries)
@@ -286,18 +288,20 @@ db.execute(sql, Extralite::Blob.new('Hello, 世界!'))
 db.execute(sql, 'Hello, 世界!'.force_encoding(Encoding::ASCII_8BIT))
 ```
 
-## Value Transforms
+## Transforms
 
 Extralite allows you to transform rows to any value your application may need by
-providing a transform proc that takes the raw row values and returns the
-transformed data. The transform proc is passed each resulting row either as a
-hash or as a list of values.
+providing a transform proc or transform object that takes the raw row values and
+returns the transformed data.
 
 Transforms are useful when you need to transform rows into ORM model instances,
 or when you need to do some other transformation on the values retrieved from
-the database.
+the database. With transforms, you can create nested, structured Ruby objects
+that represents different entities. This is especially useful for representing
+one-to-one, one-to-many or many-to-many relationships when doing joins.
 
-To transform results, pass a transform proc as the first parameter to one of the
+A transform proc is expressed as a lambda taking one or more values. To
+transform results, pass a transform proc as the first parameter to one of the
 `#query_xxx` methods:
 
 ```ruby
@@ -315,8 +319,50 @@ db.query_splat(transform, 'select a, b, c from foo')
 #=> transformed rows
 ```
 
-Value transforms can also be done with [prepared
-queries](#value-transforms-in-prepared-queries).
+### Structured Transforms
+
+To transform rows into an object graph containing entities, you can use the
+`Extralite::Transform` class, which lets you express data as an object graph.
+This is particularly useful when doing `JOIN` queries. For example, a query may
+express a many-to-many relationship between posts and tags:
+
+```sql
+select
+  posts.id, posts.content,
+  tags.id, tags.name
+from posts
+left outer join posts_tags
+  on posts_tags.post_id = posts.id
+left outer join tags
+  on posts_tags.tag_id = tags.id
+```
+
+The result rows will contain information about both post and tag entities, and
+each entity (be it a post or a tag) may be repeated in multiple rows. With a
+transform you can tell Extralite to eliminate the duplicate entities by using an
+identity map, and to include a list of tags for each post:
+
+```ruby
+transform = Extralite::Transform do
+  {
+    id:       integer.identity, # posts.id
+    content:  text,             # posts.content
+    tags:     [{
+      id:     integer.identity, # tags.id
+      name:   text              # tags.name
+    }]
+  }
+end
+```
+
+To use the transform, pass it along with the SQL string to `Database#query`:
+
+```ruby
+db.qurey(transform, sql) #=> [...]
+```
+
+Transforms can also be used with [prepared
+queries](#transforms-in-prepared-queries).
 
 ## Prepared Queries
 
@@ -470,12 +516,13 @@ iterator = Extralite::Iterator.new(query)
 iterator.each { |r| ... }
 ```
 
-### Value Transforms in Prepared Queries
+### Transforms in Prepared Queries
 
 Prepared queries can automatically transform their result sets by setting a
-transform block. The transform block receives values according to the query mode
-(hash, array or splat). To set a transform you can pass a block to one of the
-`Database#prepare_xxx` methods, or use `Query#transform`:
+transform proc or object. If a transform proc is provided, the proc will be
+invoked according to the query mode (hash, array or splat). To set a transform
+proc you can pass a block to one of the `Database#prepare_xxx` methods, or use
+`Query#transform=`:
 
 ```ruby
 q = db.prepare('select * from items where id = ?') { |h| Item.new(h) }
@@ -483,7 +530,7 @@ q.bind(42).next #=> Item instance
 
 # An equivalent
 q = db.prepare('select * from items where id = ?')
-q.transform { |h| Item.new(h) }
+q.transform = ->(h) { Item.new(h) }
 ```
 
 The same can be done for queries in `splat` or `array` mode:
@@ -494,6 +541,26 @@ db.prepare_splat('select * from foo') { |a, b, c| a + b + c }
 db.prepare_array('select * from foo') { |a| a.map(&:to_s).join }
 ```
 
+You can also use structured transforms with prepared queries:
+
+```ruby
+q = db.prepare <<~SQL
+  select posts.id, posts.content, author.id, author.name
+  from posts left join authors
+    on posts.author_id = authors.id
+SQL
+q.transform = Extralite::Transform.new do
+  {
+    id:       integer.identity,
+    content:  text,
+    author: {
+      id:     integer.identity,
+      name:   text
+    }
+  }
+end
+```
+
 ## Batch Execution of Queries
 
 Extralite provides methods for batch execution of queries, with multiple sets of
@@ -764,16 +831,13 @@ Extralite provides a comprehensive set of tools for dealing with concurrency
 issues, and for making sure that running queries on SQLite databases does not
 cause the app to freeze.
 
-**Note**: In order to allow concurrent access your the database, it is highly
-recommended that you set your database to use [WAL journaling
-mode](https://www.sqlite.org/wal.html) for *all* database connections.
-Otherwise, you risking running into performance problems and having queries fail
-with `BusyError` exceptions. You can easily open your database in WAL journaling
-mode by passing a `wal: true` option:
+**Note**: By default Extralite automatically configures SQLite to [WAL
+journaling mode](https://www.sqlite.org/wal.html) for *all* database
+connections. To open a legacy database without WAL journaling, you can open the
+database and pass the `legacy: true` option:
 
 ```ruby
-# This will set PRAGMA journal_mode=1 and PRAGMA synchronous=1
-db = Extralite::Database.new('path/to/db', wal: true)
+db = Extralite::Database.new('path/to/db', legacy: true)
 ```
 
 ### The Ruby GVL
@@ -1248,7 +1312,7 @@ p articles.to_a
 
 A benchmark script is included, creating a table of various row counts, then
 fetching the entire table using either `sqlite3` or `extralite`. This benchmark
-shows Extralite to be up to ~4.5 times faster than `sqlite3` when fetching a
+shows Extralite to be up to ~2.35 times faster than `sqlite3` when fetching a
 large number of rows.
 
 ### Rows as Hashes
@@ -1256,36 +1320,36 @@ large number of rows.
 [Benchmark source
 code](https://github.com/digital-fabric/extralite/blob/main/test/perf_hash.rb)
 
-|Row count|sqlite3 2.6.0|Extralite 2.12|Advantage|
+|Row count|sqlite3 2.9.5|Extralite 2.15|Advantage|
 |-:|-:|-:|-:|
-|10|629.0K rows/s|950.4K rows/s|__1.51x__|
-|1K|1770.5K rows/s|4321.5K rows/s|__2.44x__|
-|100K|1028.8K rows/s|4088.7K rows/s|__3.97x__|
+|10|111.6K rows/s|177.3K rows/s|__1.59x__|
+|1K|2732.8K rows/s|5863.0K rows/s|__2.15x__|
+|100K|2129.0K rows/s|4863.1K rows/s|__2.28x__|
 
 ### Rows as Arrays
 
 [Benchmark source
 code](https://github.com/digital-fabric/extralite/blob/main/test/perf_array.rb)
 
-|Row count|sqlite3 2.6.0|Extralite 2.12|Advantage|
+|Row count|sqlite3 2.9.5|Extralite 2.15|Advantage|
 |-:|-:|-:|-:|
-|10|889.4K rows/s|1000.1K rows/s|__1.13x__|
-|1K|4518.1K rows/s|5381.5K rows/s|__1.19x__|
-|100K|4454.0K rows/s|5083.8K rows/s|__1.14x__|
+|10|1635.0K rows/s|1892.6K rows/s|__1.16x__|
+|1K|6197.4K rows/s|6691.5K rows/s|__1.08x__|
+|100K|5845.3K rows/s|5560.3K rows/s|__0.95x__|
 
 ### Prepared Queries (Prepared Statements)
 
 [Benchmark source
 code](https://github.com/digital-fabric/extralite/blob/main/test/perf_hash_prepared.rb)
 
-|Row count|sqlite3 2.6.0|Extralite 2.12|Advantage|
+|Row count|sqlite3 2.9.5|Extralite 2.15|Advantage|
 |-:|-:|-:|-:|
-|10|783.1K rows/s|1115.1K rows/s|__1.42x__|
-|1K|1782.5K rows/s|4635.5K rows/s|__2.60x__|
-|100K|1018.1K rows/s|4599.4K rows/s|__4.52x__|
+|10|1584.5K rows/s|2420.4K rows/s|__1.53x__|
+|1K|2738.2K rows/s|6124.3K rows/s|__2.24x__|
+|100K|2137.6K rows/s|5015.1K rows/s|__2.35x__|
 
-As those benchmarks show, Extralite is capabale of reading up to 4.5M rows per
-second, and can be more than 4 times faster than the `sqlite3` gem.
+As those benchmarks show, Extralite is capabale of reading up to 6.7M rows per
+second, and can be more than 2 times faster than the `sqlite3` gem.
 
 Note that the benchmarks above were performed on synthetic data, in a
 single-threaded environment, with the GVL release threshold set to -1, which

data/TODO.md

@@ -1,16 +1,2 @@
-- Transform objects:
-
-
-
-
-- More database methods:
-
-  - `Database#quote`
-  - `Database#cache_flush` https://sqlite.org/c3ref/db_cacheflush.html
-  - `Database#release_memory` https://sqlite.org/c3ref/db_release_memory.html
-
-- Security
-
-  - Enable extension loading by using
-    [SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION](https://www.sqlite.org/c3ref/c_dbconfig_defensive.html#sqlitedbconfigenableloadextension)
-    in order to prevent usage of `load_extension()` SQL function.
+- [ ] Version 3.0
+  - [ ] Run benchmarks again against latest version of sqlite3 gem

data/examples/transform.rb

@@ -0,0 +1,61 @@
+# frozen_string_literal: true
+
+require 'bundler/inline'
+
+gemfile do
+  gem 'extralite', path: '.'
+end
+
+db = Extralite::Database.new(':memory:')
+db.pragma('foreign_keys' => 1)
+db.execute <<~SQL
+  create table posts (
+    id integer primary key,
+    title text,
+    content text
+  );
+  create table tags (
+    id integer primary key,
+    name text
+  );
+  create table posts_tags (
+    post_id integer references posts(id) on delete cascade,
+    tag_id integer references tags(id) on delete cascade
+  );
+
+  insert into posts (title, content) values ('T1', 'C1');
+  insert into posts (title, content) values ('T2', 'C2');
+  insert into tags (name) values ('tag1');
+  insert into tags (name) values ('tag2');
+  insert into tags (name) values ('tag3');
+
+  insert into posts_tags(post_id, tag_id) values (1, 1);
+  insert into posts_tags(post_id, tag_id) values (1, 2);
+  insert into posts_tags(post_id, tag_id) values (2, 2);
+  insert into posts_tags(post_id, tag_id) values (2, 3);
+SQL
+
+sql = <<~SQL
+  select
+    posts.id, posts.title, posts.content,
+    tags.id, tags.name
+  from posts
+  left outer join posts_tags on posts_tags.post_id = posts.id
+  left outer join tags on posts_tags.tag_id = tags.id
+  order by posts.id, tags.id
+SQL
+
+transform = Extralite::Transform.new do
+  {
+    id:       integer.identity,
+    title:    text,
+    content:  text,
+    tags:     [{
+      id:     integer.identity,
+      name:   text
+    }]
+  }
+end
+
+require 'pp'
+PP.pp db.query(transform, sql), $stdout, 40

data/ext/extralite/changeset.c

@@ -117,7 +117,7 @@ VALUE cleanup_track(struct track_ctx *ctx) {
 /* Tracks changes in the given block and collects them into the changeset.
  * Changes are tracked only for the given tables. If nil is supplied as the
  * given tables, changes are tracked for all tables.
- * 
+ *
  *     # track changes for the foo and bar tables
  *     changeset.track(db, [:foo, :bar]) do
  *       run_some_queries
@@ -207,7 +207,7 @@ VALUE changeset_iter_info(sqlite3_changeset_iter *iter) {
   VALUE new_values = Qnil;
   VALUE converted = Qnil;
   VALUE row = rb_ary_new2(4);
-  
+
   const char *tbl_name;
   int column_count;
   int op_int;
@@ -298,7 +298,7 @@ inline void verify_changeset(Changeset_t *changeset) {
  * Each change entry is an array containing the operation (:insert / :update /
  * :delete), the table name, an array containing the old values, and an array
  * containing the new values.
- * 
+ *
  *     changeset.each do |(op, table, old_values, new_values)|
  *       ...
  *     end
@@ -322,7 +322,7 @@ VALUE Changeset_each(VALUE self) {
  * is an array containing the operation (:insert / :update / :delete), the table
  * name, an array containing the old values, and an array containing the new
  * values.
- * 
+ *
  * @return [Array<Array>] changes in the changeset
  */
 VALUE Changeset_to_a(VALUE self) {
@@ -344,7 +344,7 @@ static int xConflict(void *pCtx, int eConflict, sqlite3_changeset_iter *pIter){
 }
 
 /* Applies the changeset to the given database.
- * 
+ *
  * @param db [Extralite::Database] database to apply changes to
  * @return [Extralite::Changeset] changeset
  */
@@ -374,7 +374,7 @@ VALUE Changeset_apply(VALUE self, VALUE db) {
  *
  *     # undo changes
  *     changeset.invert.apply(db)
- * 
+ *
  * @return [Extralite::Changeset] inverted changeset
  */
 VALUE Changeset_invert(VALUE self) {
@@ -399,7 +399,7 @@ VALUE Changeset_invert(VALUE self) {
  * changeset BLOB can be stored to file for later retrieval.
  *
  *     File.open('my.changes', 'w+') { |f| f << changeset.to_blob }
- * 
+ *
  * @return [String] changeset BLOB
  */
 VALUE Changeset_to_blob(VALUE self) {
@@ -417,7 +417,7 @@ VALUE Changeset_to_blob(VALUE self) {
  *     changeset = Extralite::Changeset.new
  *     changeset.load(IO.read('my.changes'))
  *     changeset.apply(db)
- * 
+ *
  * @param blob [String] changeset BLOB
  * @return [Extralite::Changeset] changeset
  */
@@ -452,9 +452,9 @@ void Init_ExtraliteChangeset(void) {
   rb_define_method(cChangeset, "to_blob", Changeset_to_blob, 0);
   rb_define_method(cChangeset, "track", Changeset_track, 2);
 
-  SYM_delete = ID2SYM(rb_intern("delete"));
-  SYM_insert = ID2SYM(rb_intern("insert"));
-  SYM_update = ID2SYM(rb_intern("update"));
+  SYM_delete = ID2SYM(rb_intern_const("delete"));
+  SYM_insert = ID2SYM(rb_intern_const("insert"));
+  SYM_update = ID2SYM(rb_intern_const("update"));
 
   rb_gc_register_mark_object(SYM_delete);
   rb_gc_register_mark_object(SYM_insert);