extralite 2.6 → 2.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.gitignore +1 -0
- data/.yardopts +1 -1
- data/CHANGELOG.md +32 -17
- data/Gemfile +4 -0
- data/Gemfile-bundle +1 -1
- data/README.md +262 -75
- data/Rakefile +18 -0
- data/TODO.md +0 -9
- data/examples/kv_store.rb +49 -0
- data/examples/multi_fiber.rb +16 -0
- data/examples/on_progress.rb +9 -0
- data/examples/pubsub_store_polyphony.rb +194 -0
- data/examples/pubsub_store_threads.rb +204 -0
- data/ext/extralite/changeset.c +3 -3
- data/ext/extralite/common.c +173 -87
- data/ext/extralite/database.c +650 -315
- data/ext/extralite/extconf.rb +7 -11
- data/ext/extralite/extralite.h +89 -48
- data/ext/extralite/iterator.c +6 -84
- data/ext/extralite/query.c +165 -256
- data/extralite-bundle.gemspec +1 -1
- data/extralite.gemspec +1 -1
- data/gemspec.rb +10 -11
- data/lib/extralite/version.rb +1 -1
- data/lib/extralite.rb +27 -17
- data/lib/sequel/adapters/extralite.rb +1 -1
- data/test/helper.rb +2 -1
- data/test/perf_argv_transform.rb +74 -0
- data/test/perf_hash_transform.rb +66 -0
- data/test/perf_polyphony.rb +74 -0
- data/test/test_changeset.rb +2 -2
- data/test/test_database.rb +531 -115
- data/test/test_extralite.rb +2 -2
- data/test/test_iterator.rb +28 -13
- data/test/test_query.rb +348 -111
- data/test/test_sequel.rb +4 -4
- metadata +20 -14
- data/Gemfile.lock +0 -37
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 2cbb8c1507e23141a2b069306e31939b7cd5fea83e6941426972d7eeadc86ac0
|
4
|
+
data.tar.gz: e283cd0d2b18ed840020d73fb71d47696b64b8ff99c42a2a431928b6a25db189
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: cbe1b66c813d0e35bd667a06e14bc68673655807d256771dac231ecb2f559660a05773267ef804e40f9d747482974d897b78c0df370340c55e7cfd290bdc4a41
|
7
|
+
data.tar.gz: 385b3ec651b1c2365392c7a1dddb0b7b63adddc6ade662f8ba7d72d6d49f025dff0b55490df6296181d361ac5792e888ed7db90c8b11b2129b5d6474fae2c8de
|
data/.gitignore
CHANGED
data/.yardopts
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,32 +1,47 @@
|
|
1
|
+
# 2.7.1 2024-02-11
|
2
|
+
|
3
|
+
- Fix API docs.
|
4
|
+
|
5
|
+
# 2.7 2024-02-09
|
6
|
+
|
7
|
+
- Improve progress handler API, add mode, period, tick options, global progress
|
8
|
+
handler. [#68](https://github.com/digital-fabric/extralite/pull/68)
|
9
|
+
- Rework `Database#initialize` options
|
10
|
+
- Add argv row mode (for passing column values as argv)
|
11
|
+
- Streamline and improve query methods
|
12
|
+
[#67](https://github.com/digital-fabric/extralite/pull/67)
|
13
|
+
- Implement row transforms
|
14
|
+
|
1
15
|
# 2.6 2024-01-23
|
2
16
|
|
3
|
-
- Implement changeset API
|
17
|
+
- Implement changeset API.
|
4
18
|
[#58](https://github.com/digital-fabric/extralite/issues/58)
|
5
|
-
- Reorganize README, update benchmarks
|
19
|
+
- Reorganize README, update benchmarks.
|
6
20
|
[#63](https://github.com/digital-fabric/extralite/issues/63)
|
7
|
-
- Implement progress handler API
|
21
|
+
- Implement progress handler API.
|
8
22
|
[#62](https://github.com/digital-fabric/extralite/issues/62)
|
9
|
-
- Implement savepoint methods
|
23
|
+
- Implement savepoint methods.
|
10
24
|
|
11
25
|
# 2.5 2024-01-16
|
12
26
|
|
13
|
-
- Update bundled sqlite to version 3.45.0
|
14
|
-
- Implement `Database#batch_query` and related methods
|
27
|
+
- Update bundled sqlite to version 3.45.0.
|
28
|
+
- Implement `Database#batch_query` and related methods.
|
15
29
|
[53](https://github.com/digital-fabric/extralite/issues/53)
|
16
|
-
- Accept more options in `Database#initialize
|
30
|
+
- Accept more options in `Database#initialize`.
|
17
31
|
[48](https://github.com/digital-fabric/extralite/issues/48)
|
18
|
-
- Fix `Database#pragma` to return single value when reading pragma value
|
19
|
-
- Accept database name in `Database#tables` method
|
32
|
+
- Fix `Database#pragma` to return single value when reading pragma value.
|
33
|
+
- Accept database name in `Database#tables` method.
|
20
34
|
- Improve `Database#batch_execute` - now accepts Enumerable and Callable
|
21
|
-
parameters [52](https://github.com/digital-fabric/extralite/issues/52)
|
22
|
-
- Rename `Database#execute_multi` to `Database#batch_execute
|
23
|
-
- Implement Query#clone
|
35
|
+
parameters. [52](https://github.com/digital-fabric/extralite/issues/52)
|
36
|
+
- Rename `Database#execute_multi` to `Database#batch_execute`.
|
37
|
+
- Implement `Query#clone`.
|
24
38
|
[51](https://github.com/digital-fabric/extralite/issues/51)
|
25
|
-
- Add support for GC compaction
|
26
|
-
- Remove support for Ruby 2.7
|
27
|
-
- Implement Query
|
28
|
-
-
|
29
|
-
-
|
39
|
+
- Add support for GC compaction.
|
40
|
+
- Remove support for Ruby 2.7.
|
41
|
+
- Implement `Query#<<`.
|
42
|
+
[49](https://github.com/digital-fabric/extralite/issues/49)
|
43
|
+
- Allow passing parameters in array.
|
44
|
+
- Add support for ractors.
|
30
45
|
[#50](https://github.com/digital-fabric/extralite/issues/50)
|
31
46
|
|
32
47
|
# 2.4 2023-12-24
|
data/Gemfile
CHANGED
data/Gemfile-bundle
CHANGED
@@ -2,4 +2,4 @@ source 'https://rubygems.org'
|
|
2
2
|
|
3
3
|
gemspec name: 'extralite'
|
4
4
|
|
5
|
-
gem
|
5
|
+
gem 'extralite-bundle', git: "file://#{File.expand_path(__dir__)}", ref: `git rev-parse --abbrev-ref HEAD`.strip
|
data/README.md
CHANGED
@@ -3,7 +3,7 @@
|
|
3
3
|
Extralite
|
4
4
|
</h1>
|
5
5
|
|
6
|
-
<h4 align="center">
|
6
|
+
<h4 align="center">Ruby on SQLite</h4>
|
7
7
|
|
8
8
|
<p align="center">
|
9
9
|
<a href="http://rubygems.org/gems/extralite">
|
@@ -26,7 +26,7 @@
|
|
26
26
|
Extralite is a fast and innovative SQLite wrapper for Ruby with a rich set of
|
27
27
|
features. It provides multiple ways of retrieving data from SQLite databases,
|
28
28
|
makes it possible to use SQLite databases in multi-threaded and multi-fibered
|
29
|
-
Ruby apps, and
|
29
|
+
Ruby apps, and includes a comprehensive set of tools for managing SQLite
|
30
30
|
databases.
|
31
31
|
|
32
32
|
Extralite comes in two flavors: the `extralite` gem which uses the
|
@@ -37,27 +37,33 @@ latest features and enhancements.
|
|
37
37
|
|
38
38
|
## Features
|
39
39
|
|
40
|
-
- Best-in-class performance (up to 14X the performance of the
|
40
|
+
- Best-in-class [performance](#performance) (up to 14X the performance of the
|
41
41
|
[sqlite3](https://github.com/sparklemotion/sqlite3-ruby) gem).
|
42
42
|
- Support for [concurrency](#concurrency) out of the box for multi-threaded
|
43
|
-
apps.
|
44
|
-
- A variety of
|
45
|
-
|
46
|
-
- Support for external iteration
|
47
|
-
batches of records.
|
48
|
-
- Prepared queries.
|
49
|
-
- Parameter binding.
|
50
|
-
- Batch execution of queries.
|
51
|
-
-
|
52
|
-
-
|
53
|
-
-
|
54
|
-
-
|
43
|
+
and multi-fibered apps.
|
44
|
+
- A variety of ways to [retrieve data](#query-modes) - hashes, arrays, single
|
45
|
+
columns, single rows, [transforms](#value-transforms).
|
46
|
+
- Support for [external iteration](#iterating-over-records-in-a-prepared-query),
|
47
|
+
allowing iterating through single records or batches of records.
|
48
|
+
- [Prepared queries](#prepared-queries).
|
49
|
+
- [Parameter binding](#parameter-binding).
|
50
|
+
- [Batch execution](#batch-execution-of-queries) of queries.
|
51
|
+
- [transactions and savepoints](#transactions-and-savepoints).
|
52
|
+
- Advanced features: load [SQLite extensions](#loading-extensions), create
|
53
|
+
[backups](#creating-backups), retrieve [status
|
54
|
+
information](#retrieving-status-information), work with
|
55
|
+
[changesets](#working-with-changesets), interrogate [database
|
56
|
+
limits](#working-with-database-limits), [trace](#tracing-sql-statements)
|
57
|
+
queries.
|
58
|
+
- [Sequel](#usage-with-sequel) adapter.
|
55
59
|
|
56
60
|
## Table of Content
|
57
61
|
|
58
62
|
- [Installing Extralite](#installing-extralite)
|
59
|
-
- [
|
63
|
+
- [Getting Started](#getting-started)
|
64
|
+
- [Query Modes](#query-modes)
|
60
65
|
- [Parameter binding](#parameter-binding)
|
66
|
+
- [Value Transforms](#value-transforms)
|
61
67
|
- [Data Types](#data-types)
|
62
68
|
- [Prepared Queries](#prepared-queries)
|
63
69
|
- [Batch Execution of Queries](#batch-execution-of-queries)
|
@@ -97,10 +103,9 @@ SQLite source code.
|
|
97
103
|
Usage of the `extralite-bundle` gem is identical to the usage of the normal
|
98
104
|
`extralite` gem, using `require 'extralite'` to load the gem.
|
99
105
|
|
100
|
-
##
|
106
|
+
## Getting Started
|
101
107
|
|
102
|
-
|
103
|
-
queries:
|
108
|
+
The following example shows how to open an SQLite database and run some queries:
|
104
109
|
|
105
110
|
```ruby
|
106
111
|
db = Extralite::Database.new('mydb.sqlite')
|
@@ -126,24 +131,43 @@ db.query 'select * from foo' do |r|
|
|
126
131
|
end
|
127
132
|
```
|
128
133
|
|
129
|
-
|
134
|
+
## Query Modes
|
135
|
+
|
136
|
+
Extralite allows you to retrieve data from SQLite database in the form that most
|
137
|
+
a particular context. For some use cases you'll want to work with rows as
|
138
|
+
hashes. In other cases, you'll want to work with rows as arrays, or even as
|
139
|
+
single values, if you're just reading one column.
|
140
|
+
|
141
|
+
For that purpose, Extralite offers three different ways, or modes, of retrieving
|
142
|
+
records:
|
143
|
+
|
144
|
+
- `:hash`: read rows as hashes (this is the default mode).
|
145
|
+
- `:ary`: read rows as arrays.
|
146
|
+
- `:argv`: similar to the `:ary`, except that for queries with a single column,
|
147
|
+
the single column value is returned.
|
148
|
+
|
149
|
+
Extralite provides separate methods for the different modes:
|
130
150
|
|
131
151
|
```ruby
|
132
|
-
#
|
133
|
-
db.
|
134
|
-
#=> [[1, 2, 3], [4, 5, 6]]
|
152
|
+
# alias #query_hash
|
153
|
+
db.query('select 1') #=> [{ "1" => 1 }]
|
135
154
|
|
136
|
-
|
137
|
-
db.query_single_column 'select x from foo'
|
138
|
-
#=> [1, 4]
|
155
|
+
db.query_ary('select 1') #=> [[1]]
|
139
156
|
|
140
|
-
|
141
|
-
|
142
|
-
|
157
|
+
db.query_argv('select 1') #=> [1]
|
158
|
+
```
|
159
|
+
|
160
|
+
Notice how all the return values above are arrays. This is because the different
|
161
|
+
`#query_xxx` methods are designed to return multiple rows. If you want to just
|
162
|
+
get back a single row, use one of the `query_single_xxx` methods:
|
163
|
+
|
164
|
+
```ruby
|
165
|
+
# alias #query_single_hash
|
166
|
+
db.query('select 1') #=> { "1" => 1 }
|
167
|
+
|
168
|
+
db.query_single_ary('select 1') #=> [1]
|
143
169
|
|
144
|
-
|
145
|
-
db.query_single_value 'select z from foo order by x desc limit 1'
|
146
|
-
#=> 6
|
170
|
+
db.query_single_argv('select 1') #=> 1
|
147
171
|
```
|
148
172
|
|
149
173
|
## Parameter binding
|
@@ -157,14 +181,16 @@ statement:
|
|
157
181
|
db.query('select x from my_table where y = ? and z = ?', 'foo', 'bar')
|
158
182
|
```
|
159
183
|
|
160
|
-
You can also express place holders by specifying their index (starting from 1)
|
184
|
+
You can also express place holders by specifying their index (starting from 1)
|
185
|
+
using `?IDX`:
|
161
186
|
|
162
187
|
```ruby
|
163
188
|
# use the same value for both place holders:
|
164
189
|
db.query('select x from my_table where y = ?1 and z = ?1', 42)
|
165
190
|
```
|
166
191
|
|
167
|
-
Another possibility is to use named parameters, which can be done by expressing
|
192
|
+
Another possibility is to use named parameters, which can be done by expressing
|
193
|
+
place holders as `:KEY`, `@KEY` or `$KEY`:
|
168
194
|
|
169
195
|
```ruby
|
170
196
|
db.query('select x from my_table where y = $y and z = $z', y: 'foo', z: 'bar')
|
@@ -223,6 +249,38 @@ db.execute(sql, Extralite::Blob.new('Hello, 世界!'))
|
|
223
249
|
db.execute(sql, 'Hello, 世界!'.force_encoding(Encoding::ASCII_8BIT))
|
224
250
|
```
|
225
251
|
|
252
|
+
## Value Transforms
|
253
|
+
|
254
|
+
Extralite allows you to transform rows to any value your application may need by
|
255
|
+
providing a transform proc that takes the raw row values and returns the
|
256
|
+
transformed data. The transform proc is passed each resulting row either as a
|
257
|
+
hash or as a list of values.
|
258
|
+
|
259
|
+
Transforms are useful when you need to transform rows into ORM model instances,
|
260
|
+
or when you need to do some other transformation on the values retrieved from
|
261
|
+
the database.
|
262
|
+
|
263
|
+
To transform results, pass a transform proc as the first parameter to one of the
|
264
|
+
`#query_xxx` methods:
|
265
|
+
|
266
|
+
```ruby
|
267
|
+
transform = ->(h) { MyModel.new(h) }
|
268
|
+
db.query(transform, 'select * from foo')
|
269
|
+
#=> rows as instances of MyModel
|
270
|
+
```
|
271
|
+
|
272
|
+
When using the `argv` mode, the different column values are passed as individual
|
273
|
+
values to the transform proc:
|
274
|
+
|
275
|
+
```ruby
|
276
|
+
transform = ->(a, b, c) { { a:a, b: b, c: JSON.parse(c) } }
|
277
|
+
db.query_argv(transform, 'select a, b, c from foo')
|
278
|
+
#=> transformed rows
|
279
|
+
```
|
280
|
+
|
281
|
+
Value transforms can also be done with [prepared
|
282
|
+
queries](#value-transforms-in-prepared-queries).
|
283
|
+
|
226
284
|
## Prepared Queries
|
227
285
|
|
228
286
|
Prepared queries (also known as prepared statements) allow you to maximize
|
@@ -240,7 +298,8 @@ query.bind(1).to_a
|
|
240
298
|
|
241
299
|
### Binding Values to Prepared Queries
|
242
300
|
|
243
|
-
To bind parameter values to the query, use the `#bind` method. The parameters
|
301
|
+
To bind parameter values to the query, use the `#bind` method. The parameters
|
302
|
+
will remain bound to the query until `#bind` is called again.
|
244
303
|
|
245
304
|
```ruby
|
246
305
|
query.bind(1)
|
@@ -261,23 +320,33 @@ query = db.prepare('select * from foo where x = ?', 1)
|
|
261
320
|
|
262
321
|
### Fetching Records from a Prepared Query
|
263
322
|
|
264
|
-
Just like the `Database` interface, prepared queries
|
265
|
-
|
266
|
-
|
323
|
+
Just like the `Database` interface, prepared queries support getting data using
|
324
|
+
three different modes: as a hash, an array or as individual column values. To
|
325
|
+
set the mode, you can use one of the `#prepare_xxx` methods:
|
267
326
|
|
268
327
|
```ruby
|
269
|
-
#
|
270
|
-
|
328
|
+
# hash mode
|
329
|
+
db.prepare('select * from foo').to_a
|
330
|
+
#=> [{ x: 1, y: 2, z: 3}]
|
331
|
+
|
332
|
+
# argv mode
|
333
|
+
db.prepare_argv('select x from foo').to_a
|
334
|
+
#=> [1]
|
335
|
+
|
336
|
+
# ary mode
|
337
|
+
db.prepare_ary('select * from foo').to_a
|
271
338
|
#=> [[1, 2, 3]]
|
339
|
+
```
|
272
340
|
|
273
|
-
|
274
|
-
query2 = db.prepare('select z from foo where x = ?', 1)
|
275
|
-
query2.to_a_single_column
|
276
|
-
#=> [3]
|
341
|
+
You can also set the query mode by getting or setting `#mode`:
|
277
342
|
|
278
|
-
|
279
|
-
|
280
|
-
#=> 3
|
343
|
+
```ruby
|
344
|
+
q = db.prepare('select * from foo')
|
345
|
+
q.to_a #=> [{ x: 1, y: 2, z: 3}]
|
346
|
+
|
347
|
+
q.mode #=> :hash
|
348
|
+
q.mode = :ary
|
349
|
+
q.to_a "=> [[1, 2, 3]]
|
281
350
|
```
|
282
351
|
|
283
352
|
### Fetching Single Records or Batches of Records
|
@@ -299,12 +368,12 @@ query.next(10)
|
|
299
368
|
#=> [{ x: 1, y: 2, z: 3 }, { x: 4, y: 5, z: 6 }]
|
300
369
|
|
301
370
|
# Fetch the next row as an array
|
302
|
-
query.
|
303
|
-
query.
|
371
|
+
query = db.prepare_ary('select * from foo')
|
372
|
+
query.next
|
304
373
|
#=> [1, 2, 3]
|
305
374
|
|
306
375
|
# Fetch the next row as a single column
|
307
|
-
db.
|
376
|
+
db.prepare_argv('select z from foo').next
|
308
377
|
#=> 3
|
309
378
|
```
|
310
379
|
|
@@ -327,13 +396,16 @@ using the familiar `#each` method:
|
|
327
396
|
|
328
397
|
```ruby
|
329
398
|
# iterate over records as hashes
|
399
|
+
query = db.prepare('select * from foo')
|
330
400
|
query.each { |r| ... }
|
331
401
|
|
332
402
|
# iterate over records as arrays
|
333
|
-
query.
|
403
|
+
query = db.prepare_ary('select * from foo')
|
404
|
+
query.each { |r| ... }
|
334
405
|
|
335
406
|
# iterate over records as single values
|
336
|
-
query.
|
407
|
+
query = db.prepare_argv('select a, b, c from foo')
|
408
|
+
query.each { |a, b, c| ... }
|
337
409
|
```
|
338
410
|
|
339
411
|
### Prepared Query as an Enumerable
|
@@ -356,9 +428,33 @@ methods, such as `#map`, `#select`, `#inject`, `#lazy` etc. You can also
|
|
356
428
|
instantiate an iterator explicitly:
|
357
429
|
|
358
430
|
```ruby
|
359
|
-
# You need to pass the query to iterate over
|
360
|
-
|
361
|
-
iterator
|
431
|
+
# You need to pass the query to iterate over:
|
432
|
+
iterator = Extralite::Iterator(query)
|
433
|
+
iterator.each { |r| ... }
|
434
|
+
```
|
435
|
+
|
436
|
+
### Value Transforms in Prepared Queries
|
437
|
+
|
438
|
+
Prepared queries can automatically transform their result sets by setting a
|
439
|
+
transform block. The transform block receives values according to the query mode
|
440
|
+
(hash, array or argv). To set a transform you can pass a block to one of the
|
441
|
+
`Database#prepare_xxx` methods, or use `Query#transform`:
|
442
|
+
|
443
|
+
```ruby
|
444
|
+
q = db.prepare('select * from items where id = ?') { |h| Item.new(h) }
|
445
|
+
q.bind(42).next #=> Item instance
|
446
|
+
|
447
|
+
# An equivalent
|
448
|
+
q = db.prepare('select * from items where id = ?')
|
449
|
+
q.transform { |h| Item.new(h) }
|
450
|
+
```
|
451
|
+
|
452
|
+
The same can be done for queries in `argv` or `ary` mode:
|
453
|
+
|
454
|
+
```ruby
|
455
|
+
db.prepare_argv('select * from foo') { |a, b, c| a + b + c }
|
456
|
+
|
457
|
+
db.prepare_ary('select * from foo') { |a| a.map(&:to_s).join }
|
362
458
|
```
|
363
459
|
|
364
460
|
## Batch Execution of Queries
|
@@ -435,9 +531,8 @@ end
|
|
435
531
|
#=> 2
|
436
532
|
```
|
437
533
|
|
438
|
-
|
439
|
-
|
440
|
-
single values.
|
534
|
+
The `#batch_query` method, like other row fetching methods, changes the row
|
535
|
+
representation according to the query mode.
|
441
536
|
|
442
537
|
### Batch Execution of Prepared Queries
|
443
538
|
|
@@ -597,6 +692,12 @@ db.pragma(:journal_mode)
|
|
597
692
|
#=> 'wal'
|
598
693
|
```
|
599
694
|
|
695
|
+
You can also pass pragmas when opening the database:
|
696
|
+
|
697
|
+
```ruby
|
698
|
+
db = Extralite::Database.new('path/to/db', pragma: { foreign_keys: true })
|
699
|
+
```
|
700
|
+
|
600
701
|
## Error Handling
|
601
702
|
|
602
703
|
Extralite defines various exception classes that are raised when an error is
|
@@ -626,6 +727,18 @@ Extralite provides a comprehensive set of tools for dealing with concurrency
|
|
626
727
|
issues, and for making sure that running queries on SQLite databases does not
|
627
728
|
cause the app to freeze.
|
628
729
|
|
730
|
+
**Note**: In order to allow concurrent access your the database, it is highly
|
731
|
+
recommended that you set your database to use [WAL journaling
|
732
|
+
mode](https://www.sqlite.org/wal.html) for *all* database connections.
|
733
|
+
Otherwise, you risking running into performance problems and having queries fail
|
734
|
+
with `BusyError` exceptions. You can easily open your database in WAL journaling
|
735
|
+
mode by passing a `wal: true` option:
|
736
|
+
|
737
|
+
```ruby
|
738
|
+
# This will set PRAGMA journal_mode=1 and PRAGMA synchronous=1
|
739
|
+
db = Extralite::Database.new('path/to/db', wal: true)
|
740
|
+
```
|
741
|
+
|
629
742
|
### The Ruby GVL
|
630
743
|
|
631
744
|
In order to support multi-threading, Extralite releases the [Ruby
|
@@ -701,9 +814,10 @@ db2.busy_timeout = 5
|
|
701
814
|
db2.transaction { }
|
702
815
|
```
|
703
816
|
|
704
|
-
|
705
|
-
|
706
|
-
|
817
|
+
As stated above, setting the database to use WAL journaling mode greatly reduces
|
818
|
+
contention between different process/threads accessing the same database. For
|
819
|
+
most use cases, setting the busy timeout solves the problem of failing to run
|
820
|
+
queries because of a busy database, as normally transactions are short-lived.
|
707
821
|
|
708
822
|
However, in some cases, such as when running a multi-fibered app or when
|
709
823
|
implementing your own timeout mechanisms, you'll want to set a [progress
|
@@ -711,7 +825,9 @@ handler](#the-progress-handler).
|
|
711
825
|
|
712
826
|
### Interrupting a Query
|
713
827
|
|
714
|
-
To interrupt an ongoing query, use the `#interrupt` method. Normally this is
|
828
|
+
To interrupt an ongoing query, use the `#interrupt` method. Normally this is
|
829
|
+
done from a separate thread. Here's a way to implement a timeout using
|
830
|
+
`#interrupt`:
|
715
831
|
|
716
832
|
```ruby
|
717
833
|
def run_query_with_timeout(sql, timeout)
|
@@ -762,8 +878,7 @@ that specifies the approximate number of SQLite VM instructions between
|
|
762
878
|
successive calls to the progress handler:
|
763
879
|
|
764
880
|
```ruby
|
765
|
-
|
766
|
-
db.on_progress(100) do
|
881
|
+
db.on_progress do
|
767
882
|
check_for_timeout
|
768
883
|
# Allow other threads to run
|
769
884
|
Thread.pass
|
@@ -776,7 +891,7 @@ above, calling `#interrupt` causes the query to raise a
|
|
776
891
|
`Extralite::InterruptError` exception:
|
777
892
|
|
778
893
|
```ruby
|
779
|
-
db.on_progress(
|
894
|
+
db.on_progress(period: 1) { db.interrupt }
|
780
895
|
db.query('select 1')
|
781
896
|
#=> Extralite::InterruptError!
|
782
897
|
```
|
@@ -785,7 +900,7 @@ You can also interrupt queries in progress by raising an exception. The query
|
|
785
900
|
will be stopped, and the exception will propagate to the call site:
|
786
901
|
|
787
902
|
```ruby
|
788
|
-
db.on_progress(
|
903
|
+
db.on_progress(period: 1) do
|
789
904
|
raise 'BOOM!'
|
790
905
|
end
|
791
906
|
|
@@ -797,7 +912,7 @@ Here's how a timeout might be implemented using the progress handler:
|
|
797
912
|
|
798
913
|
```ruby
|
799
914
|
def setup_progress_handler
|
800
|
-
@db.on_progress
|
915
|
+
@db.on_progress do
|
801
916
|
raise TimeoutError if Time.now - @t0 >= @timeout
|
802
917
|
Thread.pass
|
803
918
|
end
|
@@ -818,6 +933,65 @@ run_query_with_timeout(slow_sql, 5)
|
|
818
933
|
#=> nil
|
819
934
|
```
|
820
935
|
|
936
|
+
**Note**: you must not issue any query from within the progress handler.
|
937
|
+
|
938
|
+
### Dealing with a Busy Database in the Progress Handler
|
939
|
+
|
940
|
+
As mentioned above, the progress handler is also called when the database is
|
941
|
+
busy, regardless of the progress period given to `#on_progress`. You can detect
|
942
|
+
if the database is busy by checking the first argument passed to the progress
|
943
|
+
handler, which will be true when busy:
|
944
|
+
|
945
|
+
```ruby
|
946
|
+
db.on_progress do |busy|
|
947
|
+
if busy
|
948
|
+
foo
|
949
|
+
else
|
950
|
+
bar
|
951
|
+
end
|
952
|
+
end
|
953
|
+
```
|
954
|
+
|
955
|
+
This allows you to implement separate logic to deal with busy states, for
|
956
|
+
example sleeping for a small period of time, or implementing a different timeout
|
957
|
+
period.
|
958
|
+
|
959
|
+
### Advanced Progress Handler Settings
|
960
|
+
|
961
|
+
You can further tune the behaviour of the progress handler with the following
|
962
|
+
options passed to `#on_progress`:
|
963
|
+
|
964
|
+
- `:mode`: the following modes are supported:
|
965
|
+
- `:none` : the progress handler is disabled.
|
966
|
+
- `:normal`: the progress handler is called on query progress (this is the
|
967
|
+
default mode).
|
968
|
+
- `:once`: the progress handler is called once before running the query.
|
969
|
+
- `:at_least_once`: the progress handler is called once before running the
|
970
|
+
query, and on query progress.
|
971
|
+
- `:period`: controls the approximate number of SQLite VM instructions executed
|
972
|
+
between consecutive calls to the progress handler. Default value: 1000.
|
973
|
+
- `:tick`: controls the granularity of the progress handler. This is the value
|
974
|
+
passed internally to the SQLite library. Default value: 10.
|
975
|
+
|
976
|
+
```ruby
|
977
|
+
db.on_progress(mode: :at_least_once, period: 640, tick: 5) { snooze }
|
978
|
+
```
|
979
|
+
|
980
|
+
### Global Progress Handler Settings
|
981
|
+
|
982
|
+
You can set the global progress handler behaviour by calling
|
983
|
+
`Extralite.on_progress`. You can use this API to set the global progress
|
984
|
+
settings, without needing to set a progress handler individually for each
|
985
|
+
`Database` instance. This method takes the same options as
|
986
|
+
`Database#on_progress`:
|
987
|
+
|
988
|
+
```ruby
|
989
|
+
Extralite.on_progress(mode: :at_least_once, period: 640, tick: 5) { snooze }
|
990
|
+
|
991
|
+
# the new database instance uses the global progress handler settings
|
992
|
+
db = Database.new(':memory:')
|
993
|
+
```
|
994
|
+
|
821
995
|
### Extralite and Fibers
|
822
996
|
|
823
997
|
The progress handler can also be used to switch between fibers in a
|
@@ -829,7 +1003,7 @@ handler. This will work for switching between fibers using either Polyphony or
|
|
829
1003
|
any fiber scheduler gem, such as Async et al:
|
830
1004
|
|
831
1005
|
```ruby
|
832
|
-
db.on_progress
|
1006
|
+
db.on_progress { sleep(0) }
|
833
1007
|
```
|
834
1008
|
|
835
1009
|
For Polyphony-based apps, you can also call `snooze` to allow other fibers to
|
@@ -837,7 +1011,7 @@ run while a query is progressing. If your Polyphony app is multi-threaded,
|
|
837
1011
|
you'll also need to call `Thread.pass` in order to allow other threads to run:
|
838
1012
|
|
839
1013
|
```ruby
|
840
|
-
db.on_progress
|
1014
|
+
db.on_progress do
|
841
1015
|
snooze
|
842
1016
|
Thread.pass
|
843
1017
|
end
|
@@ -848,7 +1022,7 @@ use the regular `#move_on_after` and `#cancel_after` methods to implement
|
|
848
1022
|
timeouts for queries:
|
849
1023
|
|
850
1024
|
```ruby
|
851
|
-
db.on_progress
|
1025
|
+
db.on_progress { snooze }
|
852
1026
|
|
853
1027
|
cancel_after(3) do
|
854
1028
|
db.query(long_running_query)
|
@@ -868,7 +1042,14 @@ as long as the following conditions are met:
|
|
868
1042
|
|
869
1043
|
### Use with Ractors
|
870
1044
|
|
871
|
-
Extralite databases can safely be used inside ractors.
|
1045
|
+
Extralite databases can safely be used inside ractors. A ractor has the benefit
|
1046
|
+
of using a separate GVL from the maine one, which allows true parallelism for
|
1047
|
+
Ruby apps. So when you use Extralite to access SQLite databases from within a
|
1048
|
+
ractor, you can do so without any considerations for what's happening outside
|
1049
|
+
the ractor when it runs queries.
|
1050
|
+
|
1051
|
+
**Note**: Ractors are considered an experimental feature of Ruby. You may
|
1052
|
+
encounter errors or inconsistent behaviour when using ractors.
|
872
1053
|
|
873
1054
|
## Advanced Usage
|
874
1055
|
|
@@ -1027,7 +1208,8 @@ large number of rows.
|
|
1027
1208
|
|
1028
1209
|
### Rows as Hashes
|
1029
1210
|
|
1030
|
-
|
1211
|
+
[Benchmark source
|
1212
|
+
code](https://github.com/digital-fabric/extralite/blob/main/test/perf_hash.rb)
|
1031
1213
|
|
1032
1214
|
|Row count|sqlite3 1.7.0|Extralite 2.5|Advantage|
|
1033
1215
|
|-:|-:|-:|-:|
|
@@ -1037,7 +1219,8 @@ large number of rows.
|
|
1037
1219
|
|
1038
1220
|
### Rows as Arrays
|
1039
1221
|
|
1040
|
-
[Benchmark source
|
1222
|
+
[Benchmark source
|
1223
|
+
code](https://github.com/digital-fabric/extralite/blob/main/test/perf_ary.rb)
|
1041
1224
|
|
1042
1225
|
|Row count|sqlite3 1.7.0|Extralite 2.5|Advantage|
|
1043
1226
|
|-:|-:|-:|-:|
|
@@ -1047,7 +1230,8 @@ large number of rows.
|
|
1047
1230
|
|
1048
1231
|
### Prepared Queries (Prepared Statements)
|
1049
1232
|
|
1050
|
-
[Benchmark source
|
1233
|
+
[Benchmark source
|
1234
|
+
code](https://github.com/digital-fabric/extralite/blob/main/test/perf_hash_prepared.rb)
|
1051
1235
|
|
1052
1236
|
|Row count|sqlite3 1.7.0|Extralite 2.5|Advantage|
|
1053
1237
|
|-:|-:|-:|-:|
|
@@ -1058,7 +1242,10 @@ large number of rows.
|
|
1058
1242
|
As those benchmarks show, Extralite is capabale of reading up to 2.4M rows per
|
1059
1243
|
second, and can be more than 14 times faster than the `sqlite3` gem.
|
1060
1244
|
|
1061
|
-
Note that the benchmarks above were performed on synthetic data, in a
|
1245
|
+
Note that the benchmarks above were performed on synthetic data, in a
|
1246
|
+
single-threaded environment, with the GVL release threshold set to -1, which
|
1247
|
+
means that both Extralite and the `sqlite3` gem hold the GVL for the duration of
|
1248
|
+
the query.
|
1062
1249
|
|
1063
1250
|
## License
|
1064
1251
|
|