extralite 2.6 → 2.7.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitignore +1 -0
- data/.yardopts +1 -1
- data/CHANGELOG.md +32 -17
- data/Gemfile +4 -0
- data/Gemfile-bundle +1 -1
- data/README.md +262 -75
- data/Rakefile +18 -0
- data/TODO.md +0 -9
- data/examples/kv_store.rb +49 -0
- data/examples/multi_fiber.rb +16 -0
- data/examples/on_progress.rb +9 -0
- data/examples/pubsub_store_polyphony.rb +194 -0
- data/examples/pubsub_store_threads.rb +204 -0
- data/ext/extralite/changeset.c +3 -3
- data/ext/extralite/common.c +173 -87
- data/ext/extralite/database.c +650 -315
- data/ext/extralite/extconf.rb +7 -11
- data/ext/extralite/extralite.h +89 -48
- data/ext/extralite/iterator.c +6 -84
- data/ext/extralite/query.c +165 -256
- data/extralite-bundle.gemspec +1 -1
- data/extralite.gemspec +1 -1
- data/gemspec.rb +10 -11
- data/lib/extralite/version.rb +1 -1
- data/lib/extralite.rb +27 -17
- data/lib/sequel/adapters/extralite.rb +1 -1
- data/test/helper.rb +2 -1
- data/test/perf_argv_transform.rb +74 -0
- data/test/perf_hash_transform.rb +66 -0
- data/test/perf_polyphony.rb +74 -0
- data/test/test_changeset.rb +2 -2
- data/test/test_database.rb +531 -115
- data/test/test_extralite.rb +2 -2
- data/test/test_iterator.rb +28 -13
- data/test/test_query.rb +348 -111
- data/test/test_sequel.rb +4 -4
- metadata +20 -14
- data/Gemfile.lock +0 -37
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 2cbb8c1507e23141a2b069306e31939b7cd5fea83e6941426972d7eeadc86ac0
|
4
|
+
data.tar.gz: e283cd0d2b18ed840020d73fb71d47696b64b8ff99c42a2a431928b6a25db189
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: cbe1b66c813d0e35bd667a06e14bc68673655807d256771dac231ecb2f559660a05773267ef804e40f9d747482974d897b78c0df370340c55e7cfd290bdc4a41
|
7
|
+
data.tar.gz: 385b3ec651b1c2365392c7a1dddb0b7b63adddc6ade662f8ba7d72d6d49f025dff0b55490df6296181d361ac5792e888ed7db90c8b11b2129b5d6474fae2c8de
|
data/.gitignore
CHANGED
data/.yardopts
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,32 +1,47 @@
|
|
1
|
+
# 2.7.1 2024-02-11
|
2
|
+
|
3
|
+
- Fix API docs.
|
4
|
+
|
5
|
+
# 2.7 2024-02-09
|
6
|
+
|
7
|
+
- Improve progress handler API, add mode, period, tick options, global progress
|
8
|
+
handler. [#68](https://github.com/digital-fabric/extralite/pull/68)
|
9
|
+
- Rework `Database#initialize` options
|
10
|
+
- Add argv row mode (for passing column values as argv)
|
11
|
+
- Streamline and improve query methods
|
12
|
+
[#67](https://github.com/digital-fabric/extralite/pull/67)
|
13
|
+
- Implement row transforms
|
14
|
+
|
1
15
|
# 2.6 2024-01-23
|
2
16
|
|
3
|
-
- Implement changeset API
|
17
|
+
- Implement changeset API.
|
4
18
|
[#58](https://github.com/digital-fabric/extralite/issues/58)
|
5
|
-
- Reorganize README, update benchmarks
|
19
|
+
- Reorganize README, update benchmarks.
|
6
20
|
[#63](https://github.com/digital-fabric/extralite/issues/63)
|
7
|
-
- Implement progress handler API
|
21
|
+
- Implement progress handler API.
|
8
22
|
[#62](https://github.com/digital-fabric/extralite/issues/62)
|
9
|
-
- Implement savepoint methods
|
23
|
+
- Implement savepoint methods.
|
10
24
|
|
11
25
|
# 2.5 2024-01-16
|
12
26
|
|
13
|
-
- Update bundled sqlite to version 3.45.0
|
14
|
-
- Implement `Database#batch_query` and related methods
|
27
|
+
- Update bundled sqlite to version 3.45.0.
|
28
|
+
- Implement `Database#batch_query` and related methods.
|
15
29
|
[53](https://github.com/digital-fabric/extralite/issues/53)
|
16
|
-
- Accept more options in `Database#initialize
|
30
|
+
- Accept more options in `Database#initialize`.
|
17
31
|
[48](https://github.com/digital-fabric/extralite/issues/48)
|
18
|
-
- Fix `Database#pragma` to return single value when reading pragma value
|
19
|
-
- Accept database name in `Database#tables` method
|
32
|
+
- Fix `Database#pragma` to return single value when reading pragma value.
|
33
|
+
- Accept database name in `Database#tables` method.
|
20
34
|
- Improve `Database#batch_execute` - now accepts Enumerable and Callable
|
21
|
-
parameters [52](https://github.com/digital-fabric/extralite/issues/52)
|
22
|
-
- Rename `Database#execute_multi` to `Database#batch_execute
|
23
|
-
- Implement Query#clone
|
35
|
+
parameters. [52](https://github.com/digital-fabric/extralite/issues/52)
|
36
|
+
- Rename `Database#execute_multi` to `Database#batch_execute`.
|
37
|
+
- Implement `Query#clone`.
|
24
38
|
[51](https://github.com/digital-fabric/extralite/issues/51)
|
25
|
-
- Add support for GC compaction
|
26
|
-
- Remove support for Ruby 2.7
|
27
|
-
- Implement Query
|
28
|
-
-
|
29
|
-
-
|
39
|
+
- Add support for GC compaction.
|
40
|
+
- Remove support for Ruby 2.7.
|
41
|
+
- Implement `Query#<<`.
|
42
|
+
[49](https://github.com/digital-fabric/extralite/issues/49)
|
43
|
+
- Allow passing parameters in array.
|
44
|
+
- Add support for ractors.
|
30
45
|
[#50](https://github.com/digital-fabric/extralite/issues/50)
|
31
46
|
|
32
47
|
# 2.4 2023-12-24
|
data/Gemfile
CHANGED
data/Gemfile-bundle
CHANGED
@@ -2,4 +2,4 @@ source 'https://rubygems.org'
|
|
2
2
|
|
3
3
|
gemspec name: 'extralite'
|
4
4
|
|
5
|
-
gem
|
5
|
+
gem 'extralite-bundle', git: "file://#{File.expand_path(__dir__)}", ref: `git rev-parse --abbrev-ref HEAD`.strip
|
data/README.md
CHANGED
@@ -3,7 +3,7 @@
|
|
3
3
|
Extralite
|
4
4
|
</h1>
|
5
5
|
|
6
|
-
<h4 align="center">
|
6
|
+
<h4 align="center">Ruby on SQLite</h4>
|
7
7
|
|
8
8
|
<p align="center">
|
9
9
|
<a href="http://rubygems.org/gems/extralite">
|
@@ -26,7 +26,7 @@
|
|
26
26
|
Extralite is a fast and innovative SQLite wrapper for Ruby with a rich set of
|
27
27
|
features. It provides multiple ways of retrieving data from SQLite databases,
|
28
28
|
makes it possible to use SQLite databases in multi-threaded and multi-fibered
|
29
|
-
Ruby apps, and
|
29
|
+
Ruby apps, and includes a comprehensive set of tools for managing SQLite
|
30
30
|
databases.
|
31
31
|
|
32
32
|
Extralite comes in two flavors: the `extralite` gem which uses the
|
@@ -37,27 +37,33 @@ latest features and enhancements.
|
|
37
37
|
|
38
38
|
## Features
|
39
39
|
|
40
|
-
- Best-in-class performance (up to 14X the performance of the
|
40
|
+
- Best-in-class [performance](#performance) (up to 14X the performance of the
|
41
41
|
[sqlite3](https://github.com/sparklemotion/sqlite3-ruby) gem).
|
42
42
|
- Support for [concurrency](#concurrency) out of the box for multi-threaded
|
43
|
-
apps.
|
44
|
-
- A variety of
|
45
|
-
|
46
|
-
- Support for external iteration
|
47
|
-
batches of records.
|
48
|
-
- Prepared queries.
|
49
|
-
- Parameter binding.
|
50
|
-
- Batch execution of queries.
|
51
|
-
-
|
52
|
-
-
|
53
|
-
-
|
54
|
-
-
|
43
|
+
and multi-fibered apps.
|
44
|
+
- A variety of ways to [retrieve data](#query-modes) - hashes, arrays, single
|
45
|
+
columns, single rows, [transforms](#value-transforms).
|
46
|
+
- Support for [external iteration](#iterating-over-records-in-a-prepared-query),
|
47
|
+
allowing iterating through single records or batches of records.
|
48
|
+
- [Prepared queries](#prepared-queries).
|
49
|
+
- [Parameter binding](#parameter-binding).
|
50
|
+
- [Batch execution](#batch-execution-of-queries) of queries.
|
51
|
+
- [transactions and savepoints](#transactions-and-savepoints).
|
52
|
+
- Advanced features: load [SQLite extensions](#loading-extensions), create
|
53
|
+
[backups](#creating-backups), retrieve [status
|
54
|
+
information](#retrieving-status-information), work with
|
55
|
+
[changesets](#working-with-changesets), interrogate [database
|
56
|
+
limits](#working-with-database-limits), [trace](#tracing-sql-statements)
|
57
|
+
queries.
|
58
|
+
- [Sequel](#usage-with-sequel) adapter.
|
55
59
|
|
56
60
|
## Table of Content
|
57
61
|
|
58
62
|
- [Installing Extralite](#installing-extralite)
|
59
|
-
- [
|
63
|
+
- [Getting Started](#getting-started)
|
64
|
+
- [Query Modes](#query-modes)
|
60
65
|
- [Parameter binding](#parameter-binding)
|
66
|
+
- [Value Transforms](#value-transforms)
|
61
67
|
- [Data Types](#data-types)
|
62
68
|
- [Prepared Queries](#prepared-queries)
|
63
69
|
- [Batch Execution of Queries](#batch-execution-of-queries)
|
@@ -97,10 +103,9 @@ SQLite source code.
|
|
97
103
|
Usage of the `extralite-bundle` gem is identical to the usage of the normal
|
98
104
|
`extralite` gem, using `require 'extralite'` to load the gem.
|
99
105
|
|
100
|
-
##
|
106
|
+
## Getting Started
|
101
107
|
|
102
|
-
|
103
|
-
queries:
|
108
|
+
The following example shows how to open an SQLite database and run some queries:
|
104
109
|
|
105
110
|
```ruby
|
106
111
|
db = Extralite::Database.new('mydb.sqlite')
|
@@ -126,24 +131,43 @@ db.query 'select * from foo' do |r|
|
|
126
131
|
end
|
127
132
|
```
|
128
133
|
|
129
|
-
|
134
|
+
## Query Modes
|
135
|
+
|
136
|
+
Extralite allows you to retrieve data from SQLite database in the form that most
|
137
|
+
a particular context. For some use cases you'll want to work with rows as
|
138
|
+
hashes. In other cases, you'll want to work with rows as arrays, or even as
|
139
|
+
single values, if you're just reading one column.
|
140
|
+
|
141
|
+
For that purpose, Extralite offers three different ways, or modes, of retrieving
|
142
|
+
records:
|
143
|
+
|
144
|
+
- `:hash`: read rows as hashes (this is the default mode).
|
145
|
+
- `:ary`: read rows as arrays.
|
146
|
+
- `:argv`: similar to the `:ary`, except that for queries with a single column,
|
147
|
+
the single column value is returned.
|
148
|
+
|
149
|
+
Extralite provides separate methods for the different modes:
|
130
150
|
|
131
151
|
```ruby
|
132
|
-
#
|
133
|
-
db.
|
134
|
-
#=> [[1, 2, 3], [4, 5, 6]]
|
152
|
+
# alias #query_hash
|
153
|
+
db.query('select 1') #=> [{ "1" => 1 }]
|
135
154
|
|
136
|
-
|
137
|
-
db.query_single_column 'select x from foo'
|
138
|
-
#=> [1, 4]
|
155
|
+
db.query_ary('select 1') #=> [[1]]
|
139
156
|
|
140
|
-
|
141
|
-
|
142
|
-
|
157
|
+
db.query_argv('select 1') #=> [1]
|
158
|
+
```
|
159
|
+
|
160
|
+
Notice how all the return values above are arrays. This is because the different
|
161
|
+
`#query_xxx` methods are designed to return multiple rows. If you want to just
|
162
|
+
get back a single row, use one of the `query_single_xxx` methods:
|
163
|
+
|
164
|
+
```ruby
|
165
|
+
# alias #query_single_hash
|
166
|
+
db.query('select 1') #=> { "1" => 1 }
|
167
|
+
|
168
|
+
db.query_single_ary('select 1') #=> [1]
|
143
169
|
|
144
|
-
|
145
|
-
db.query_single_value 'select z from foo order by x desc limit 1'
|
146
|
-
#=> 6
|
170
|
+
db.query_single_argv('select 1') #=> 1
|
147
171
|
```
|
148
172
|
|
149
173
|
## Parameter binding
|
@@ -157,14 +181,16 @@ statement:
|
|
157
181
|
db.query('select x from my_table where y = ? and z = ?', 'foo', 'bar')
|
158
182
|
```
|
159
183
|
|
160
|
-
You can also express place holders by specifying their index (starting from 1)
|
184
|
+
You can also express place holders by specifying their index (starting from 1)
|
185
|
+
using `?IDX`:
|
161
186
|
|
162
187
|
```ruby
|
163
188
|
# use the same value for both place holders:
|
164
189
|
db.query('select x from my_table where y = ?1 and z = ?1', 42)
|
165
190
|
```
|
166
191
|
|
167
|
-
Another possibility is to use named parameters, which can be done by expressing
|
192
|
+
Another possibility is to use named parameters, which can be done by expressing
|
193
|
+
place holders as `:KEY`, `@KEY` or `$KEY`:
|
168
194
|
|
169
195
|
```ruby
|
170
196
|
db.query('select x from my_table where y = $y and z = $z', y: 'foo', z: 'bar')
|
@@ -223,6 +249,38 @@ db.execute(sql, Extralite::Blob.new('Hello, 世界!'))
|
|
223
249
|
db.execute(sql, 'Hello, 世界!'.force_encoding(Encoding::ASCII_8BIT))
|
224
250
|
```
|
225
251
|
|
252
|
+
## Value Transforms
|
253
|
+
|
254
|
+
Extralite allows you to transform rows to any value your application may need by
|
255
|
+
providing a transform proc that takes the raw row values and returns the
|
256
|
+
transformed data. The transform proc is passed each resulting row either as a
|
257
|
+
hash or as a list of values.
|
258
|
+
|
259
|
+
Transforms are useful when you need to transform rows into ORM model instances,
|
260
|
+
or when you need to do some other transformation on the values retrieved from
|
261
|
+
the database.
|
262
|
+
|
263
|
+
To transform results, pass a transform proc as the first parameter to one of the
|
264
|
+
`#query_xxx` methods:
|
265
|
+
|
266
|
+
```ruby
|
267
|
+
transform = ->(h) { MyModel.new(h) }
|
268
|
+
db.query(transform, 'select * from foo')
|
269
|
+
#=> rows as instances of MyModel
|
270
|
+
```
|
271
|
+
|
272
|
+
When using the `argv` mode, the different column values are passed as individual
|
273
|
+
values to the transform proc:
|
274
|
+
|
275
|
+
```ruby
|
276
|
+
transform = ->(a, b, c) { { a:a, b: b, c: JSON.parse(c) } }
|
277
|
+
db.query_argv(transform, 'select a, b, c from foo')
|
278
|
+
#=> transformed rows
|
279
|
+
```
|
280
|
+
|
281
|
+
Value transforms can also be done with [prepared
|
282
|
+
queries](#value-transforms-in-prepared-queries).
|
283
|
+
|
226
284
|
## Prepared Queries
|
227
285
|
|
228
286
|
Prepared queries (also known as prepared statements) allow you to maximize
|
@@ -240,7 +298,8 @@ query.bind(1).to_a
|
|
240
298
|
|
241
299
|
### Binding Values to Prepared Queries
|
242
300
|
|
243
|
-
To bind parameter values to the query, use the `#bind` method. The parameters
|
301
|
+
To bind parameter values to the query, use the `#bind` method. The parameters
|
302
|
+
will remain bound to the query until `#bind` is called again.
|
244
303
|
|
245
304
|
```ruby
|
246
305
|
query.bind(1)
|
@@ -261,23 +320,33 @@ query = db.prepare('select * from foo where x = ?', 1)
|
|
261
320
|
|
262
321
|
### Fetching Records from a Prepared Query
|
263
322
|
|
264
|
-
Just like the `Database` interface, prepared queries
|
265
|
-
|
266
|
-
|
323
|
+
Just like the `Database` interface, prepared queries support getting data using
|
324
|
+
three different modes: as a hash, an array or as individual column values. To
|
325
|
+
set the mode, you can use one of the `#prepare_xxx` methods:
|
267
326
|
|
268
327
|
```ruby
|
269
|
-
#
|
270
|
-
|
328
|
+
# hash mode
|
329
|
+
db.prepare('select * from foo').to_a
|
330
|
+
#=> [{ x: 1, y: 2, z: 3}]
|
331
|
+
|
332
|
+
# argv mode
|
333
|
+
db.prepare_argv('select x from foo').to_a
|
334
|
+
#=> [1]
|
335
|
+
|
336
|
+
# ary mode
|
337
|
+
db.prepare_ary('select * from foo').to_a
|
271
338
|
#=> [[1, 2, 3]]
|
339
|
+
```
|
272
340
|
|
273
|
-
|
274
|
-
query2 = db.prepare('select z from foo where x = ?', 1)
|
275
|
-
query2.to_a_single_column
|
276
|
-
#=> [3]
|
341
|
+
You can also set the query mode by getting or setting `#mode`:
|
277
342
|
|
278
|
-
|
279
|
-
|
280
|
-
#=> 3
|
343
|
+
```ruby
|
344
|
+
q = db.prepare('select * from foo')
|
345
|
+
q.to_a #=> [{ x: 1, y: 2, z: 3}]
|
346
|
+
|
347
|
+
q.mode #=> :hash
|
348
|
+
q.mode = :ary
|
349
|
+
q.to_a "=> [[1, 2, 3]]
|
281
350
|
```
|
282
351
|
|
283
352
|
### Fetching Single Records or Batches of Records
|
@@ -299,12 +368,12 @@ query.next(10)
|
|
299
368
|
#=> [{ x: 1, y: 2, z: 3 }, { x: 4, y: 5, z: 6 }]
|
300
369
|
|
301
370
|
# Fetch the next row as an array
|
302
|
-
query.
|
303
|
-
query.
|
371
|
+
query = db.prepare_ary('select * from foo')
|
372
|
+
query.next
|
304
373
|
#=> [1, 2, 3]
|
305
374
|
|
306
375
|
# Fetch the next row as a single column
|
307
|
-
db.
|
376
|
+
db.prepare_argv('select z from foo').next
|
308
377
|
#=> 3
|
309
378
|
```
|
310
379
|
|
@@ -327,13 +396,16 @@ using the familiar `#each` method:
|
|
327
396
|
|
328
397
|
```ruby
|
329
398
|
# iterate over records as hashes
|
399
|
+
query = db.prepare('select * from foo')
|
330
400
|
query.each { |r| ... }
|
331
401
|
|
332
402
|
# iterate over records as arrays
|
333
|
-
query.
|
403
|
+
query = db.prepare_ary('select * from foo')
|
404
|
+
query.each { |r| ... }
|
334
405
|
|
335
406
|
# iterate over records as single values
|
336
|
-
query.
|
407
|
+
query = db.prepare_argv('select a, b, c from foo')
|
408
|
+
query.each { |a, b, c| ... }
|
337
409
|
```
|
338
410
|
|
339
411
|
### Prepared Query as an Enumerable
|
@@ -356,9 +428,33 @@ methods, such as `#map`, `#select`, `#inject`, `#lazy` etc. You can also
|
|
356
428
|
instantiate an iterator explicitly:
|
357
429
|
|
358
430
|
```ruby
|
359
|
-
# You need to pass the query to iterate over
|
360
|
-
|
361
|
-
iterator
|
431
|
+
# You need to pass the query to iterate over:
|
432
|
+
iterator = Extralite::Iterator(query)
|
433
|
+
iterator.each { |r| ... }
|
434
|
+
```
|
435
|
+
|
436
|
+
### Value Transforms in Prepared Queries
|
437
|
+
|
438
|
+
Prepared queries can automatically transform their result sets by setting a
|
439
|
+
transform block. The transform block receives values according to the query mode
|
440
|
+
(hash, array or argv). To set a transform you can pass a block to one of the
|
441
|
+
`Database#prepare_xxx` methods, or use `Query#transform`:
|
442
|
+
|
443
|
+
```ruby
|
444
|
+
q = db.prepare('select * from items where id = ?') { |h| Item.new(h) }
|
445
|
+
q.bind(42).next #=> Item instance
|
446
|
+
|
447
|
+
# An equivalent
|
448
|
+
q = db.prepare('select * from items where id = ?')
|
449
|
+
q.transform { |h| Item.new(h) }
|
450
|
+
```
|
451
|
+
|
452
|
+
The same can be done for queries in `argv` or `ary` mode:
|
453
|
+
|
454
|
+
```ruby
|
455
|
+
db.prepare_argv('select * from foo') { |a, b, c| a + b + c }
|
456
|
+
|
457
|
+
db.prepare_ary('select * from foo') { |a| a.map(&:to_s).join }
|
362
458
|
```
|
363
459
|
|
364
460
|
## Batch Execution of Queries
|
@@ -435,9 +531,8 @@ end
|
|
435
531
|
#=> 2
|
436
532
|
```
|
437
533
|
|
438
|
-
|
439
|
-
|
440
|
-
single values.
|
534
|
+
The `#batch_query` method, like other row fetching methods, changes the row
|
535
|
+
representation according to the query mode.
|
441
536
|
|
442
537
|
### Batch Execution of Prepared Queries
|
443
538
|
|
@@ -597,6 +692,12 @@ db.pragma(:journal_mode)
|
|
597
692
|
#=> 'wal'
|
598
693
|
```
|
599
694
|
|
695
|
+
You can also pass pragmas when opening the database:
|
696
|
+
|
697
|
+
```ruby
|
698
|
+
db = Extralite::Database.new('path/to/db', pragma: { foreign_keys: true })
|
699
|
+
```
|
700
|
+
|
600
701
|
## Error Handling
|
601
702
|
|
602
703
|
Extralite defines various exception classes that are raised when an error is
|
@@ -626,6 +727,18 @@ Extralite provides a comprehensive set of tools for dealing with concurrency
|
|
626
727
|
issues, and for making sure that running queries on SQLite databases does not
|
627
728
|
cause the app to freeze.
|
628
729
|
|
730
|
+
**Note**: In order to allow concurrent access your the database, it is highly
|
731
|
+
recommended that you set your database to use [WAL journaling
|
732
|
+
mode](https://www.sqlite.org/wal.html) for *all* database connections.
|
733
|
+
Otherwise, you risking running into performance problems and having queries fail
|
734
|
+
with `BusyError` exceptions. You can easily open your database in WAL journaling
|
735
|
+
mode by passing a `wal: true` option:
|
736
|
+
|
737
|
+
```ruby
|
738
|
+
# This will set PRAGMA journal_mode=1 and PRAGMA synchronous=1
|
739
|
+
db = Extralite::Database.new('path/to/db', wal: true)
|
740
|
+
```
|
741
|
+
|
629
742
|
### The Ruby GVL
|
630
743
|
|
631
744
|
In order to support multi-threading, Extralite releases the [Ruby
|
@@ -701,9 +814,10 @@ db2.busy_timeout = 5
|
|
701
814
|
db2.transaction { }
|
702
815
|
```
|
703
816
|
|
704
|
-
|
705
|
-
|
706
|
-
|
817
|
+
As stated above, setting the database to use WAL journaling mode greatly reduces
|
818
|
+
contention between different process/threads accessing the same database. For
|
819
|
+
most use cases, setting the busy timeout solves the problem of failing to run
|
820
|
+
queries because of a busy database, as normally transactions are short-lived.
|
707
821
|
|
708
822
|
However, in some cases, such as when running a multi-fibered app or when
|
709
823
|
implementing your own timeout mechanisms, you'll want to set a [progress
|
@@ -711,7 +825,9 @@ handler](#the-progress-handler).
|
|
711
825
|
|
712
826
|
### Interrupting a Query
|
713
827
|
|
714
|
-
To interrupt an ongoing query, use the `#interrupt` method. Normally this is
|
828
|
+
To interrupt an ongoing query, use the `#interrupt` method. Normally this is
|
829
|
+
done from a separate thread. Here's a way to implement a timeout using
|
830
|
+
`#interrupt`:
|
715
831
|
|
716
832
|
```ruby
|
717
833
|
def run_query_with_timeout(sql, timeout)
|
@@ -762,8 +878,7 @@ that specifies the approximate number of SQLite VM instructions between
|
|
762
878
|
successive calls to the progress handler:
|
763
879
|
|
764
880
|
```ruby
|
765
|
-
|
766
|
-
db.on_progress(100) do
|
881
|
+
db.on_progress do
|
767
882
|
check_for_timeout
|
768
883
|
# Allow other threads to run
|
769
884
|
Thread.pass
|
@@ -776,7 +891,7 @@ above, calling `#interrupt` causes the query to raise a
|
|
776
891
|
`Extralite::InterruptError` exception:
|
777
892
|
|
778
893
|
```ruby
|
779
|
-
db.on_progress(
|
894
|
+
db.on_progress(period: 1) { db.interrupt }
|
780
895
|
db.query('select 1')
|
781
896
|
#=> Extralite::InterruptError!
|
782
897
|
```
|
@@ -785,7 +900,7 @@ You can also interrupt queries in progress by raising an exception. The query
|
|
785
900
|
will be stopped, and the exception will propagate to the call site:
|
786
901
|
|
787
902
|
```ruby
|
788
|
-
db.on_progress(
|
903
|
+
db.on_progress(period: 1) do
|
789
904
|
raise 'BOOM!'
|
790
905
|
end
|
791
906
|
|
@@ -797,7 +912,7 @@ Here's how a timeout might be implemented using the progress handler:
|
|
797
912
|
|
798
913
|
```ruby
|
799
914
|
def setup_progress_handler
|
800
|
-
@db.on_progress
|
915
|
+
@db.on_progress do
|
801
916
|
raise TimeoutError if Time.now - @t0 >= @timeout
|
802
917
|
Thread.pass
|
803
918
|
end
|
@@ -818,6 +933,65 @@ run_query_with_timeout(slow_sql, 5)
|
|
818
933
|
#=> nil
|
819
934
|
```
|
820
935
|
|
936
|
+
**Note**: you must not issue any query from within the progress handler.
|
937
|
+
|
938
|
+
### Dealing with a Busy Database in the Progress Handler
|
939
|
+
|
940
|
+
As mentioned above, the progress handler is also called when the database is
|
941
|
+
busy, regardless of the progress period given to `#on_progress`. You can detect
|
942
|
+
if the database is busy by checking the first argument passed to the progress
|
943
|
+
handler, which will be true when busy:
|
944
|
+
|
945
|
+
```ruby
|
946
|
+
db.on_progress do |busy|
|
947
|
+
if busy
|
948
|
+
foo
|
949
|
+
else
|
950
|
+
bar
|
951
|
+
end
|
952
|
+
end
|
953
|
+
```
|
954
|
+
|
955
|
+
This allows you to implement separate logic to deal with busy states, for
|
956
|
+
example sleeping for a small period of time, or implementing a different timeout
|
957
|
+
period.
|
958
|
+
|
959
|
+
### Advanced Progress Handler Settings
|
960
|
+
|
961
|
+
You can further tune the behaviour of the progress handler with the following
|
962
|
+
options passed to `#on_progress`:
|
963
|
+
|
964
|
+
- `:mode`: the following modes are supported:
|
965
|
+
- `:none` : the progress handler is disabled.
|
966
|
+
- `:normal`: the progress handler is called on query progress (this is the
|
967
|
+
default mode).
|
968
|
+
- `:once`: the progress handler is called once before running the query.
|
969
|
+
- `:at_least_once`: the progress handler is called once before running the
|
970
|
+
query, and on query progress.
|
971
|
+
- `:period`: controls the approximate number of SQLite VM instructions executed
|
972
|
+
between consecutive calls to the progress handler. Default value: 1000.
|
973
|
+
- `:tick`: controls the granularity of the progress handler. This is the value
|
974
|
+
passed internally to the SQLite library. Default value: 10.
|
975
|
+
|
976
|
+
```ruby
|
977
|
+
db.on_progress(mode: :at_least_once, period: 640, tick: 5) { snooze }
|
978
|
+
```
|
979
|
+
|
980
|
+
### Global Progress Handler Settings
|
981
|
+
|
982
|
+
You can set the global progress handler behaviour by calling
|
983
|
+
`Extralite.on_progress`. You can use this API to set the global progress
|
984
|
+
settings, without needing to set a progress handler individually for each
|
985
|
+
`Database` instance. This method takes the same options as
|
986
|
+
`Database#on_progress`:
|
987
|
+
|
988
|
+
```ruby
|
989
|
+
Extralite.on_progress(mode: :at_least_once, period: 640, tick: 5) { snooze }
|
990
|
+
|
991
|
+
# the new database instance uses the global progress handler settings
|
992
|
+
db = Database.new(':memory:')
|
993
|
+
```
|
994
|
+
|
821
995
|
### Extralite and Fibers
|
822
996
|
|
823
997
|
The progress handler can also be used to switch between fibers in a
|
@@ -829,7 +1003,7 @@ handler. This will work for switching between fibers using either Polyphony or
|
|
829
1003
|
any fiber scheduler gem, such as Async et al:
|
830
1004
|
|
831
1005
|
```ruby
|
832
|
-
db.on_progress
|
1006
|
+
db.on_progress { sleep(0) }
|
833
1007
|
```
|
834
1008
|
|
835
1009
|
For Polyphony-based apps, you can also call `snooze` to allow other fibers to
|
@@ -837,7 +1011,7 @@ run while a query is progressing. If your Polyphony app is multi-threaded,
|
|
837
1011
|
you'll also need to call `Thread.pass` in order to allow other threads to run:
|
838
1012
|
|
839
1013
|
```ruby
|
840
|
-
db.on_progress
|
1014
|
+
db.on_progress do
|
841
1015
|
snooze
|
842
1016
|
Thread.pass
|
843
1017
|
end
|
@@ -848,7 +1022,7 @@ use the regular `#move_on_after` and `#cancel_after` methods to implement
|
|
848
1022
|
timeouts for queries:
|
849
1023
|
|
850
1024
|
```ruby
|
851
|
-
db.on_progress
|
1025
|
+
db.on_progress { snooze }
|
852
1026
|
|
853
1027
|
cancel_after(3) do
|
854
1028
|
db.query(long_running_query)
|
@@ -868,7 +1042,14 @@ as long as the following conditions are met:
|
|
868
1042
|
|
869
1043
|
### Use with Ractors
|
870
1044
|
|
871
|
-
Extralite databases can safely be used inside ractors.
|
1045
|
+
Extralite databases can safely be used inside ractors. A ractor has the benefit
|
1046
|
+
of using a separate GVL from the maine one, which allows true parallelism for
|
1047
|
+
Ruby apps. So when you use Extralite to access SQLite databases from within a
|
1048
|
+
ractor, you can do so without any considerations for what's happening outside
|
1049
|
+
the ractor when it runs queries.
|
1050
|
+
|
1051
|
+
**Note**: Ractors are considered an experimental feature of Ruby. You may
|
1052
|
+
encounter errors or inconsistent behaviour when using ractors.
|
872
1053
|
|
873
1054
|
## Advanced Usage
|
874
1055
|
|
@@ -1027,7 +1208,8 @@ large number of rows.
|
|
1027
1208
|
|
1028
1209
|
### Rows as Hashes
|
1029
1210
|
|
1030
|
-
|
1211
|
+
[Benchmark source
|
1212
|
+
code](https://github.com/digital-fabric/extralite/blob/main/test/perf_hash.rb)
|
1031
1213
|
|
1032
1214
|
|Row count|sqlite3 1.7.0|Extralite 2.5|Advantage|
|
1033
1215
|
|-:|-:|-:|-:|
|
@@ -1037,7 +1219,8 @@ large number of rows.
|
|
1037
1219
|
|
1038
1220
|
### Rows as Arrays
|
1039
1221
|
|
1040
|
-
[Benchmark source
|
1222
|
+
[Benchmark source
|
1223
|
+
code](https://github.com/digital-fabric/extralite/blob/main/test/perf_ary.rb)
|
1041
1224
|
|
1042
1225
|
|Row count|sqlite3 1.7.0|Extralite 2.5|Advantage|
|
1043
1226
|
|-:|-:|-:|-:|
|
@@ -1047,7 +1230,8 @@ large number of rows.
|
|
1047
1230
|
|
1048
1231
|
### Prepared Queries (Prepared Statements)
|
1049
1232
|
|
1050
|
-
[Benchmark source
|
1233
|
+
[Benchmark source
|
1234
|
+
code](https://github.com/digital-fabric/extralite/blob/main/test/perf_hash_prepared.rb)
|
1051
1235
|
|
1052
1236
|
|Row count|sqlite3 1.7.0|Extralite 2.5|Advantage|
|
1053
1237
|
|-:|-:|-:|-:|
|
@@ -1058,7 +1242,10 @@ large number of rows.
|
|
1058
1242
|
As those benchmarks show, Extralite is capabale of reading up to 2.4M rows per
|
1059
1243
|
second, and can be more than 14 times faster than the `sqlite3` gem.
|
1060
1244
|
|
1061
|
-
Note that the benchmarks above were performed on synthetic data, in a
|
1245
|
+
Note that the benchmarks above were performed on synthetic data, in a
|
1246
|
+
single-threaded environment, with the GVL release threshold set to -1, which
|
1247
|
+
means that both Extralite and the `sqlite3` gem hold the GVL for the duration of
|
1248
|
+
the query.
|
1062
1249
|
|
1063
1250
|
## License
|
1064
1251
|
|