extralite 1.21 → 1.22
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +4 -0
- data/Gemfile.lock +1 -1
- data/README.md +118 -64
- data/ext/extralite/database.c +13 -2
- data/lib/extralite/version.rb +1 -1
- data/lib/extralite.rb +12 -14
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 0f89c90ddd416f49777f9962a2abeac953eb1303cee79b2c612f6936cf7eb033
|
|
4
|
+
data.tar.gz: fc2604b1b6ef38a7e52bdeaaf579a84728885a09e0005c4a436064b4a919f5d7
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 892b73c7eb255b0e65f7a7b23b4b07fb9ad824a0b4766bcbcfff9d5996fb6954f664725f7bff0682a09b10c0d45690d53311b1e59038fce3b23805f919f53909
|
|
7
|
+
data.tar.gz: 2e1a19b0d9053e40a13ae905fc6f78acead1d6a625c009e8d55b9b47402b392012c68ee2fa2acd0d6f1a78aa8c5da1bed946d6a20d85db44209f9a1b897eeca1
|
data/CHANGELOG.md
CHANGED
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
|
@@ -1,64 +1,42 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
<p align="center">
|
|
8
|
-
<a href="http://rubygems.org/gems/extralite">
|
|
9
|
-
<img src="https://badge.fury.io/rb/extralite.svg" alt="Ruby gem">
|
|
10
|
-
</a>
|
|
11
|
-
<a href="https://github.com/digital-fabric/extralite/actions?query=workflow%3ATests">
|
|
12
|
-
<img src="https://github.com/digital-fabric/extralite/workflows/Tests/badge.svg" alt="Tests">
|
|
13
|
-
</a>
|
|
14
|
-
<a href="https://github.com/digital-fabric/extralite/blob/master/LICENSE">
|
|
15
|
-
<img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="MIT License">
|
|
16
|
-
</a>
|
|
17
|
-
</p>
|
|
18
|
-
|
|
19
|
-
<p align="center">
|
|
20
|
-
<a href="https://www.rubydoc.info/gems/extralite">DOCS</a> |
|
|
21
|
-
<a href="https://noteflakes.com/articles/2021-12-15-extralite">BLOG POST</a>
|
|
22
|
-
</p>
|
|
1
|
+
# Extralite - A super fast Ruby gem for working with SQLite3 databases
|
|
2
|
+
|
|
3
|
+
* Source code: https://github.com/digital-fabric/extralite
|
|
4
|
+
* Documentation: http://www.rubydoc.info/gems/extralite
|
|
5
|
+
|
|
6
|
+
[](https://rubygems.org/gems/extralite) [](https://github.com/digital-fabric/extralite/actions?query=workflow%3ATests) [](https://github.com/digital-fabric/extralite/blob/master/LICENSE)
|
|
23
7
|
|
|
24
8
|
## What is Extralite?
|
|
25
9
|
|
|
26
|
-
Extralite is a fast, extra-lightweight (about
|
|
27
|
-
wrapper for Ruby. It provides a minimal set of methods for interacting
|
|
28
|
-
SQLite3 database, as well as prepared statements.
|
|
10
|
+
Extralite is a super fast, extra-lightweight (about 1300 lines of C-code)
|
|
11
|
+
SQLite3 wrapper for Ruby. It provides a minimal set of methods for interacting
|
|
12
|
+
with an SQLite3 database, as well as prepared statements.
|
|
29
13
|
|
|
30
14
|
Extralite comes in two flavors: the `extralite` gem which uses the
|
|
31
15
|
system-installed sqlite3 library, and the `extralite-bundle` gem which bundles
|
|
32
16
|
the latest version of SQLite
|
|
33
|
-
([3.
|
|
17
|
+
([3.40.1](https://sqlite.org/releaselog/3_40_1.html)), offering access to the
|
|
34
18
|
latest features and enhancements.
|
|
35
19
|
|
|
36
20
|
## Features
|
|
37
21
|
|
|
22
|
+
- Super fast - [up to 10x faster](#performance) than the
|
|
23
|
+
[sqlite3](https://github.com/sparklemotion/sqlite3-ruby) gem (see also
|
|
24
|
+
[comparison](#why-not-just-use-the-sqlite3-gem).)
|
|
38
25
|
- A variety of methods for different data access patterns: rows as hashes, rows
|
|
39
26
|
as arrays, single row, single column, single value.
|
|
40
27
|
- Prepared statements.
|
|
28
|
+
- Parameter binding.
|
|
41
29
|
- Use system-installed sqlite3, or the [bundled latest version of
|
|
42
30
|
SQLite3](#installing-the-extralite-sqlite3-bundle).
|
|
43
|
-
- Super fast - [up to 12.5x faster](#performance) than the
|
|
44
|
-
[sqlite3](https://github.com/sparklemotion/sqlite3-ruby) gem (see also
|
|
45
|
-
[comparison](#why-not-just-use-the-sqlite3-gem).)
|
|
46
31
|
- Improved [concurrency](#concurrency) for multithreaded apps: the Ruby GVL is
|
|
47
32
|
released while preparing SQL statements and while iterating over results.
|
|
48
33
|
- Iterate over records with a block, or collect records into an array.
|
|
49
|
-
- Parameter binding.
|
|
50
34
|
- Automatically execute SQL strings containing multiple semicolon-separated
|
|
51
35
|
queries (handy for creating/modifying schemas).
|
|
52
|
-
- Get last insert rowid.
|
|
53
|
-
- Get number of rows changed by last query.
|
|
54
36
|
- Execute the same query with multiple parameter lists (useful for inserting records).
|
|
55
37
|
- Load extensions (loading of extensions is autmatically enabled. You can find
|
|
56
38
|
some useful extensions here: https://github.com/nalgeon/sqlean.)
|
|
57
39
|
- Includes a [Sequel adapter](#usage-with-sequel).
|
|
58
|
-
- Other features:
|
|
59
|
-
- Backup databases
|
|
60
|
-
- Interrupt long-running queries (from another thread)
|
|
61
|
-
- Get runtime status, database status and prepared statement status values.
|
|
62
40
|
|
|
63
41
|
## Installation
|
|
64
42
|
|
|
@@ -77,13 +55,14 @@ system-installed version of SQLite3, or would like to use the latest version of
|
|
|
77
55
|
SQLite3, you can install the `extralite-bundle` gem, which integrates the
|
|
78
56
|
SQLite3 source code.
|
|
79
57
|
|
|
80
|
-
> **Important note**: The `extralite-bundle` will take a while to install
|
|
81
|
-
> modest machine it takes about a minute), due to the size of the sqlite3
|
|
58
|
+
> **Important note**: The `extralite-bundle` gem will take a while to install
|
|
59
|
+
> (on my modest machine it takes about a minute), due to the size of the sqlite3
|
|
60
|
+
> code.
|
|
82
61
|
|
|
83
62
|
Usage of the `extralite-bundle` gem is identical to the usage of the normal
|
|
84
|
-
`extralite` gem.
|
|
63
|
+
`extralite` gem, using `require 'extralite'` to load the gem.
|
|
85
64
|
|
|
86
|
-
##
|
|
65
|
+
## Synopsis
|
|
87
66
|
|
|
88
67
|
```ruby
|
|
89
68
|
require 'extralite'
|
|
@@ -144,6 +123,9 @@ rowid = db.last_insert_rowid
|
|
|
144
123
|
# get number of rows changed in last query
|
|
145
124
|
number_of_rows_affected = db.changes
|
|
146
125
|
|
|
126
|
+
# get column names for the given sql
|
|
127
|
+
db.columns('select a, b, c from foo') => [:a, :b, :c]
|
|
128
|
+
|
|
147
129
|
# get db filename
|
|
148
130
|
db.filename #=> "/tmp/my.db"
|
|
149
131
|
|
|
@@ -163,6 +145,78 @@ db.close
|
|
|
163
145
|
db.closed? #=> true
|
|
164
146
|
```
|
|
165
147
|
|
|
148
|
+
## More features
|
|
149
|
+
|
|
150
|
+
### Interrupting long-running queries
|
|
151
|
+
|
|
152
|
+
When running long-running queries, you can use `Database#interrupt` to interrupt
|
|
153
|
+
the query:
|
|
154
|
+
|
|
155
|
+
```ruby
|
|
156
|
+
timeout_thread = Thread.new do
|
|
157
|
+
sleep 10
|
|
158
|
+
db.interrupt
|
|
159
|
+
end
|
|
160
|
+
|
|
161
|
+
result = begin
|
|
162
|
+
db.query(super_slow_sql)
|
|
163
|
+
rescue Extralite::InterruptError
|
|
164
|
+
nil
|
|
165
|
+
ensure
|
|
166
|
+
timeout_thread.kill
|
|
167
|
+
timeout_thread.join
|
|
168
|
+
end
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
### Creating backups
|
|
172
|
+
|
|
173
|
+
You can use `Database#backup` to create backup copies of a database. The
|
|
174
|
+
`#backup` method takes either a filename or a database instance:
|
|
175
|
+
|
|
176
|
+
```ruby
|
|
177
|
+
# with a filename
|
|
178
|
+
db.backup('backup.db')
|
|
179
|
+
|
|
180
|
+
# with an instance
|
|
181
|
+
target = Extralite::Database.new('backup.db')
|
|
182
|
+
db.backup(target)
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
For big databases, you can also track the backup progress by providing a block
|
|
186
|
+
that takes two arguments - the number of remaining pages, and the total number pages:
|
|
187
|
+
|
|
188
|
+
```ruby
|
|
189
|
+
db.backup('backup.db') do |remaining, total|
|
|
190
|
+
puts "backup progress: #{(remaining.to_f/total * 100).round}%"
|
|
191
|
+
end
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
### Retrieve status information
|
|
195
|
+
|
|
196
|
+
Extralite provides methods for retrieving status information about the sqlite
|
|
197
|
+
runtime, database-specific status and prepared statement-specific status,
|
|
198
|
+
`Extralite.runtime_status`, `Database#status` and `PreparedStatement#status`
|
|
199
|
+
respectively. You can also reset the high water mark for the specific status
|
|
200
|
+
code by providing true as the reset argument. The status codes mirror those
|
|
201
|
+
defined by the SQLite API. Some examples:
|
|
202
|
+
|
|
203
|
+
```ruby
|
|
204
|
+
# The Extralite.runtime_status returns a tuple consisting of the current value
|
|
205
|
+
# and the high water mark value.
|
|
206
|
+
current, high_watermark = Extralite.runtime_status(Extralite::SQLITE_STATUS_MEMORY_USED)
|
|
207
|
+
|
|
208
|
+
# To reset the high water mark, pass true as a second argument:
|
|
209
|
+
current, high_watermark = Extralite.runtime_status(Extralite::SQLITE_STATUS_MEMORY_USED, true)
|
|
210
|
+
|
|
211
|
+
# Similarly, you can interrogate a database's status (pass true as a second
|
|
212
|
+
# argument in order to reset the high watermark):
|
|
213
|
+
current, high_watermark = db.status(Extralite::SQLITE_DBSTATUS_CACHE_USED)
|
|
214
|
+
|
|
215
|
+
# The PreparedStatement#status method returns a single value (pass true as a
|
|
216
|
+
# second argument in order to reset the high watermark):
|
|
217
|
+
value = stmt.status(Extralite::SQLITE_STMTSTATUS_RUN)
|
|
218
|
+
```
|
|
219
|
+
|
|
166
220
|
## Usage with Sequel
|
|
167
221
|
|
|
168
222
|
Extralite includes an adapter for
|
|
@@ -179,20 +233,20 @@ p articles.to_a
|
|
|
179
233
|
|
|
180
234
|
## Why not just use the sqlite3 gem?
|
|
181
235
|
|
|
182
|
-
The [sqlite3
|
|
236
|
+
The [sqlite3](https://github.com/sparklemotion/sqlite3-ruby) gem is a
|
|
183
237
|
popular, solid, well-maintained project, used by thousands of developers. I've
|
|
184
238
|
been doing a lot of work with SQLite3 databases lately, and wanted to have a
|
|
185
239
|
simpler API that gives me query results in a variety of ways. Thus extralite was
|
|
186
240
|
born.
|
|
187
241
|
|
|
188
|
-
Extralite is
|
|
189
|
-
[thread-friendly](#concurrency). On the other hand, Extralite does not have
|
|
242
|
+
Extralite is significantly [faster](#performance) than the `sqlite3` gem and is
|
|
243
|
+
also [thread-friendly](#concurrency). On the other hand, Extralite does not have
|
|
190
244
|
support for defining custom functions, aggregates and collations. If you're
|
|
191
|
-
using any of those features, you'll have to stick to sqlite3
|
|
245
|
+
using any of those features, you'll have to stick to the `sqlite3` gem.
|
|
192
246
|
|
|
193
247
|
Here's a table summarizing the differences between the two gems:
|
|
194
248
|
|
|
195
|
-
| |sqlite3
|
|
249
|
+
| |sqlite3 1.6.0|Extralite 1.21|
|
|
196
250
|
|-|-|-|
|
|
197
251
|
|SQLite3 dependency|depends on OS-installed libsqlite3|Use either system sqlite3 or [bundled latest version of SQLite3](#installing-the-extralite-sqlite3-bundle)|
|
|
198
252
|
|API design|multiple classes|single class|
|
|
@@ -203,56 +257,56 @@ Here's a table summarizing the differences between the two gems:
|
|
|
203
257
|
|custom collations|yes|no|
|
|
204
258
|
|custom aggregate functions|yes|no|
|
|
205
259
|
|Multithread friendly|no|[yes](#concurrency)|
|
|
206
|
-
|Code size|~2650LoC|~
|
|
207
|
-
|Performance|1x|1.5x to
|
|
260
|
+
|Code size|~2650LoC|~1300LoC|
|
|
261
|
+
|Performance|1x|1.5x to 10x (see [below](#performance))|
|
|
208
262
|
|
|
209
263
|
## Concurrency
|
|
210
264
|
|
|
211
265
|
Extralite releases the GVL while making blocking calls to the sqlite3 library,
|
|
212
266
|
that is while preparing SQL statements and fetching rows. Releasing the GVL
|
|
213
267
|
allows other threads to run while the sqlite3 library is busy compiling SQL into
|
|
214
|
-
bytecode, or fetching the next row. This does not
|
|
215
|
-
performance:
|
|
268
|
+
bytecode, or fetching the next row. This *does not* hurt Extralite's
|
|
269
|
+
performance, as you can see:
|
|
216
270
|
|
|
217
271
|
## Performance
|
|
218
272
|
|
|
219
273
|
A benchmark script is included, creating a table of various row counts, then
|
|
220
274
|
fetching the entire table using either `sqlite3` or `extralite`. This benchmark
|
|
221
|
-
shows Extralite to be up to ~
|
|
275
|
+
shows Extralite to be up to ~10 times faster than `sqlite3` when fetching a
|
|
222
276
|
large number of rows.
|
|
223
277
|
|
|
224
278
|
### Rows as hashes
|
|
225
279
|
|
|
226
280
|
[Benchmark source code](https://github.com/digital-fabric/extralite/blob/main/test/perf_hash.rb)
|
|
227
281
|
|
|
228
|
-
|Row count|sqlite3
|
|
282
|
+
|Row count|sqlite3 1.6.0|Extralite 1.21|Advantage|
|
|
229
283
|
|-:|-:|-:|-:|
|
|
230
|
-
|10|
|
|
231
|
-
|1K|
|
|
232
|
-
|100K|
|
|
284
|
+
|10|63.7K rows/s|94.0K rows/s|__1.48x__|
|
|
285
|
+
|1K|299.2K rows/s|1.983M rows/s|__6.63x__|
|
|
286
|
+
|100K|185.4K rows/s|2.033M rows/s|__10.97x__|
|
|
233
287
|
|
|
234
288
|
### Rows as arrays
|
|
235
289
|
|
|
236
290
|
[Benchmark source code](https://github.com/digital-fabric/extralite/blob/main/test/perf_ary.rb)
|
|
237
291
|
|
|
238
|
-
|Row count|sqlite3
|
|
292
|
+
|Row count|sqlite3 1.6.0|Extralite 1.21|Advantage|
|
|
239
293
|
|-:|-:|-:|-:|
|
|
240
|
-
|10|
|
|
241
|
-
|1K|
|
|
242
|
-
|100K|
|
|
294
|
+
|10|71.2K rows/s|92.1K rows/s|__1.29x__|
|
|
295
|
+
|1K|502.1K rows/s|2.065M rows/s|__4.11x__|
|
|
296
|
+
|100K|455.7K rows/s|2.511M rows/s|__5.51x__|
|
|
243
297
|
|
|
244
298
|
### Prepared statements
|
|
245
299
|
|
|
246
300
|
[Benchmark source code](https://github.com/digital-fabric/extralite/blob/main/test/perf_prepared.rb)
|
|
247
301
|
|
|
248
|
-
|Row count|sqlite3
|
|
302
|
+
|Row count|sqlite3 1.6.0|Extralite 1.21|Advantage|
|
|
249
303
|
|-:|-:|-:|-:|
|
|
250
|
-
|10|
|
|
251
|
-
|1K|
|
|
252
|
-
|100K|
|
|
304
|
+
|10|232.2K rows/s|741.6K rows/s|__3.19x__|
|
|
305
|
+
|1K|299.8K rows/s|2386.0M rows/s|__7.96x__|
|
|
306
|
+
|100K|183.1K rows/s|1.893M rows/s|__10.34x__|
|
|
253
307
|
|
|
254
|
-
As those benchmarks show, Extralite is capabale of reading up to
|
|
255
|
-
when fetching rows as arrays, and up to
|
|
308
|
+
As those benchmarks show, Extralite is capabale of reading up to 2.5M
|
|
309
|
+
rows/second when fetching rows as arrays, and up to 2M rows/second when fetching
|
|
256
310
|
rows as hashes.
|
|
257
311
|
|
|
258
312
|
## License
|
data/ext/extralite/database.c
CHANGED
|
@@ -470,6 +470,17 @@ VALUE backup_cleanup(VALUE ptr) {
|
|
|
470
470
|
return Qnil;
|
|
471
471
|
}
|
|
472
472
|
|
|
473
|
+
/* call-seq:
|
|
474
|
+
* db.backup(dest) -> db
|
|
475
|
+
* db.backup(dest) { |remaining, total| } -> db
|
|
476
|
+
*
|
|
477
|
+
* Creates a backup of the database to the given destination, which can be
|
|
478
|
+
* either a filename or a database instance. In order to monitor the backup
|
|
479
|
+
* progress you can pass a block that will be called periodically by the backup
|
|
480
|
+
* method with two arguments: the remaining page count, and the total page
|
|
481
|
+
* count, which can be used to display the progress to the user or to collect
|
|
482
|
+
* statistics.
|
|
483
|
+
*/
|
|
473
484
|
VALUE Database_backup(int argc, VALUE *argv, VALUE self) {
|
|
474
485
|
VALUE dst;
|
|
475
486
|
VALUE src_name;
|
|
@@ -513,8 +524,8 @@ VALUE Database_backup(int argc, VALUE *argv, VALUE self) {
|
|
|
513
524
|
return self;
|
|
514
525
|
}
|
|
515
526
|
|
|
516
|
-
/*
|
|
517
|
-
*
|
|
527
|
+
/* call-seq:
|
|
528
|
+
* Extralite.runtime_status(op[, reset]) -> [value, highwatermark]
|
|
518
529
|
*
|
|
519
530
|
* Returns runtime status values for the given op as an array containing the
|
|
520
531
|
* current value and the high water mark value. To reset the high water mark,
|
data/lib/extralite/version.rb
CHANGED
data/lib/extralite.rb
CHANGED
|
@@ -70,14 +70,26 @@ module Extralite
|
|
|
70
70
|
AND name NOT LIKE 'sqlite_%';
|
|
71
71
|
SQL
|
|
72
72
|
|
|
73
|
+
# Returns the list of currently defined tables.
|
|
74
|
+
#
|
|
75
|
+
# @return [Array] list of tables
|
|
73
76
|
def tables
|
|
74
77
|
query_single_column(TABLES_SQL)
|
|
75
78
|
end
|
|
76
79
|
|
|
80
|
+
# Gets or sets one or more pragmas:
|
|
81
|
+
#
|
|
82
|
+
# db.pragma(:cache_size) # get
|
|
83
|
+
# db.pragma(cache_size: -2000) # set
|
|
84
|
+
#
|
|
85
|
+
# @param value [Symbol, String, Hash] pragma name or hash mapping names to values
|
|
86
|
+
# @return [Hash] query result
|
|
77
87
|
def pragma(value)
|
|
78
88
|
value.is_a?(Hash) ? pragma_set(value) : pragma_get(value)
|
|
79
89
|
end
|
|
80
90
|
|
|
91
|
+
private
|
|
92
|
+
|
|
81
93
|
def pragma_set(values)
|
|
82
94
|
sql = values.inject(+'') { |s, (k, v)| s += "pragma #{k}=#{v}; " }
|
|
83
95
|
query(sql)
|
|
@@ -87,18 +99,4 @@ module Extralite
|
|
|
87
99
|
query("pragma #{key}")
|
|
88
100
|
end
|
|
89
101
|
end
|
|
90
|
-
|
|
91
|
-
# An SQLite backup
|
|
92
|
-
class Backup
|
|
93
|
-
# def initialize(dst, dst_name, src, src_name); end
|
|
94
|
-
|
|
95
|
-
# def dst; end
|
|
96
|
-
# def src; end
|
|
97
|
-
|
|
98
|
-
# def step(pages); end
|
|
99
|
-
# def finish; end
|
|
100
|
-
|
|
101
|
-
# def pagecount; end
|
|
102
|
-
# def remaining; end
|
|
103
|
-
end
|
|
104
102
|
end
|