litestream 0.15.0 → 0.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +82 -68
- data/app/views/litestream/processes/show.html.erb +43 -55
- data/lib/litestream/commands.rb +20 -10
- data/lib/litestream/generators/litestream/templates/config.yml.erb +11 -6
- data/lib/litestream/upstream.rb +15 -4
- data/lib/litestream/version.rb +1 -1
- data/lib/litestream.rb +9 -11
- data/lib/tasks/litestream_tasks.rake +33 -16
- metadata +1 -15
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: d2ce16c53c02cc54eb6e402900815afe3808cab5ba49e033f7324c9159f7083c
|
|
4
|
+
data.tar.gz: 32a21e3c20fa3fe8a58f985d770faeb3d862f316a1c7a89934c5b1cb34bde832
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 837b32acb2dbb433753729da2a41d31913dbfecad1e1aff78695e713ecf44b8b27f522ebf5ca0006121c4358872d0e1928208452b6152ac0bc08396ecf19800a
|
|
7
|
+
data.tar.gz: 6162700c2cc73ed56b88bac09f386a4b6d2ce10d2d84acbe718e7406908efbfdd305ab5933f44a8ccf3ae3aa646139b1a673786ecde1c32cd08e46022b306b50
|
data/README.md
CHANGED
|
@@ -85,16 +85,22 @@ The gem streamlines the configuration process by providing a default configurati
|
|
|
85
85
|
The default configuration file looks like this if you only have one SQLite database:
|
|
86
86
|
|
|
87
87
|
```yaml
|
|
88
|
+
snapshot:
|
|
89
|
+
interval: 24h
|
|
90
|
+
retention: 24h
|
|
91
|
+
|
|
88
92
|
dbs:
|
|
89
93
|
- path: storage/production.sqlite3
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
94
|
+
replica:
|
|
95
|
+
type: s3
|
|
96
|
+
path: storage/production.sqlite3
|
|
97
|
+
bucket: $LITESTREAM_REPLICA_BUCKET
|
|
98
|
+
access-key-id: $LITESTREAM_ACCESS_KEY_ID
|
|
99
|
+
secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY
|
|
96
100
|
```
|
|
97
101
|
|
|
102
|
+
In Litestream 0.5, snapshot interval and retention are global settings rather than per-replica settings.
|
|
103
|
+
|
|
98
104
|
This is the default for Amazon S3. The full range of possible replica types (e.g. other S3-compatible object storage servers) are covered in Litestream's [replica guides](https://litestream.io/guides/#replica-guides).
|
|
99
105
|
|
|
100
106
|
The gem also provides a default initializer file at `config/initializers/litestream.rb` that allows you to configure various variables referenced in the configuration file in Ruby. By providing a Ruby interface to these environment variables, you can use your preferred method of storing secrets. For example, the default generated file uses Rails' encrypted credentials to store your secrets.
|
|
@@ -126,6 +132,28 @@ end
|
|
|
126
132
|
|
|
127
133
|
However, if you need manual control over the Litestream configuration, you can edit the `config/litestream.yml` file. The full range of possible configurations are covered in Litestream's [configuration reference](https://litestream.io/reference/config/).
|
|
128
134
|
|
|
135
|
+
#### Logging
|
|
136
|
+
|
|
137
|
+
Litestream writes its logs to stdout, where they mix with your application's. Its `logging:` block can make them easy to separate, for example as JSON on stderr:
|
|
138
|
+
|
|
139
|
+
```yaml
|
|
140
|
+
logging:
|
|
141
|
+
level: info
|
|
142
|
+
type: json
|
|
143
|
+
stderr: true
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
#### Replicas that authenticate with a credentials file
|
|
147
|
+
|
|
148
|
+
Some replica types read credentials from a file. Google Cloud Storage, for example, reads the path in `GOOGLE_APPLICATION_CREDENTIALS`. A file on your machine is not in the container that Kamal deploys, so pass its contents as a secret and write it out before the app starts, for example in `bin/docker-entrypoint`:
|
|
149
|
+
|
|
150
|
+
```shell
|
|
151
|
+
# .kamal/secrets: GCS_CREDENTIALS_JSON=$(cat config/gcs.json)
|
|
152
|
+
# config/deploy.yml: env: { secret: [GCS_CREDENTIALS_JSON] }
|
|
153
|
+
printf '%s' "$GCS_CREDENTIALS_JSON" > /rails/tmp/gcs.json
|
|
154
|
+
export GOOGLE_APPLICATION_CREDENTIALS=/rails/tmp/gcs.json
|
|
155
|
+
```
|
|
156
|
+
|
|
129
157
|
### Replication
|
|
130
158
|
|
|
131
159
|
In order to stream changes to your configured replicas, you need to start the Litestream replication process.
|
|
@@ -179,7 +207,7 @@ You can restore any replicated database at any point using the gem's provided `l
|
|
|
179
207
|
> [!NOTE]
|
|
180
208
|
> During the restoration process, you need to prevent any interaction with ActiveRecord/SQLite, such as from a running `rails server` or `rails console` instance. If there is any interaction, Rails might regenerate the production database and prevent restoration via litestream. If this happens, you might get a "cannot restore, output path already exists" error.
|
|
181
209
|
|
|
182
|
-
1. Rename the production
|
|
210
|
+
1. Rename the production database and its SQLite sidecar files (**recommended**) or alternatively delete them. To delete the production databases locally, you can run the following at your own risk:
|
|
183
211
|
```shell
|
|
184
212
|
# DANGEROUS OPERATION, consider renaming database files instead
|
|
185
213
|
bin/rails db:drop DISABLE_DATABASE_ENVIRONMENT_CHECK=1
|
|
@@ -215,26 +243,19 @@ You can forward arguments in whatever order you like, you simply need to ensure
|
|
|
215
243
|
-if-replica-exists
|
|
216
244
|
Returns exit code of 0 if no backups found.
|
|
217
245
|
|
|
218
|
-
-
|
|
219
|
-
|
|
220
|
-
Defaults to 8
|
|
221
|
-
|
|
222
|
-
-replica NAME
|
|
223
|
-
Restore from a specific replica.
|
|
224
|
-
Defaults to replica with latest data.
|
|
225
|
-
|
|
226
|
-
-generation NAME
|
|
227
|
-
Restore from a specific generation.
|
|
228
|
-
Defaults to generation with latest data.
|
|
229
|
-
|
|
230
|
-
-index NUM
|
|
231
|
-
Restore up to a specific WAL index (inclusive).
|
|
232
|
-
Defaults to use the highest available index.
|
|
246
|
+
-txid TXID
|
|
247
|
+
Restore through a specific 16-character hexadecimal transaction ID.
|
|
233
248
|
|
|
234
249
|
-timestamp TIMESTAMP
|
|
235
250
|
Restore to a specific point-in-time.
|
|
236
251
|
Defaults to use the latest available backup.
|
|
237
252
|
|
|
253
|
+
-json
|
|
254
|
+
Print the restore result as JSON.
|
|
255
|
+
|
|
256
|
+
-dry-run
|
|
257
|
+
Print the restore plan without writing the database.
|
|
258
|
+
|
|
238
259
|
-config PATH
|
|
239
260
|
Specifies the configuration file.
|
|
240
261
|
Defaults to /etc/litestream.yml
|
|
@@ -387,81 +408,59 @@ bin/rails litestream:databases
|
|
|
387
408
|
This will return a list of databases and their configured replicas:
|
|
388
409
|
|
|
389
410
|
```
|
|
390
|
-
path
|
|
411
|
+
path replica
|
|
391
412
|
/Users/you/Code/your-app/storage/production.sqlite3 s3
|
|
392
413
|
```
|
|
393
414
|
|
|
394
|
-
You can
|
|
415
|
+
You can inspect local replication status for all databases or one specific database:
|
|
395
416
|
|
|
396
417
|
```shell
|
|
397
|
-
bin/rails litestream:
|
|
418
|
+
bin/rails litestream:status -- --database=storage/production.sqlite3
|
|
398
419
|
```
|
|
399
420
|
|
|
400
|
-
This
|
|
421
|
+
This reads local state and does not require a running Litestream process:
|
|
401
422
|
|
|
402
423
|
```
|
|
403
|
-
|
|
404
|
-
|
|
424
|
+
database status local txid wal size
|
|
425
|
+
/Users/you/Code/your-app/storage/production.sqlite3 ok 000000000000000a 128 kB
|
|
405
426
|
```
|
|
406
427
|
|
|
407
|
-
You can list
|
|
428
|
+
You can list all remote LTX files, including the level-9 snapshot:
|
|
408
429
|
|
|
409
430
|
```shell
|
|
410
|
-
bin/rails litestream:
|
|
431
|
+
bin/rails litestream:ltx -- --database=storage/production.sqlite3 --level=all
|
|
411
432
|
```
|
|
412
433
|
|
|
413
|
-
|
|
434
|
+
The command returns the compaction level, transaction range, byte size, and creation time:
|
|
414
435
|
|
|
415
436
|
```
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
```
|
|
419
|
-
|
|
420
|
-
Finally, you can list the wal files available for a database:
|
|
421
|
-
|
|
422
|
-
```shell
|
|
423
|
-
bin/rails litestream:wal -- --database=storage/production.sqlite3
|
|
424
|
-
```
|
|
425
|
-
|
|
426
|
-
This command lists wal files available for that specified database:
|
|
427
|
-
|
|
428
|
-
```
|
|
429
|
-
replica generation index offset size created
|
|
430
|
-
s3 a295b16a796689f3 1 0 2036 2024-04-17T00:01:19Z
|
|
437
|
+
level min_txid max_txid size created
|
|
438
|
+
0 0000000000000008 000000000000000a 1013 2026-09-08T03:16:43Z
|
|
431
439
|
```
|
|
432
440
|
|
|
433
441
|
### Running commands from Ruby
|
|
434
442
|
|
|
435
443
|
In addition to the provided rake tasks, you can also run Litestream commands directly from Ruby. The gem provides a `Litestream::Commands` module that wraps the Litestream CLI commands. This is particularly useful for the introspection commands, as you can use the output in your Ruby code.
|
|
436
444
|
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
The `Litestream::Commands.databases` method returns an array of hashes with the "path" and "replicas" keys for each database:
|
|
445
|
+
Pass `json: true` to return the 0.5 CLI's JSON values with string keys. The `Litestream::Commands.databases` method returns each database path and replica type:
|
|
440
446
|
|
|
441
447
|
```ruby
|
|
442
|
-
Litestream::Commands.databases
|
|
443
|
-
# => [{"path"=>"/Users/you/Code/your-app/storage/production.sqlite3", "
|
|
448
|
+
Litestream::Commands.databases(json: true)
|
|
449
|
+
# => [{"path"=>"/Users/you/Code/your-app/storage/production.sqlite3", "replica"=>"s3"}]
|
|
444
450
|
```
|
|
445
451
|
|
|
446
|
-
The `
|
|
452
|
+
The `status` method returns local state without connecting to the replica or requiring a daemon:
|
|
447
453
|
|
|
448
454
|
```ruby
|
|
449
|
-
Litestream::Commands.
|
|
450
|
-
# => [{"
|
|
455
|
+
Litestream::Commands.status("storage/production.sqlite3", json: true)
|
|
456
|
+
# => [{"database"=>"/Users/you/Code/your-app/storage/production.sqlite3", "status"=>"ok", "local_txid"=>"000000000000000a", "wal_size"=>"128 kB"}]
|
|
451
457
|
```
|
|
452
458
|
|
|
453
|
-
The `
|
|
459
|
+
The `ltx` method lists remote LTX files. An unreplicated database returns an empty array:
|
|
454
460
|
|
|
455
461
|
```ruby
|
|
456
|
-
Litestream::Commands.
|
|
457
|
-
# => [{"
|
|
458
|
-
```
|
|
459
|
-
|
|
460
|
-
The `Litestream::Commands.wal` method returns an array of hashes with the "replica", "generation", "index", "offset","size", and "created" keys for each wal:
|
|
461
|
-
|
|
462
|
-
```ruby
|
|
463
|
-
Litestream::Commands.wal('storage/production.sqlite3')
|
|
464
|
-
# => [{"replica"=>"s3", "generation"=>"5f4341bc3d22d615", "index"=>"0", "offset"=>"0", "size"=>"2036", "created"=>"2024-04-17T19:48:09Z"}]
|
|
462
|
+
Litestream::Commands.ltx("storage/production.sqlite3", json: true, "--level" => "all")
|
|
463
|
+
# => [{"level"=>0, "min_txid"=>"0000000000000008", "max_txid"=>"000000000000000a", "size"=>1013, "timestamp"=>"2026-09-08T03:16:43Z"}]
|
|
465
464
|
```
|
|
466
465
|
|
|
467
466
|
You can also restore a database programmatically using the `Litestream::Commands.restore` method, which returns the path to the restored database:
|
|
@@ -481,14 +480,29 @@ The full set of commands available to the `litestream` executable are covered in
|
|
|
481
480
|
|
|
482
481
|
```shell
|
|
483
482
|
litestream databases [arguments]
|
|
484
|
-
litestream
|
|
483
|
+
litestream info [arguments]
|
|
484
|
+
litestream list [arguments]
|
|
485
|
+
litestream ltx [arguments] DB_PATH
|
|
486
|
+
litestream register [arguments]
|
|
485
487
|
litestream replicate [arguments]
|
|
486
|
-
litestream
|
|
487
|
-
litestream
|
|
488
|
+
litestream reset [arguments]
|
|
489
|
+
litestream restore [arguments] DB_PATH
|
|
490
|
+
litestream start [arguments]
|
|
491
|
+
litestream status [arguments] [DB_PATH]
|
|
492
|
+
litestream stop [arguments]
|
|
493
|
+
litestream sync [arguments]
|
|
494
|
+
litestream unregister [arguments]
|
|
488
495
|
litestream version
|
|
489
|
-
litestream wal [arguments] DB_PATH|REPLICA_URL
|
|
490
496
|
```
|
|
491
497
|
|
|
498
|
+
### Upgrading from 0.3
|
|
499
|
+
|
|
500
|
+
Litestream 0.5 replicates each database to a single `replica:` map and moves retention and snapshot settings into the root-level `snapshot:` block. A `replicas:` list with one entry still works. A list with more than one entry stops `litestream replicate` from starting ("multiple replicas on a single database are no longer supported"), and when the Puma plugin sees Litestream exit it stops Puma, so reduce each database to one replica before upgrading. The `databases` command and Ruby method now return a `"replica"` key instead of `"replicas"`.
|
|
501
|
+
|
|
502
|
+
The removed `generations`, `snapshots`, and `wal` command methods and rake tasks now raise an error pointing to `ltx`, which replaces all three forms of remote backup introspection.
|
|
503
|
+
|
|
504
|
+
The 0.5 `restore` command auto-detects both 0.3 and LTX backups and prefers whichever backup is newer. Existing 0.3 `generations/` objects are left in place; you can delete them after the LTX history covers your full retention window. To roll back, reinstall a 0.15.x release of this gem, which bundles Litestream 0.3.
|
|
505
|
+
|
|
492
506
|
### Using in development
|
|
493
507
|
|
|
494
508
|
By default, if you install the gem and configure via `puma.rb` or `Procfile`, Litestream will not start in development.
|
|
@@ -56,63 +56,51 @@
|
|
|
56
56
|
</div>
|
|
57
57
|
|
|
58
58
|
<br />
|
|
59
|
-
<section
|
|
60
|
-
<% database['
|
|
61
|
-
<
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
59
|
+
<section class="ml-6">
|
|
60
|
+
<% if database['error'] %>
|
|
61
|
+
<p class="text-red-600 dark:text-red-400"><%= database['error'] %></p>
|
|
62
|
+
<% else %>
|
|
63
|
+
<dl class="grid grid-cols-[fit-content(100%)_1fr] gap-x-4">
|
|
64
|
+
<dt class="font-bold">Status</dt>
|
|
65
|
+
<dd><%= database.dig('status', 'status') %></dd>
|
|
66
|
+
<dt class="font-bold">Local TXID</dt>
|
|
67
|
+
<dd><code><%= database.dig('status', 'local_txid') %></code></dd>
|
|
68
|
+
<dt class="font-bold">WAL size</dt>
|
|
69
|
+
<dd><%= number_to_human_size database.dig('status', 'wal_size') %></dd>
|
|
70
|
+
</dl>
|
|
66
71
|
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
72
|
+
<br />
|
|
73
|
+
<% if database['ltx'] == [] %>
|
|
74
|
+
<p>No LTX files yet</p>
|
|
75
|
+
<% else %>
|
|
76
|
+
<table class="min-w-full divide-y divide-gray-300">
|
|
77
|
+
<thead>
|
|
78
|
+
<tr>
|
|
79
|
+
<th scope="col" class="whitespace-nowrap px-2 py-2 text-right text-sm font-semibold text-gray-900 dark:text-gray-100">Level</th>
|
|
80
|
+
<th scope="col" class="whitespace-nowrap px-2 py-2 text-left text-sm font-semibold text-gray-900 dark:text-gray-100">Min TXID</th>
|
|
81
|
+
<th scope="col" class="whitespace-nowrap px-2 py-2 text-left text-sm font-semibold text-gray-900 dark:text-gray-100">Max TXID</th>
|
|
82
|
+
<th scope="col" class="whitespace-nowrap px-2 py-2 text-right text-sm font-semibold text-gray-900 dark:text-gray-100">Size</th>
|
|
83
|
+
<th scope="col" class="whitespace-nowrap px-2 py-2 text-left text-sm font-semibold text-gray-900 dark:text-gray-100">Created</th>
|
|
84
|
+
</tr>
|
|
85
|
+
</thead>
|
|
74
86
|
|
|
75
|
-
<
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
</thead>
|
|
93
|
-
|
|
94
|
-
<tbody class="bg-white dark:bg-gray-900">
|
|
95
|
-
<% generation['snapshots'].each do |snapshot| %>
|
|
96
|
-
<tr class="align-top even:bg-gray-50 dark:even:bg-gray-800">
|
|
97
|
-
<td scope="col" class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100">
|
|
98
|
-
<abbr title="<%= snapshot['created'] %>" class="underline decoration-dashed decoration-gray-500 cursor-help">
|
|
99
|
-
<time datetime="<%= snapshot['created'] %>"><%= DateTime.parse(snapshot['created']).to_formatted_s(:db) %></time>
|
|
100
|
-
</abbr>
|
|
101
|
-
</td>
|
|
102
|
-
<td scope="col" class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100 text-right">
|
|
103
|
-
<%= snapshot['index'] %>
|
|
104
|
-
</td>
|
|
105
|
-
<td scope="col" class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100 text-right">
|
|
106
|
-
<%= number_to_human_size snapshot['size'] %>
|
|
107
|
-
</td>
|
|
108
|
-
</tr>
|
|
109
|
-
<% end %>
|
|
110
|
-
</tbody>
|
|
111
|
-
</table>
|
|
112
|
-
</dd>
|
|
113
|
-
</div>
|
|
114
|
-
</dl>
|
|
115
|
-
</details>
|
|
87
|
+
<tbody class="bg-white dark:bg-gray-900">
|
|
88
|
+
<% database['ltx'].each do |ltx| %>
|
|
89
|
+
<tr class="align-top even:bg-gray-50 dark:even:bg-gray-800">
|
|
90
|
+
<td class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100 text-right"><%= ltx['level'] %></td>
|
|
91
|
+
<td class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100"><code><%= ltx['min_txid'] %></code></td>
|
|
92
|
+
<td class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100"><code><%= ltx['max_txid'] %></code></td>
|
|
93
|
+
<td class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100 text-right"><%= number_to_human_size ltx['size'] %></td>
|
|
94
|
+
<td class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100">
|
|
95
|
+
<abbr title="<%= ltx['timestamp'] %>" class="underline decoration-dashed decoration-gray-500 cursor-help">
|
|
96
|
+
<time datetime="<%= ltx['timestamp'] %>"><%= DateTime.parse(ltx['timestamp']).to_formatted_s(:db) %></time>
|
|
97
|
+
</abbr>
|
|
98
|
+
</td>
|
|
99
|
+
</tr>
|
|
100
|
+
<% end %>
|
|
101
|
+
</tbody>
|
|
102
|
+
</table>
|
|
103
|
+
<% end %>
|
|
116
104
|
<% end %>
|
|
117
105
|
</section>
|
|
118
106
|
</li>
|
data/lib/litestream/commands.rb
CHANGED
|
@@ -124,22 +124,30 @@ module Litestream
|
|
|
124
124
|
execute("databases", argv)
|
|
125
125
|
end
|
|
126
126
|
|
|
127
|
-
def
|
|
128
|
-
raise DatabaseRequiredException, "database argument is required for
|
|
127
|
+
def ltx(database, **argv)
|
|
128
|
+
raise DatabaseRequiredException, "database argument is required for ltx command, e.g. litestream:ltx -- --database=path/to/database.sqlite" if database.nil?
|
|
129
129
|
|
|
130
|
-
execute("
|
|
130
|
+
execute("ltx", argv, database)
|
|
131
131
|
end
|
|
132
132
|
|
|
133
|
-
|
|
134
|
-
|
|
133
|
+
# Litestream 0.5 filters `status` by the absolute database path, while
|
|
134
|
+
# `ltx` and `restore` match the path as written in the config file, so a
|
|
135
|
+
# relative path is expanded here to keep the documented
|
|
136
|
+
# `--database=storage/production.sqlite3` form working.
|
|
137
|
+
def status(database = nil, **argv)
|
|
138
|
+
execute("status", argv, database && File.expand_path(database))
|
|
139
|
+
end
|
|
135
140
|
|
|
136
|
-
|
|
141
|
+
def generations(*)
|
|
142
|
+
raise CommandFailedException, "`generations` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)"
|
|
137
143
|
end
|
|
138
144
|
|
|
139
|
-
def
|
|
140
|
-
raise
|
|
145
|
+
def snapshots(*)
|
|
146
|
+
raise CommandFailedException, "`snapshots` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)"
|
|
147
|
+
end
|
|
141
148
|
|
|
142
|
-
|
|
149
|
+
def wal(*)
|
|
150
|
+
raise CommandFailedException, "`wal` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)"
|
|
143
151
|
end
|
|
144
152
|
|
|
145
153
|
private
|
|
@@ -232,8 +240,10 @@ module Litestream
|
|
|
232
240
|
raise CommandFailedException, "Unexpected output from `#{cmd[1]}`: #{stdout.lines.first.to_s.strip[0, 200]}"
|
|
233
241
|
end
|
|
234
242
|
|
|
243
|
+
# Litestream pads columns with at least two spaces, and some headers and
|
|
244
|
+
# values hold a single one ("local txid", "231 kB", "no database").
|
|
235
245
|
def parse_table(stdout)
|
|
236
|
-
keys, *rows = stdout.strip.split("\n").map { _1.split(/\s
|
|
246
|
+
keys, *rows = stdout.strip.split("\n").map { _1.strip.split(/\s{2,}/) }
|
|
237
247
|
return [] unless keys
|
|
238
248
|
|
|
239
249
|
rows.map { keys.zip(_1).to_h }
|
|
@@ -9,13 +9,18 @@
|
|
|
9
9
|
# `replicate` command.
|
|
10
10
|
#
|
|
11
11
|
# For more details, see: https://litestream.io/reference/config/
|
|
12
|
+
# Retention and snapshot settings are global in Litestream 0.5.
|
|
13
|
+
snapshot:
|
|
14
|
+
interval: 24h
|
|
15
|
+
retention: 24h
|
|
16
|
+
|
|
12
17
|
dbs:
|
|
13
18
|
<%- production_sqlite_databases.each do |database| -%>
|
|
14
19
|
- path: <%= database %>
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
20
|
+
replica:
|
|
21
|
+
type: s3
|
|
22
|
+
bucket: $LITESTREAM_REPLICA_BUCKET
|
|
23
|
+
path: <%= database %>
|
|
24
|
+
access-key-id: $LITESTREAM_ACCESS_KEY_ID
|
|
25
|
+
secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY
|
|
21
26
|
<%- end -%>
|
data/lib/litestream/upstream.rb
CHANGED
|
@@ -1,14 +1,25 @@
|
|
|
1
1
|
module Litestream
|
|
2
2
|
module Upstream
|
|
3
|
-
VERSION = "
|
|
3
|
+
VERSION = "0.5.17"
|
|
4
4
|
|
|
5
5
|
# rubygems platform name => upstream release filename
|
|
6
6
|
NATIVE_PLATFORMS = {
|
|
7
7
|
"aarch64-linux" => "litestream-#{VERSION}-linux-arm64.tar.gz",
|
|
8
|
-
"arm64-darwin" => "litestream-#{VERSION}-darwin-arm64.
|
|
8
|
+
"arm64-darwin" => "litestream-#{VERSION}-darwin-arm64.tar.gz",
|
|
9
9
|
"arm64-linux" => "litestream-#{VERSION}-linux-arm64.tar.gz",
|
|
10
|
-
"x86_64-darwin" => "litestream-#{VERSION}-darwin-
|
|
11
|
-
"x86_64-linux" => "litestream-#{VERSION}-linux-
|
|
10
|
+
"x86_64-darwin" => "litestream-#{VERSION}-darwin-x86_64.tar.gz",
|
|
11
|
+
"x86_64-linux" => "litestream-#{VERSION}-linux-x86_64.tar.gz"
|
|
12
12
|
}
|
|
13
|
+
|
|
14
|
+
CHECKSUMS = {
|
|
15
|
+
"litestream-0.5.17-linux-x86_64.tar.gz" => "cfb371176d164437ae869f8351cfde49bd1804ae71c61923f75c9cba9c9c006d",
|
|
16
|
+
"litestream-0.5.17-linux-arm64.tar.gz" => "f8ca4a050095c1efbda2c4365172e61bf9d955ea0d9ac42f448b52e51819baa5",
|
|
17
|
+
"litestream-0.5.17-darwin-x86_64.tar.gz" => "891875af09db152e93a4b31a8a79f538ce7ce702c132803cfe0a831e7cb1b7db",
|
|
18
|
+
"litestream-0.5.17-darwin-arm64.tar.gz" => "e211f68ff7658d19f193f2914417afdf8f89a053ff8f263e5d6b3b1d3bbc7b08"
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
def self.download_url(filename)
|
|
22
|
+
"https://github.com/benbjohnson/litestream/releases/download/v#{VERSION}/#{filename}"
|
|
23
|
+
end
|
|
13
24
|
end
|
|
14
25
|
end
|
data/lib/litestream/version.rb
CHANGED
data/lib/litestream.rb
CHANGED
|
@@ -120,20 +120,18 @@ module Litestream
|
|
|
120
120
|
end
|
|
121
121
|
|
|
122
122
|
def databases
|
|
123
|
-
databases = Commands.databases
|
|
123
|
+
databases = Commands.databases(json: true)
|
|
124
124
|
|
|
125
125
|
databases.each do |db|
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
replica = generation["name"]
|
|
133
|
-
generation["snapshots"] = snapshots.select { |snapshot| snapshot["generation"] == id && snapshot["replica"] == replica }
|
|
134
|
-
.map { |s| s.slice("index", "size", "created") }
|
|
135
|
-
generation.slice("generation", "name", "lag", "start", "end", "snapshots")
|
|
126
|
+
path = db["path"]
|
|
127
|
+
begin
|
|
128
|
+
db["status"] = Commands.status(path, json: true).first
|
|
129
|
+
db["ltx"] = Commands.ltx(path, :json => true, "--level" => "all")
|
|
130
|
+
rescue Commands::CommandFailedException => error
|
|
131
|
+
db["error"] = error.message
|
|
136
132
|
end
|
|
133
|
+
|
|
134
|
+
db["path"] = path.gsub(Rails.root.to_s, "[ROOT]")
|
|
137
135
|
end
|
|
138
136
|
end
|
|
139
137
|
|
|
@@ -32,40 +32,57 @@ namespace :litestream do
|
|
|
32
32
|
puts Litestream::Commands::Output.format(Litestream::Commands.databases(**options))
|
|
33
33
|
end
|
|
34
34
|
|
|
35
|
-
desc "List all
|
|
36
|
-
task
|
|
35
|
+
desc "List all LTX files for a database or replica, for example `rake litestream:ltx -- -database=storage/production.sqlite3`"
|
|
36
|
+
task ltx: :environment do
|
|
37
37
|
options = parse_argv_options
|
|
38
38
|
database = options.delete(:"--database") || options.delete(:"-database")
|
|
39
39
|
|
|
40
|
-
puts Litestream::Commands::Output.format(Litestream::Commands.
|
|
40
|
+
puts Litestream::Commands::Output.format(Litestream::Commands.ltx(database, **options))
|
|
41
41
|
end
|
|
42
42
|
|
|
43
|
-
desc "
|
|
44
|
-
task
|
|
43
|
+
desc "Show replication status, for example `rake litestream:status -- -database=storage/production.sqlite3`"
|
|
44
|
+
task status: :environment do
|
|
45
45
|
options = parse_argv_options
|
|
46
46
|
database = options.delete(:"--database") || options.delete(:"-database")
|
|
47
47
|
|
|
48
|
-
puts Litestream::Commands::Output.format(Litestream::Commands.
|
|
48
|
+
puts Litestream::Commands::Output.format(Litestream::Commands.status(database, **options))
|
|
49
49
|
end
|
|
50
50
|
|
|
51
|
-
desc "
|
|
52
|
-
task
|
|
53
|
-
|
|
54
|
-
|
|
51
|
+
desc "Explain the removal of the generations command"
|
|
52
|
+
task generations: :environment do
|
|
53
|
+
abort "`generations` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)"
|
|
54
|
+
end
|
|
55
55
|
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
)
|
|
56
|
+
desc "Explain the removal of the snapshots command"
|
|
57
|
+
task snapshots: :environment do
|
|
58
|
+
abort "`snapshots` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)"
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
desc "Explain the removal of the wal command"
|
|
62
|
+
task wal: :environment do
|
|
63
|
+
abort "`wal` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)"
|
|
59
64
|
end
|
|
60
65
|
|
|
61
66
|
private
|
|
62
67
|
|
|
68
|
+
# Options after `--` come as `-flag`, `-flag=value` or `-flag value`. Rake
|
|
69
|
+
# also reads a bare word there as one more task to run, so each value gets
|
|
70
|
+
# an empty task of its own to keep Rake from aborting once this one is done.
|
|
63
71
|
def parse_argv_options
|
|
64
72
|
options = {}
|
|
65
73
|
if (separator_index = ARGV.index("--"))
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
.
|
|
74
|
+
awaiting_value = nil
|
|
75
|
+
ARGV.drop(separator_index + 1).each do |arg|
|
|
76
|
+
if arg.start_with?("-")
|
|
77
|
+
key, value = arg.split("=", 2)
|
|
78
|
+
options[key] = value
|
|
79
|
+
awaiting_value = value.nil? ? key : nil
|
|
80
|
+
elsif awaiting_value
|
|
81
|
+
options[awaiting_value] = arg
|
|
82
|
+
awaiting_value = nil
|
|
83
|
+
Rake::Task.define_task(arg)
|
|
84
|
+
end
|
|
85
|
+
end
|
|
69
86
|
end
|
|
70
87
|
options.symbolize_keys!
|
|
71
88
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: litestream
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.16.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Stephen Margheim
|
|
@@ -113,20 +113,6 @@ dependencies:
|
|
|
113
113
|
- - ">="
|
|
114
114
|
- !ruby/object:Gem::Version
|
|
115
115
|
version: '0'
|
|
116
|
-
- !ruby/object:Gem::Dependency
|
|
117
|
-
name: rubyzip
|
|
118
|
-
requirement: !ruby/object:Gem::Requirement
|
|
119
|
-
requirements:
|
|
120
|
-
- - ">="
|
|
121
|
-
- !ruby/object:Gem::Version
|
|
122
|
-
version: '0'
|
|
123
|
-
type: :development
|
|
124
|
-
prerelease: false
|
|
125
|
-
version_requirements: !ruby/object:Gem::Requirement
|
|
126
|
-
requirements:
|
|
127
|
-
- - ">="
|
|
128
|
-
- !ruby/object:Gem::Version
|
|
129
|
-
version: '0'
|
|
130
116
|
email:
|
|
131
117
|
- stephen.margheim@gmail.com
|
|
132
118
|
executables:
|