litestream 0.3.3-arm64-darwin → 0.4.0-arm64-darwin

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 39f5751dd238303fd70e98087d37e08c98d62697efe0df0715959eb5ded32ed0
4
- data.tar.gz: ca0707ceb8dda436cc01650ea812c7ca42b22516dec78e4475a264f7ab3d5ae6
3
+ metadata.gz: 7de2d916b01f455a5321ec8cf30b4e89a02fc55c7a2d849ed8f5019565aed96d
4
+ data.tar.gz: bcd5e6b7648cd4184cca408806d1cdd7527766139ab2f598b9d0363601054b72
5
5
  SHA512:
6
- metadata.gz: 1c284c21e39d48c0aa3ef27896d3b40be4988a542e01d238132a9ea381e2466ab6a4f3e3f04ab233c52c8bb1fc84fb62088c12e2933a277e6ed09897f985d42e
7
- data.tar.gz: 57cf1e0217444fa58cf71966d146e947b1e32e42427a0547a6949d16fcd4a7d294b66972b93c5f82e7d09dc761af87c0fbdb3db997a824e5d937b98172b95f86
6
+ metadata.gz: 7823e7431f145aa34b405b3face42ad9e852b36eb3f2a8f1fc6946a6832ebf6ebcb1e4bb65942f5e4cb70d039ac4d2b90e47817adce694514e34c80200642970
7
+ data.tar.gz: f682feed2f370299d46e5971b8e78bf147d17471bacc43e334a6516055f620ba3dcd9c34df1b51363c4a40bd4d5ac6115cbd65900314e18d263cd8b00a7fdf7e
data/README.md CHANGED
@@ -49,13 +49,13 @@ LITESTREAM_INSTALL_DIR=.bin
49
49
 
50
50
  You configure the Litestream executable through the [`config/litestream.yml` file](https://litestream.io/reference/config/), which is a standard Litestream configuration file as if Litestream was running in a traditional installation.
51
51
 
52
- The gem streamlines the configuration process by providing a default configuration file for you. This configuration file will backup one application database to one replication bucket. In order to ensure that no secrets are stored in plain-text in your repository, this configuration file leverages Litestream's support for environment variables. The default configuration file looks like this:
52
+ The gem streamlines the configuration process by providing a default configuration file for you. This configuration file will backup all SQLite databases defined in your `config/database.yml` file to one replication bucket. In order to ensure that no secrets are stored in plain-text in your repository, this configuration file leverages Litestream's support for environment variables. The default configuration file looks like this if you only have one SQLite database:
53
53
 
54
54
  ```yaml
55
55
  dbs:
56
56
  - path: $LITESTREAM_DATABASE_PATH
57
57
  replicas:
58
- - url: $LITESTREAM_REPLICA_URL
58
+ - url: $LITESTREAM_REPLICA_BUCKET
59
59
  access-key-id: $LITESTREAM_ACCESS_KEY_ID
60
60
  secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY
61
61
  ```
@@ -66,7 +66,7 @@ The gem also provides a default initializer file at `config/initializers/litestr
66
66
  litestream_credentials = Rails.application.credentials.litestream
67
67
  Litestream.configure do |config|
68
68
  config.database_path = ActiveRecord::Base.connection_db_config.database
69
- config.replica_url = litestream_credentials.replica_url
69
+ config.replica_bucket = litestream_credentials.replica_bucket
70
70
  config.replica_key_id = litestream_credentials.replica_key_id
71
71
  config.replica_access_key = litestream_credentials.replica_access_key
72
72
  end
@@ -92,9 +92,94 @@ bin/rails litestream:replicate -- -exec "foreman start"
92
92
 
93
93
  This example utilizes the `-exec` option available on [the `replicate` command](https://litestream.io/reference/replicate/) which provides basic process management, since Litestream will exit when the child process exits. In this example, we only launch our collection of Rails application processes (like Rails and SolidQueue, for example) after the Litestream replication process is ready.
94
94
 
95
- The rake task is the recommended way to interact with the Litestream replication process in your Rails application or Ruby project. But, you _can_ take full manual control over the replication process and simply run the `litestream replicate --config config/litestream.yml` command to start the Litestream process. Since the gem installs the native executable via Bundler, the `litestream` command will be available in your `PATH`.
95
+ The Litestream `replicate` command supports the following options, which can be passed through the rake task:
96
+ ```shell
97
+ -config PATH
98
+ Specifies the configuration file.
99
+ Defaults to /etc/litestream.yml
100
+
101
+ -exec CMD
102
+ Executes a subcommand. Litestream will exit when the child
103
+ process exits. Useful for simple process management.
104
+
105
+ -no-expand-env
106
+ Disables environment variable expansion in configuration file.
107
+ ```
108
+
109
+ ### Restoration
96
110
 
97
- The full set of commands available to the `litestream` executable are covered in Litestream's [command reference](https://litestream.io/reference/). Currently, only the `replicate` command is provided as a rake task by the gem.
111
+ You can restore any replicated database at any point using the gem's provided `litestream:restore` rake task. This rake task requires that you specify which specific database you want to restore. As with the `litestream:replicate` task, you pass arguments to the rake task via argument forwarding. For example, to restore the production database, you would run:
112
+
113
+ ```shell
114
+ bin/rails litestream:restore -- --database=storage/production.sqlite3
115
+ # or
116
+ bundle exec rake litestream:restore -- --database=storage/production.sqlite3
117
+ ```
118
+
119
+ You can restore any of the databases specified in your `config/litestream.yml` file. The `--database` argument should be the path to the database file you want to restore and must match the value for the `path` key of one of your configured databases. The `litestream:restore` rake task will automatically load the configuration file and set the environment variables before calling the Litestream executable.
120
+
121
+ If you need to pass arguments through the rake task to the underlying `litestream` command, that can be done with additional forwarded arguments:
122
+
123
+ ```shell
124
+ bin/rails litestream:replicate -- --database=storage/production.sqlite3 --if-db-not-exists
125
+ ```
126
+
127
+ You can forward arguments in whatever order you like, you simply need to ensure that the `--database` argument is present. You can also use either a single-dash `-database` or double-dash `--database` argument format. The Litestream `restore` command supports the following options, which can be passed through the rake task:
128
+
129
+ ```shell
130
+ -o PATH
131
+ Output path of the restored database.
132
+ Defaults to original DB path.
133
+
134
+ -if-db-not-exists
135
+ Returns exit code of 0 if the database already exists.
136
+
137
+ -if-replica-exists
138
+ Returns exit code of 0 if no backups found.
139
+
140
+ -parallelism NUM
141
+ Determines the number of WAL files downloaded in parallel.
142
+ Defaults to 8
143
+
144
+ -replica NAME
145
+ Restore from a specific replica.
146
+ Defaults to replica with latest data.
147
+
148
+ -generation NAME
149
+ Restore from a specific generation.
150
+ Defaults to generation with latest data.
151
+
152
+ -index NUM
153
+ Restore up to a specific WAL index (inclusive).
154
+ Defaults to use the highest available index.
155
+
156
+ -timestamp TIMESTAMP
157
+ Restore to a specific point-in-time.
158
+ Defaults to use the latest available backup.
159
+
160
+ -config PATH
161
+ Specifies the configuration file.
162
+ Defaults to /etc/litestream.yml
163
+
164
+ -no-expand-env
165
+ Disables environment variable expansion in configuration file.
166
+ ```
167
+
168
+ ### Additional commands
169
+
170
+ The rake tasks are the recommended way to interact with the Litestream utility in your Rails application or Ruby project. But, you _can_ work directly with the Litestream CLI. Since the gem installs the native executable via Bundler, the `litestream` command will be available in your `PATH`.
171
+
172
+ The full set of commands available to the `litestream` executable are covered in Litestream's [command reference](https://litestream.io/reference/), but can be summarized as:
173
+
174
+ ```shell
175
+ litestream databases [arguments]
176
+ litestream generations [arguments] DB_PATH|REPLICA_URL
177
+ litestream replicate [arguments]
178
+ litestream restore [arguments] DB_PATH|REPLICA_URL
179
+ litestream snapshots [arguments] DB_PATH|REPLICA_URL
180
+ litestream version
181
+ litestream wal [arguments] DB_PATH|REPLICA_URL
182
+ ```
98
183
 
99
184
  ### Using in development
100
185
 
@@ -113,7 +198,7 @@ Once you have a MinIO server running, you can create a bucket for Litestream to
113
198
  ```ruby
114
199
  Litestream.configure do |config|
115
200
  config.database_path = ActiveRecord::Base.connection_db_config.database
116
- config.replica_url = "s3://mybkt.localhost:9000/"
201
+ config.replica_bucket = "s3://mybkt.localhost:9000/"
117
202
  config.replica_key_id = "minioadmin"
118
203
  config.replica_access_key = "minioadmin"
119
204
  end
data/exe/litestream CHANGED
@@ -5,7 +5,6 @@ require "litestream/commands"
5
5
 
6
6
  begin
7
7
  command = [Litestream::Commands.executable, *ARGV]
8
- puts command.inspect
9
8
  exec(*command)
10
9
  rescue Litestream::Commands::UnsupportedPlatformException, Litestream::Commands::ExecutableNotFoundException => e
11
10
  warn("ERROR: " + e.message)
@@ -6,16 +6,13 @@ module Litestream
6
6
  GEM_NAME = "litestream"
7
7
 
8
8
  # raised when the host platform is not supported by upstream litestream's binary releases
9
- class UnsupportedPlatformException < StandardError
10
- end
9
+ UnsupportedPlatformException = Class.new(StandardError)
11
10
 
12
11
  # raised when the litestream executable could not be found where we expected it to be
13
- class ExecutableNotFoundException < StandardError
14
- end
12
+ ExecutableNotFoundException = Class.new(StandardError)
15
13
 
16
14
  # raised when LITESTREAM_INSTALL_DIR does not exist
17
- class DirectoryNotFoundException < StandardError
18
- end
15
+ DirectoryNotFoundException = Class.new(StandardError)
19
16
 
20
17
  def self.platform
21
18
  [:cpu, :os].map { |m| Gem::Platform.local.send(m) }.join("-")
@@ -74,10 +71,9 @@ module Litestream
74
71
 
75
72
  def self.replicate(argv = {})
76
73
  if Litestream.configuration
77
- ENV["LITESTREAM_DATABASE_PATH"] = Litestream.configuration.database_path
78
- ENV["LITESTREAM_REPLICA_URL"] = Litestream.configuration.replica_url
79
- ENV["LITESTREAM_ACCESS_KEY_ID"] = Litestream.configuration.replica_key_id
80
- ENV["LITESTREAM_SECRET_ACCESS_KEY"] = Litestream.configuration.replica_access_key
74
+ ENV["LITESTREAM_REPLICA_BUCKET"] ||= Litestream.configuration.replica_bucket
75
+ ENV["LITESTREAM_ACCESS_KEY_ID"] ||= Litestream.configuration.replica_key_id
76
+ ENV["LITESTREAM_SECRET_ACCESS_KEY"] ||= Litestream.configuration.replica_access_key
81
77
  end
82
78
 
83
79
  args = {
@@ -93,5 +89,32 @@ module Litestream
93
89
  exec(*command)
94
90
  end
95
91
  end
92
+
93
+ def self.restore(database, argv = {})
94
+ if Litestream.configuration
95
+ ENV["LITESTREAM_REPLICA_BUCKET"] ||= Litestream.configuration.replica_bucket
96
+ ENV["LITESTREAM_ACCESS_KEY_ID"] ||= Litestream.configuration.replica_key_id
97
+ ENV["LITESTREAM_SECRET_ACCESS_KEY"] ||= Litestream.configuration.replica_access_key
98
+ end
99
+
100
+ dir, file = File.split(database)
101
+ ext = File.extname(file)
102
+ base = File.basename(file, ext)
103
+ now = Time.now.utc.strftime("%Y%m%d%H%M%S")
104
+
105
+ args = {
106
+ "--config" => Rails.root.join("config", "litestream.yml").to_s,
107
+ "-o" => File.join(dir, "#{base}-#{now}#{ext}")
108
+ }.merge(argv).to_a.flatten.compact
109
+
110
+ command = [executable, "restore", *args, database]
111
+ puts command.inspect
112
+
113
+ # To release the resources of the Ruby process, just fork and exit.
114
+ # The forked process executes litestream and replaces itself.
115
+ if fork.nil?
116
+ exec(*command)
117
+ end
118
+ end
96
119
  end
97
120
  end
@@ -8,7 +8,7 @@ module Litestream
8
8
  source_root File.expand_path("templates", __dir__)
9
9
 
10
10
  def copy_config_file
11
- template "config.yml", "config/litestream.yml"
11
+ template "config.yml.erb", "config/litestream.yml"
12
12
  end
13
13
 
14
14
  def copy_initializer_file
@@ -27,6 +27,16 @@ module Litestream
27
27
  end
28
28
  end
29
29
  end
30
+
31
+ private
32
+
33
+ def production_sqlite_databases
34
+ ActiveRecord::Base
35
+ .configurations
36
+ .configs_for(env_name: "production", include_hidden: true)
37
+ .select { |config| ["sqlite3", "litedb"].include? config.adapter }
38
+ .map(&:database)
39
+ end
30
40
  end
31
41
  end
32
42
  end
@@ -10,8 +10,12 @@
10
10
  #
11
11
  # For more details, see: https://litestream.io/reference/config/
12
12
  dbs:
13
- - path: $LITESTREAM_DATABASE_PATH
13
+ <%- production_sqlite_databases.each do |database| -%>
14
+ - path: <%= database %>
14
15
  replicas:
15
- - url: $LITESTREAM_REPLICA_URL
16
+ - type: s3
17
+ bucket: $LITESTREAM_REPLICA_BUCKET
18
+ path: <%= database %>
16
19
  access-key-id: $LITESTREAM_ACCESS_KEY_ID
17
20
  secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY
21
+ <%- end -%>
@@ -1,30 +1,25 @@
1
1
  # Use this hook to configure the litestream-ruby gem.
2
2
  # All configuration options will be available as environment variables, e.g.
3
- # config.database_path becomes LITESTREAM_DATABASE_PATH
3
+ # config.replica_bucket becomes LITESTREAM_REPLICA_BUCKET
4
4
  # This allows you to configure Litestream using Rails encrypted credentials,
5
5
  # or some other mechanism where the values are only avaialble at runtime.
6
6
 
7
7
  Litestream.configure do |config|
8
8
  # An example of using Rails encrypted credentials to configure Litestream.
9
9
  # litestream_credentials = Rails.application.credentials.litestream
10
- #
11
- # The absolute or relative path to a SQLite database file.
12
- # Litestream will monitor this file for changes and replicate them to the
13
- # any of the configured replicas specified for this database in the
14
- # `litestream.yml` configuration file.
15
- # When using SQLite as your database engine for ActiveRecord, you should always
16
- # set this to the path of your SQLite database file. You can do so using Rails'
17
- # existing knowledge of the database path.
18
- # config.database_path = ActiveRecord::Base.connection_db_config.database
19
10
 
20
- # Short-hand form of specifying a replica location.
21
- # When using S3, a value will look like "s3://mybkt.litestream.io/db"
22
- # Litestream also supports Azure Blog Storage, Backblaze B2, DigitalOcean Spaces,
11
+ # Replica-specific bucket location.
12
+ # This will be your bucket's URL without the `https://` prefix.
13
+ # For example, if you used DigitalOcean Spaces, your bucket URL could look like:
14
+ # https://myapp.fra1.digitaloceanspaces.com
15
+ # And so you should set your `replica_bucket` to:
16
+ # myapp.fra1.digitaloceanspaces.com
17
+ # Litestream supports Azure Blog Storage, Backblaze B2, DigitalOcean Spaces,
23
18
  # Scaleway Object Storage, Google Cloud Storage, Linode Object Storage, and
24
19
  # any SFTP server.
25
20
  # In this example, we are using Rails encrypted credentials to store the URL to
26
21
  # our storage provider bucket.
27
- # config.replica_url = litestream_credentials.replica_url
22
+ # config.replica_bucket = litestream_credentials.replica_bucket
28
23
 
29
24
  # Replica-specific authentication key.
30
25
  # Litestream needs authentication credentials to access your storage provider bucket.
@@ -1,3 +1,3 @@
1
1
  module Litestream
2
- VERSION = "0.3.3"
2
+ VERSION = "0.4.0"
3
3
  end
data/lib/litestream.rb CHANGED
@@ -11,10 +11,9 @@ module Litestream
11
11
  end
12
12
 
13
13
  class Configuration
14
- attr_accessor :database_path, :replica_url, :replica_key_id, :replica_access_key
14
+ attr_accessor :database_path, :replica_bucket, :replica_key_id, :replica_access_key
15
15
 
16
16
  def initialize
17
- @mailer_sender = "donotreply@example.com"
18
17
  end
19
18
  end
20
19
  end
@@ -6,15 +6,14 @@ namespace :litestream do
6
6
  next
7
7
  end
8
8
 
9
- puts "LITESTREAM_DATABASE_PATH=#{Litestream.configuration.database_path}"
10
- puts "LITESTREAM_REPLICA_URL=#{Litestream.configuration.replica_url}"
9
+ puts "LITESTREAM_REPLICA_BUCKET=#{Litestream.configuration.replica_bucket}"
11
10
  puts "LITESTREAM_ACCESS_KEY_ID=#{Litestream.configuration.replica_key_id}"
12
11
  puts "LITESTREAM_SECRET_ACCESS_KEY=#{Litestream.configuration.replica_access_key}"
13
12
 
14
13
  true
15
14
  end
16
15
 
17
- desc "Start a process to monitor and continuously replicate the SQLite databases defined in your configuration file"
16
+ desc 'Monitor and continuously replicate SQLite databases defined in your config file, e.g. rake litestream:replicate -- -exec "foreman start"'
18
17
  task replicate: :environment do
19
18
  options = {}
20
19
  if (separator_index = ARGV.index("--"))
@@ -25,4 +24,16 @@ namespace :litestream do
25
24
 
26
25
  Litestream::Commands.replicate(options)
27
26
  end
27
+
28
+ desc "Restore a SQLite database from a Litestream replica, e.g. rake litestream:restore -- -database=storage/production.sqlite3"
29
+ task restore: :environment do
30
+ options = {}
31
+ if (separator_index = ARGV.index("--"))
32
+ ARGV.slice(separator_index + 1, ARGV.length)
33
+ .map { |pair| pair.split("=") }
34
+ .each { |opt| options[opt[0]] = opt[1] || nil }
35
+ end
36
+
37
+ Litestream::Commands.restore(options.delete("--database") || options.delete("-database"), options)
38
+ end
28
39
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: litestream
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.3.3
4
+ version: 0.4.0
5
5
  platform: arm64-darwin
6
6
  authors:
7
7
  - Stephen Margheim
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2024-02-06 00:00:00.000000000 Z
11
+ date: 2024-04-17 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rubyzip
@@ -69,7 +69,7 @@ files:
69
69
  - lib/litestream.rb
70
70
  - lib/litestream/commands.rb
71
71
  - lib/litestream/generators/litestream/install_generator.rb
72
- - lib/litestream/generators/litestream/templates/config.yml
72
+ - lib/litestream/generators/litestream/templates/config.yml.erb
73
73
  - lib/litestream/generators/litestream/templates/initializer.rb
74
74
  - lib/litestream/railtie.rb
75
75
  - lib/litestream/upstream.rb