litestream 0.14.0-x86_64-linux → 0.16.0-x86_64-linux

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f5472149be2f48a05e36c2e2f1c3feb212355100f4e3085f90ed821559d49200
-  data.tar.gz: 7d745ffdd094a4148b7660f7ec32c531c125279ecb8d78ceef72bdea149f5394
+  metadata.gz: 1eea2243f44a53621bf8fa287f9b27e8471da73646a4f5d2db4a63402f42c30d
+  data.tar.gz: c759e8b2f48804096e02191e2150de4f685f066d861c5f3aefb07310eb275805
 SHA512:
-  metadata.gz: a5ef0cfce7f7fd68d235f873882f8651c9141bb5fabf7b9a4d2b8f5144de4f0658e741f2a3c0e7f09ef381b4016f897673c5b2d80064dad58c6d4778647f801c
-  data.tar.gz: a959c042fd3c62118cad6976ce554c014ecf6259ca62e1949c398f4cd5697d6ebe0aaa4121f7fd108bcfbdfee8facb9a0cd6391f4024e982116f9a5ed52adaf2
+  metadata.gz: 94f95bf5a0239fd400b4ae031c99349db0c1297910fab2fa3a2883e05568ac9e0cfba5d5085812205d0bf4003100df0a478c36e011a67fec20571e11dde805e5
+  data.tar.gz: 82ad7f444fce743f8212f42a2c1d5196b44af2b845301441da014d44035f3ea8504a49c99cf8dab3edc55ba1779b11f7257ff584e184c2529c2829326e65e8f5

data/README.md

@@ -85,16 +85,22 @@ The gem streamlines the configuration process by providing a default configurati
 The default configuration file looks like this if you only have one SQLite database:
 
 ```yaml
+snapshot:
+  interval: 24h
+  retention: 24h
+
 dbs:
   - path: storage/production.sqlite3
-    replicas:
-      - type: s3
-        path: storage/production.sqlite3
-        bucket: $LITESTREAM_REPLICA_BUCKET
-        access-key-id: $LITESTREAM_ACCESS_KEY_ID
-        secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY
+    replica:
+      type: s3
+      path: storage/production.sqlite3
+      bucket: $LITESTREAM_REPLICA_BUCKET
+      access-key-id: $LITESTREAM_ACCESS_KEY_ID
+      secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY
 ```
 
+In Litestream 0.5, snapshot interval and retention are global settings rather than per-replica settings.
+
 This is the default for Amazon S3. The full range of possible replica types (e.g. other S3-compatible object storage servers) are covered in Litestream's [replica guides](https://litestream.io/guides/#replica-guides).
 
 The gem also provides a default initializer file at `config/initializers/litestream.rb` that allows you to configure various variables referenced in the configuration file in Ruby. By providing a Ruby interface to these environment variables, you can use your preferred method of storing secrets. For example, the default generated file uses Rails' encrypted credentials to store your secrets.
@@ -126,6 +132,28 @@ end
 
 However, if you need manual control over the Litestream configuration, you can edit the `config/litestream.yml` file. The full range of possible configurations are covered in Litestream's [configuration reference](https://litestream.io/reference/config/).  
 
+#### Logging
+
+Litestream writes its logs to stdout, where they mix with your application's. Its `logging:` block can make them easy to separate, for example as JSON on stderr:
+
+```yaml
+logging:
+  level: info
+  type: json
+  stderr: true
+```
+
+#### Replicas that authenticate with a credentials file
+
+Some replica types read credentials from a file. Google Cloud Storage, for example, reads the path in `GOOGLE_APPLICATION_CREDENTIALS`. A file on your machine is not in the container that Kamal deploys, so pass its contents as a secret and write it out before the app starts, for example in `bin/docker-entrypoint`:
+
+```shell
+# .kamal/secrets: GCS_CREDENTIALS_JSON=$(cat config/gcs.json)
+# config/deploy.yml: env: { secret: [GCS_CREDENTIALS_JSON] }
+printf '%s' "$GCS_CREDENTIALS_JSON" > /rails/tmp/gcs.json
+export GOOGLE_APPLICATION_CREDENTIALS=/rails/tmp/gcs.json
+```
+
 ### Replication
 
 In order to stream changes to your configured replicas, you need to start the Litestream replication process.
@@ -179,7 +207,7 @@ You can restore any replicated database at any point using the gem's provided `l
 > [!NOTE]  
 > During the restoration process, you need to prevent any interaction with ActiveRecord/SQLite, such as from a running `rails server` or `rails console` instance. If there is any interaction, Rails might regenerate the production database and prevent restoration via litestream. If this happens, you might get a "cannot restore, output path already exists" error.
 
-1. Rename the production (`production.sqlite3`, `production.sqlite3-shm`, and `production.sqlite3-wal`) databases (**recommended**) or alternatively delete. To delete the production databases locally, you can run the following at your own risk:
+1. Rename the production database and its SQLite sidecar files (**recommended**) or alternatively delete them. To delete the production databases locally, you can run the following at your own risk:
    ```shell
    # DANGEROUS OPERATION, consider renaming database files instead
    bin/rails db:drop DISABLE_DATABASE_ENVIRONMENT_CHECK=1
@@ -215,26 +243,19 @@ You can forward arguments in whatever order you like, you simply need to ensure
 -if-replica-exists
     Returns exit code of 0 if no backups found.
 
--parallelism NUM
-    Determines the number of WAL files downloaded in parallel.
-    Defaults to 8
-
--replica NAME
-    Restore from a specific replica.
-    Defaults to replica with latest data.
-
--generation NAME
-    Restore from a specific generation.
-    Defaults to generation with latest data.
-
--index NUM
-    Restore up to a specific WAL index (inclusive).
-    Defaults to use the highest available index.
+-txid TXID
+    Restore through a specific 16-character hexadecimal transaction ID.
 
 -timestamp TIMESTAMP
     Restore to a specific point-in-time.
     Defaults to use the latest available backup.
 
+-json
+    Print the restore result as JSON.
+
+-dry-run
+    Print the restore plan without writing the database.
+
 -config PATH
     Specifies the configuration file.
     Defaults to /etc/litestream.yml
@@ -387,79 +408,59 @@ bin/rails litestream:databases
 This will return a list of databases and their configured replicas:
 
 ```
-path                                                 replicas
+path                                                 replica
 /Users/you/Code/your-app/storage/production.sqlite3  s3
 ```
 
-You can also list the generations of a specific database:
+You can inspect local replication status for all databases or one specific database:
 
 ```shell
-bin/rails litestream:generations -- --database=storage/production.sqlite3
+bin/rails litestream:status -- --database=storage/production.sqlite3
 ```
 
-This will list all generations for the specified database, including stats about their lag behind the primary database and the time range they cover:
+This reads local state and does not require a running Litestream process:
 
 ```
-name  generation        lag     start                 end
-s3    a295b16a796689f3  -156ms  2024-04-17T00:01:19Z  2024-04-17T00:01:19Z
+database                                             status  local txid        wal size
+/Users/you/Code/your-app/storage/production.sqlite3  ok      000000000000000a  128 kB
 ```
 
-You can list the snapshots available for a database:
+You can list all remote LTX files, including the level-9 snapshot:
 
 ```shell
-bin/rails litestream:snapshots -- --database=storage/production.sqlite3
+bin/rails litestream:ltx -- --database=storage/production.sqlite3 --level=all
 ```
 
-This command lists snapshots available for that specified database:
+The command returns the compaction level, transaction range, byte size, and creation time:
 
 ```
-replica  generation        index  size     created
-s3       a295b16a796689f3  1      4645465  2024-04-17T00:01:19Z
-```
-
-Finally, you can list the wal files available for a database:
-
-```shell
-bin/rails litestream:wal -- --database=storage/production.sqlite3
-```
-
-This command lists wal files available for that specified database:
-
-```
-replica  generation        index  offset    size     created
-s3       a295b16a796689f3  1      0         2036     2024-04-17T00:01:19Z
+level  min_txid          max_txid          size  created
+0      0000000000000008  000000000000000a  1013  2026-09-08T03:16:43Z
 ```
 
 ### Running commands from Ruby
 
 In addition to the provided rake tasks, you can also run Litestream commands directly from Ruby. The gem provides a `Litestream::Commands` module that wraps the Litestream CLI commands. This is particularly useful for the introspection commands, as you can use the output in your Ruby code.
 
-The `Litestream::Commands.databases` method returns an array of hashes with the "path" and "replicas" keys for each database:
+Pass `json: true` to return the 0.5 CLI's JSON values with string keys. The `Litestream::Commands.databases` method returns each database path and replica type:
 
 ```ruby
-Litestream::Commands.databases
-# => [{"path"=>"/Users/you/Code/your-app/storage/production.sqlite3", "replicas"=>"s3"}]
+Litestream::Commands.databases(json: true)
+# => [{"path"=>"/Users/you/Code/your-app/storage/production.sqlite3", "replica"=>"s3"}]
 ```
 
-The `Litestream::Commands.generations` method returns an array of hashes with the "name", "generation", "lag", "start", and "end" keys for each generation:
+The `status` method returns local state without connecting to the replica or requiring a daemon:
 
 ```ruby
-Litestream::Commands.generations('storage/production.sqlite3')
-# => [{"name"=>"s3", "generation"=>"5f4341bc3d22d615", "lag"=>"3s", "start"=>"2024-04-17T19:48:09Z", "end"=>"2024-04-17T19:48:09Z"}]
+Litestream::Commands.status("storage/production.sqlite3", json: true)
+# => [{"database"=>"/Users/you/Code/your-app/storage/production.sqlite3", "status"=>"ok", "local_txid"=>"000000000000000a", "wal_size"=>"128 kB"}]
 ```
 
-The `Litestream::Commands.snapshots` method returns an array of hashes with the "replica", "generation", "index", "size", and "created" keys for each snapshot:
+The `ltx` method lists remote LTX files. An unreplicated database returns an empty array:
 
 ```ruby
-Litestream::Commands.snapshots('storage/production.sqlite3')
-# => [{"replica"=>"s3", "generation"=>"5f4341bc3d22d615", "index"=>"0", "size"=>"4645465", "created"=>"2024-04-17T19:48:09Z"}]
-```
-
-The `Litestream::Commands.wal` method returns an array of hashes with the "replica", "generation", "index", "offset","size", and "created" keys for each wal:
-
-```ruby
-Litestream::Commands.wal('storage/production.sqlite3')
-# => [{"replica"=>"s3", "generation"=>"5f4341bc3d22d615", "index"=>"0",  "offset"=>"0", "size"=>"2036", "created"=>"2024-04-17T19:48:09Z"}]
+Litestream::Commands.ltx("storage/production.sqlite3", json: true, "--level" => "all")
+# => [{"level"=>0, "min_txid"=>"0000000000000008", "max_txid"=>"000000000000000a", "size"=>1013, "timestamp"=>"2026-09-08T03:16:43Z"}]
 ```
 
 You can also restore a database programmatically using the `Litestream::Commands.restore` method, which returns the path to the restored database:
@@ -479,14 +480,29 @@ The full set of commands available to the `litestream` executable are covered in
 
 ```shell
 litestream databases [arguments]
-litestream generations [arguments] DB_PATH|REPLICA_URL
+litestream info [arguments]
+litestream list [arguments]
+litestream ltx [arguments] DB_PATH
+litestream register [arguments]
 litestream replicate [arguments]
-litestream restore [arguments] DB_PATH|REPLICA_URL
-litestream snapshots [arguments] DB_PATH|REPLICA_URL
+litestream reset [arguments]
+litestream restore [arguments] DB_PATH
+litestream start [arguments]
+litestream status [arguments] [DB_PATH]
+litestream stop [arguments]
+litestream sync [arguments]
+litestream unregister [arguments]
 litestream version
-litestream wal [arguments] DB_PATH|REPLICA_URL
 ```
 
+### Upgrading from 0.3
+
+Litestream 0.5 replicates each database to a single `replica:` map and moves retention and snapshot settings into the root-level `snapshot:` block. A `replicas:` list with one entry still works. A list with more than one entry stops `litestream replicate` from starting ("multiple replicas on a single database are no longer supported"), and when the Puma plugin sees Litestream exit it stops Puma, so reduce each database to one replica before upgrading. The `databases` command and Ruby method now return a `"replica"` key instead of `"replicas"`.
+
+The removed `generations`, `snapshots`, and `wal` command methods and rake tasks now raise an error pointing to `ltx`, which replaces all three forms of remote backup introspection.
+
+The 0.5 `restore` command auto-detects both 0.3 and LTX backups and prefers whichever backup is newer. Existing 0.3 `generations/` objects are left in place; you can delete them after the LTX history covers your full retention window. To roll back, reinstall a 0.15.x release of this gem, which bundles Litestream 0.3.
+
 ### Using in development
 
 By default, if you install the gem and configure via `puma.rb` or `Procfile`, Litestream will not start in development.

data/app/controllers/litestream/application_controller.rb

@@ -1,5 +1,7 @@
 module Litestream
   class ApplicationController < Litestream.base_controller_class.constantize
+    layout "litestream/application"
+
     protect_from_forgery with: :exception
 
     if Litestream.password

data/app/views/litestream/processes/show.html.erb

@@ -56,63 +56,51 @@
         </div>
 
         <br />
-        <section id="generations" class="ml-6">
-          <% database['generations'].each do |generation| %>
-            <details id="<%= generation['generation'] %>" open="open">
-              <summary class="cursor-pointer rounded p-2 hover:bg-gray-50 dark:hover:bg-gray-800">
-                <code><%= generation['generation'] %></code>
-                (<em><%= generation['lag'] %> lag</em>)
-              </summary>
+        <section class="ml-6">
+          <% if database['error'] %>
+            <p class="text-red-600 dark:text-red-400"><%= database['error'] %></p>
+          <% else %>
+            <dl class="grid grid-cols-[fit-content(100%)_1fr] gap-x-4">
+              <dt class="font-bold">Status</dt>
+              <dd><%= database.dig('status', 'status') %></dd>
+              <dt class="font-bold">Local TXID</dt>
+              <dd><code><%= database.dig('status', 'local_txid') %></code></dd>
+              <dt class="font-bold">WAL size</dt>
+              <dd><%= number_to_human_size database.dig('status', 'wal_size') %></dd>
+            </dl>
 
-              <dl class="ml-7 grid grid-cols-[fit-content(100%)_1fr] gap-x-4">
-                <dt class="font-bold">Start</dt>
-                <dd class="">
-                  <abbr title="<%= generation['start'] %>" class="underline decoration-dashed decoration-gray-500 cursor-help">
-                    <time datetime="<%= generation['start'] %>"><%= DateTime.parse(generation['start']).to_formatted_s(:db) %></time>
-                  </abbr>
-                </dd>
+            <br />
+            <% if database['ltx'] == [] %>
+              <p>No LTX files yet</p>
+            <% else %>
+              <table class="min-w-full divide-y divide-gray-300">
+                <thead>
+                  <tr>
+                    <th scope="col" class="whitespace-nowrap px-2 py-2 text-right text-sm font-semibold text-gray-900 dark:text-gray-100">Level</th>
+                    <th scope="col" class="whitespace-nowrap px-2 py-2 text-left text-sm font-semibold text-gray-900 dark:text-gray-100">Min TXID</th>
+                    <th scope="col" class="whitespace-nowrap px-2 py-2 text-left text-sm font-semibold text-gray-900 dark:text-gray-100">Max TXID</th>
+                    <th scope="col" class="whitespace-nowrap px-2 py-2 text-right text-sm font-semibold text-gray-900 dark:text-gray-100">Size</th>
+                    <th scope="col" class="whitespace-nowrap px-2 py-2 text-left text-sm font-semibold text-gray-900 dark:text-gray-100">Created</th>
+                  </tr>
+                </thead>
 
-                <dt class="font-bold">End</dt>
-                <dd class="">
-                  <abbr title="<%= generation['end'] %>" class="underline decoration-dashed decoration-gray-500 cursor-help">
-                    <time datetime="<%= generation['end'] %>"><%= DateTime.parse(generation['end']).to_formatted_s(:db) %></time>
-                  </abbr>
-                </dd>
-
-                <div class="col-span-2">
-                  <dt class="font-bold">Snapshots</dt>
-                  <dd class="">
-                    <table class="min-w-full divide-y divide-gray-300">
-                      <thead>
-                        <tr>
-                          <th scope="col" class="whitespace-nowrap px-2 py-2 text-left text-sm font-semibold text-gray-900 dark:text-gray-100">Created at</th>
-                          <th scope="col" class="whitespace-nowrap px-2 py-2 text-right text-sm font-semibold text-gray-900 dark:text-gray-100">Index</th>
-                          <th scope="col" class="whitespace-nowrap px-2 py-2 text-right text-sm font-semibold text-gray-900 dark:text-gray-100">Size</th>
-                        </tr>
-                      </thead>
-
-                      <tbody class="bg-white dark:bg-gray-900">
-                        <% generation['snapshots'].each do |snapshot| %>
-                          <tr class="align-top even:bg-gray-50 dark:even:bg-gray-800">
-                            <td scope="col" class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100">
-                              <abbr title="<%= snapshot['created'] %>" class="underline decoration-dashed decoration-gray-500 cursor-help">
-                                <time datetime="<%= snapshot['created'] %>"><%= DateTime.parse(snapshot['created']).to_formatted_s(:db) %></time>
-                              </abbr>
-                            </td>
-                            <td scope="col" class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100 text-right">
-                              <%= snapshot['index'] %>
-                            </td>
-                            <td scope="col" class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100 text-right">
-                              <%= number_to_human_size snapshot['size'] %>
-                            </td>
-                          </tr>
-                        <% end %>
-                      </tbody>
-                    </table>
-                  </dd>
-                </div>
-              </dl>
-            </details>
+                <tbody class="bg-white dark:bg-gray-900">
+                  <% database['ltx'].each do |ltx| %>
+                    <tr class="align-top even:bg-gray-50 dark:even:bg-gray-800">
+                      <td class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100 text-right"><%= ltx['level'] %></td>
+                      <td class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100"><code><%= ltx['min_txid'] %></code></td>
+                      <td class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100"><code><%= ltx['max_txid'] %></code></td>
+                      <td class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100 text-right"><%= number_to_human_size ltx['size'] %></td>
+                      <td class="whitespace-nowrap px-2 py-2 text-sm text-gray-900 dark:text-gray-100">
+                        <abbr title="<%= ltx['timestamp'] %>" class="underline decoration-dashed decoration-gray-500 cursor-help">
+                          <time datetime="<%= ltx['timestamp'] %>"><%= DateTime.parse(ltx['timestamp']).to_formatted_s(:db) %></time>
+                        </abbr>
+                      </td>
+                    </tr>
+                  <% end %>
+                </tbody>
+              </table>
+            <% end %>
           <% end %>
         </section>
       </li>

data/exe/x86_64-linux/litestream.sha256

@@ -0,0 +1 @@
+cfb371176d164437ae869f8351cfde49bd1804ae71c61923f75c9cba9c9c006d

data/lib/litestream/commands.rb

@@ -1,3 +1,5 @@
+require "json"
+require "open3"
 require_relative "upstream"
 
 module Litestream
@@ -20,6 +22,9 @@ module Litestream
     # raised when a litestream command fails
     CommandFailedException = Class.new(StandardError)
 
+    # raised when a litestream command times out
+    CommandTimeoutException = Class.new(CommandFailedException)
+
     module Output
       class << self
         def format(data)
@@ -47,7 +52,10 @@ module Litestream
         litestream_install_dir = ENV["LITESTREAM_INSTALL_DIR"]
         if litestream_install_dir
           if File.directory?(litestream_install_dir)
-            warn "NOTE: using LITESTREAM_INSTALL_DIR to find litestream executable: #{litestream_install_dir}"
+            unless @litestream_install_dir_noted
+              warn "NOTE: using LITESTREAM_INSTALL_DIR to find litestream executable: #{litestream_install_dir}"
+              @litestream_install_dir_noted = true
+            end
             exe_path = litestream_install_dir
             exe_file = File.expand_path(File.join(litestream_install_dir, "litestream"))
           else
@@ -94,12 +102,14 @@ module Litestream
         exe_file
       end
 
-      # Replicate can be run either as a fork or in the same process, depending on the context.
-      # Puma will start replication as a forked process, while running replication from a rake
-      # tasks won't.
+      # Replicate runs either as a child process or in the foreground, depending on the context.
+      # The Puma plugin starts it as a child and gets its pid back; the rake task runs it in the
+      # foreground and streams its output.
       def replicate(async: false, **argv)
         cmd = prepare("replicate", argv)
         run_replicate(cmd, async: async)
+      rescue CommandFailedException
+        raise
       rescue
         raise CommandFailedException, "Failed to execute `#{cmd.join(" ")}`"
       end
@@ -114,35 +124,48 @@ module Litestream
         execute("databases", argv)
       end
 
-      def generations(database, **argv)
-        raise DatabaseRequiredException, "database argument is required for generations command, e.g. litestream:generations -- --database=path/to/database.sqlite" if database.nil?
+      def ltx(database, **argv)
+        raise DatabaseRequiredException, "database argument is required for ltx command, e.g. litestream:ltx -- --database=path/to/database.sqlite" if database.nil?
 
-        execute("generations", argv, database)
+        execute("ltx", argv, database)
       end
 
-      def snapshots(database, **argv)
-        raise DatabaseRequiredException, "database argument is required for snapshots command, e.g. litestream:snapshots -- --database=path/to/database.sqlite" if database.nil?
+      # Litestream 0.5 filters `status` by the absolute database path, while
+      # `ltx` and `restore` match the path as written in the config file, so a
+      # relative path is expanded here to keep the documented
+      # `--database=storage/production.sqlite3` form working.
+      def status(database = nil, **argv)
+        execute("status", argv, database && File.expand_path(database))
+      end
 
-        execute("snapshots", argv, database)
+      def generations(*)
+        raise CommandFailedException, "`generations` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)"
       end
 
-      def wal(database, **argv)
-        raise DatabaseRequiredException, "database argument is required for wal command, e.g. litestream:wal -- --database=path/to/database.sqlite" if database.nil?
+      def snapshots(*)
+        raise CommandFailedException, "`snapshots` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)"
+      end
 
-        execute("wal", argv, database)
+      def wal(*)
+        raise CommandFailedException, "`wal` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)"
       end
 
       private
 
       def execute(command, argv = {}, database = nil, tabled_output: true)
-        cmd = prepare(command, argv, database)
-        results = run(cmd, tabled_output: tabled_output)
-
-        if Array === results && results.one? && results[0]["level"] == "ERROR"
-          raise CommandFailedException, "Failed to execute `#{cmd.join(" ")}`; Reason: #{results[0]["error"]}"
+        argv = argv.stringify_keys
+        timeout = argv.delete("timeout")
+        output = if argv.delete("json")
+          argv["-json"] = nil
+          :json
+        elsif tabled_output
+          :table
         else
-          results
+          :raw
         end
+
+        cmd = prepare(command, argv, database)
+        run(cmd, output: output, timeout: timeout)
       end
 
       def prepare(command, argv = {}, database = nil)
@@ -154,29 +177,101 @@ module Litestream
 
         args = {
           "--config" => Litestream.config_path.to_s
-        }.merge(argv.stringify_keys).to_a.flatten.compact
-        cmd = [executable, command, *args, database].compact
+        }.merge(argv.stringify_keys).to_a.flatten.compact.map(&:to_s)
+        cmd = [executable, command, *args, database].compact.map(&:to_s)
         puts cmd.inspect if ENV["DEBUG"]
 
         cmd
       end
 
-      def run(cmd, tabled_output:)
-        stdout = `#{cmd.join(" ")}`.chomp
-        return stdout unless tabled_output
+      # Runs the command without a shell and returns its parsed stdout. A non-zero
+      # exit raises with stderr. With a timeout, the command runs in its own
+      # process group and is killed (TERM, then KILL) when the deadline passes.
+      def run(cmd, output:, timeout: nil)
+        stdin, stdout, stderr, wait_thread = Open3.popen3(*cmd, pgroup: true)
+        stdin.close
+        stdout_reader = Thread.new { stdout.read }
+        stderr_reader = Thread.new { stderr.read }
+
+        # The readers finish when the last process holding the pipes exits, so
+        # waiting on them covers descendants the direct child may have left behind.
+        unless wait_thread.join(timeout) && stdout_reader.join(timeout) && stderr_reader.join(timeout)
+          kill_process_group("TERM", wait_thread.pid)
+          wait_thread.join(1)
+          kill_process_group("KILL", wait_thread.pid)
+          wait_thread.join
+          [stdout_reader, stderr_reader].each(&:join)
+          raise CommandTimeoutException, "Failed to execute `#{cmd[1]}`: timed out after #{timeout} seconds"
+        end
+
+        status = wait_thread.value
+        unless status.success?
+          raise CommandFailedException, "Failed to execute `#{cmd[1]}` (exit status #{status.exitstatus}): #{stderr_reader.value.strip[0, 500]}"
+        end
+
+        case output
+        when :json then parse_json(cmd, stdout_reader.value)
+        when :table then parse_table(stdout_reader.value)
+        else stdout_reader.value
+        end
+      ensure
+        [stdout_reader, stderr_reader].each { |reader| reader&.join }
+        [stdin, stdout, stderr].each { |io| io&.close unless io&.closed? }
+      end
+
+      # macOS answers EPERM rather than ESRCH when every process left in the
+      # group has exited but not yet been reaped.
+      def kill_process_group(signal, pid)
+        Process.kill(signal, -pid)
+      rescue Errno::ESRCH, Errno::EPERM
+      end
+
+      # Two opt-in restore skips (-if-db-not-exists when the output exists,
+      # -if-replica-exists with no backups) exit 0 and print one logfmt line on
+      # stdout instead of JSON. They come back as {"skipped" => true, "message" => ...}.
+      def parse_json(cmd, stdout)
+        stdout = stdout.strip
+        return if stdout.empty?
+        return JSON.parse(stdout) if stdout.start_with?("{", "[")
+
+        skipped = stdout.match(/\Atime=\S+ level=\S+ msg=(?:"([^"]*)"|(\S+))/)
+        return {"skipped" => true, "message" => skipped[1] || skipped[2]} if skipped
+
+        raise CommandFailedException, "Unexpected output from `#{cmd[1]}`: #{stdout.lines.first.to_s.strip[0, 200]}"
+      end
+
+      # Litestream pads columns with at least two spaces, and some headers and
+      # values hold a single one ("local txid", "231 kB", "no database").
+      def parse_table(stdout)
+        keys, *rows = stdout.strip.split("\n").map { _1.strip.split(/\s{2,}/) }
+        return [] unless keys
 
-        keys, *rows = stdout.split("\n").map { _1.split(/\s+/) }
         rows.map { keys.zip(_1).to_h }
       end
 
+      # Async replication returns the pid of the litestream process itself so a
+      # supervisor can signal and wait on it. Process.spawn is fork-and-exec in
+      # one call, so the pid it returns is the one that becomes litestream. The
+      # previous `exec(*cmd) if fork.nil?` also execs in a child but returns
+      # into the parent too, and the Puma plugin wrapped that in a fork of its
+      # own, so it recorded a pid that exited immediately while the daemon was
+      # reparented to init and outlived Puma.
       def run_replicate(cmd, async:)
         if async
-          exec(*cmd) if fork.nil?
+          Process.spawn(*cmd)
         else
           # When running in-process, we capture output continuously and write to stdout.
           IO.popen(cmd, err: [:child, :out]) do |io|
             io.each_line { |line| puts line }
           end
+          status = $?
+          # `replicate` runs until it is signalled, so a signal is how it is
+          # meant to end. A non-zero exit is litestream failing to start or
+          # dying on its own, which is invisible today: the task exits 0 and a
+          # supervisor sees a clean stop rather than a crash to restart.
+          if status && !status.success? && !status.signaled?
+            raise CommandFailedException, "Failed to execute `replicate` (exit status #{status.exitstatus})"
+          end
         end
       end
     end

data/lib/litestream/generators/litestream/templates/config.yml.erb

@@ -9,13 +9,18 @@
 # `replicate` command.
 #
 # For more details, see: https://litestream.io/reference/config/
+# Retention and snapshot settings are global in Litestream 0.5.
+snapshot:
+  interval: 24h
+  retention: 24h
+
 dbs:
   <%- production_sqlite_databases.each do |database| -%>
   - path: <%= database %>
-    replicas:
-      - type: s3
-        bucket: $LITESTREAM_REPLICA_BUCKET
-        path: <%= database %>
-        access-key-id: $LITESTREAM_ACCESS_KEY_ID
-        secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY
+    replica:
+      type: s3
+      bucket: $LITESTREAM_REPLICA_BUCKET
+      path: <%= database %>
+      access-key-id: $LITESTREAM_ACCESS_KEY_ID
+      secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY
   <%- end -%>

data/lib/litestream/upstream.rb

@@ -1,14 +1,25 @@
 module Litestream
   module Upstream
-    VERSION = "v0.3.13"
+    VERSION = "0.5.17"
 
     # rubygems platform name => upstream release filename
     NATIVE_PLATFORMS = {
       "aarch64-linux" => "litestream-#{VERSION}-linux-arm64.tar.gz",
-      "arm64-darwin" => "litestream-#{VERSION}-darwin-arm64.zip",
+      "arm64-darwin" => "litestream-#{VERSION}-darwin-arm64.tar.gz",
       "arm64-linux" => "litestream-#{VERSION}-linux-arm64.tar.gz",
-      "x86_64-darwin" => "litestream-#{VERSION}-darwin-amd64.zip",
-      "x86_64-linux" => "litestream-#{VERSION}-linux-amd64.tar.gz"
+      "x86_64-darwin" => "litestream-#{VERSION}-darwin-x86_64.tar.gz",
+      "x86_64-linux" => "litestream-#{VERSION}-linux-x86_64.tar.gz"
     }
+
+    CHECKSUMS = {
+      "litestream-0.5.17-linux-x86_64.tar.gz" => "cfb371176d164437ae869f8351cfde49bd1804ae71c61923f75c9cba9c9c006d",
+      "litestream-0.5.17-linux-arm64.tar.gz" => "f8ca4a050095c1efbda2c4365172e61bf9d955ea0d9ac42f448b52e51819baa5",
+      "litestream-0.5.17-darwin-x86_64.tar.gz" => "891875af09db152e93a4b31a8a79f538ce7ce702c132803cfe0a831e7cb1b7db",
+      "litestream-0.5.17-darwin-arm64.tar.gz" => "e211f68ff7658d19f193f2914417afdf8f89a053ff8f263e5d6b3b1d3bbc7b08"
+    }
+
+    def self.download_url(filename)
+      "https://github.com/benbjohnson/litestream/releases/download/v#{VERSION}/#{filename}"
+    end
   end
 end

data/lib/litestream/version.rb

@@ -1,3 +1,3 @@
 module Litestream
-  VERSION = "0.14.0"
+  VERSION = "0.16.0"
 end

data/lib/litestream.rb

@@ -1,10 +1,23 @@
 # frozen_string_literal: true
 
+require "etc"
 require "sqlite3"
+require "time"
 
 module Litestream
   VerificationFailure = Class.new(StandardError)
 
+  PROCESS_STATES = {
+    "D" => "uninterruptible",
+    "I" => "idle",
+    "R" => "running",
+    "S" => "sleeping",
+    "T" => "stopped",
+    "U" => "uninterruptible",
+    "Z" => "zombie"
+  }.freeze
+  private_constant :PROCESS_STATES
+
   class << self
     attr_writer :configuration
 
@@ -107,20 +120,18 @@ module Litestream
     end
 
     def databases
-      databases = Commands.databases
+      databases = Commands.databases(json: true)
 
       databases.each do |db|
-        generations = Commands.generations(db["path"])
-        snapshots = Commands.snapshots(db["path"])
-        db["path"] = db["path"].gsub(Rails.root.to_s, "[ROOT]")
-
-        db["generations"] = generations.map do |generation|
-          id = generation["generation"]
-          replica = generation["name"]
-          generation["snapshots"] = snapshots.select { |snapshot| snapshot["generation"] == id && snapshot["replica"] == replica }
-            .map { |s| s.slice("index", "size", "created") }
-          generation.slice("generation", "name", "lag", "start", "end", "snapshots")
+        path = db["path"]
+        begin
+          db["status"] = Commands.status(path, json: true).first
+          db["ltx"] = Commands.ltx(path, :json => true, "--level" => "all")
+        rescue Commands::CommandFailedException => error
+          db["error"] = error.message
         end
+
+        db["path"] = path.gsub(Rails.root.to_s, "[ROOT]")
       end
     end
 
@@ -163,30 +174,58 @@ module Litestream
       info
     end
 
+    # Linux has every process under /proc, even in images that ship no `ps`
+    # (Debian slim) or BusyBox's, which lacks `-x` and `lstart` (Alpine).
     def process_info
-      litestream_replicate_ps = `ps -ax | grep litestream | grep replicate`
-      exit_code = $?.exitstatus
-      return unless exit_code.zero?
+      File.directory?("/proc/self") ? proc_process_info : ps_process_info
+    end
 
-      info = {}
-      litestream_replicate_ps.chomp.split("\n").each do |line|
-        next unless line.include?("litestream replicate")
-
-        pid, * = line.split(" ")
-        info[:pid] = pid
-        state, _, lstart = `ps -o "state,lstart" #{pid}`.chomp.split("\n").last.partition(/\s+/)
-
-        info[:status] = case state[0]
-        when "I" then "idle"
-        when "R" then "running"
-        when "S" then "sleeping"
-        when "T" then "stopped"
-        when "U" then "uninterruptible"
-        when "Z" then "zombie"
-        end
-        info[:started] = DateTime.strptime(lstart.strip, "%a %b %d %H:%M:%S %Y")
+    def proc_process_info(proc_dir = "/proc")
+      boot_time = File.foreach(File.join(proc_dir, "stat")).grep(/\Abtime /).first.split.last.to_i
+
+      Dir.glob(File.join(proc_dir, "[0-9]*")).each do |dir|
+        executable, command = File.read(File.join(dir, "cmdline")).split("\0")
+        next unless File.basename(executable.to_s) == "litestream" && command == "replicate"
+
+        # Fields are counted after the command name's closing paren, since the
+        # name itself may contain spaces and parens.
+        state, *fields = File.read(File.join(dir, "stat")).rpartition(")").last.split
+        start_ticks = fields[18].to_i
+
+        return {
+          pid: File.basename(dir),
+          status: PROCESS_STATES[state[0]],
+          started: Time.at(boot_time + start_ticks.fdiv(Etc.sysconf(Etc::SC_CLK_TCK))).to_datetime
+        }
+      rescue SystemCallError
+        # the process exited while being read, or belongs to another user
+        next
       end
-      info
+      nil
+    end
+
+    # macOS. The command name is matched on its own so that a shell or editor
+    # whose arguments mention "litestream replicate" is not mistaken for it.
+    def ps_process_info
+      output = `ps -axo pid=,stat=,lstart=,comm=`
+      return unless $?.success?
+
+      output.each_line do |line|
+        # lstart is always five words ("Mon Jul  1 11:10:58 2024"); comm is last
+        # because it may contain spaces.
+        pid, state, *lstart, command = line.strip.split(" ", 8)
+        next unless File.basename(command.to_s) == "litestream"
+        next unless `ps -o args= -p #{pid}`.match?(/litestream replicate(?: |$)/)
+
+        return {
+          pid: pid,
+          status: PROCESS_STATES[state[0]],
+          started: Time.strptime(lstart.join(" "), "%a %b %d %H:%M:%S %Y").to_datetime
+        }
+      end
+      nil
+    rescue Errno::ENOENT
+      nil
     end
   end
 end

data/lib/puma/plugin/litestream.rb

@@ -2,17 +2,13 @@ require "puma/plugin"
 
 # Copied from https://github.com/rails/solid_queue/blob/15408647f1780033dad223d3198761ea2e1e983e/lib/puma/plugin/solid_queue.rb
 Puma::Plugin.create do
-  attr_reader :puma_pid, :litestream_pid, :log_writer
+  attr_reader :litestream_pid, :log_writer
 
   def start(launcher)
     @log_writer = launcher.log_writer
-    @puma_pid = $$
 
     launcher.events.on_booted do
-      @litestream_pid = fork do
-        Thread.new { monitor_puma }
-        Litestream::Commands.replicate(async: true)
-      end
+      @litestream_pid = Litestream::Commands.replicate(async: true)
 
       in_background do
         monitor_litestream
@@ -33,10 +29,6 @@ Puma::Plugin.create do
   rescue Errno::ECHILD, Errno::ESRCH
   end
 
-  def monitor_puma
-    monitor(:puma_dead?, "Detected Puma has gone away, stopping Litestream...")
-  end
-
   def monitor_litestream
     monitor(:litestream_dead?, "Detected Litestream has gone away, stopping Puma...")
   end
@@ -59,10 +51,6 @@ Puma::Plugin.create do
     true
   end
 
-  def puma_dead?
-    Process.ppid != puma_pid
-  end
-
   def log(...)
     log_writer.log(...)
   end

data/lib/tasks/litestream_tasks.rake

@@ -32,40 +32,57 @@ namespace :litestream do
     puts Litestream::Commands::Output.format(Litestream::Commands.databases(**options))
   end
 
-  desc "List all generations for a database or replica, for example `rake litestream:generations -- -database=storage/production.sqlite3`"
-  task generations: :environment do
+  desc "List all LTX files for a database or replica, for example `rake litestream:ltx -- -database=storage/production.sqlite3`"
+  task ltx: :environment do
     options = parse_argv_options
     database = options.delete(:"--database") || options.delete(:"-database")
 
-    puts Litestream::Commands::Output.format(Litestream::Commands.generations(database, **options))
+    puts Litestream::Commands::Output.format(Litestream::Commands.ltx(database, **options))
   end
 
-  desc "List all snapshots for a database or replica, for example `rake litestream:snapshots -- -database=storage/production.sqlite3`"
-  task snapshots: :environment do
+  desc "Show replication status, for example `rake litestream:status -- -database=storage/production.sqlite3`"
+  task status: :environment do
     options = parse_argv_options
     database = options.delete(:"--database") || options.delete(:"-database")
 
-    puts Litestream::Commands::Output.format(Litestream::Commands.snapshots(database, **options))
+    puts Litestream::Commands::Output.format(Litestream::Commands.status(database, **options))
   end
 
-  desc "List all wal files for a database or replica, for example `rake litestream:wal -- -database=storage/production.sqlite3`"
-  task wal: :environment do
-    options = parse_argv_options
-    database = options.delete(:"--database") || options.delete(:"-database")
+  desc "Explain the removal of the generations command"
+  task generations: :environment do
+    abort "`generations` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)"
+  end
 
-    puts Litestream::Commands::Output.format(
-      Litestream::Commands.wal(database, **options)
-    )
+  desc "Explain the removal of the snapshots command"
+  task snapshots: :environment do
+    abort "`snapshots` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)"
+  end
+
+  desc "Explain the removal of the wal command"
+  task wal: :environment do
+    abort "`wal` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)"
   end
 
   private
 
+  # Options after `--` come as `-flag`, `-flag=value` or `-flag value`. Rake
+  # also reads a bare word there as one more task to run, so each value gets
+  # an empty task of its own to keep Rake from aborting once this one is done.
   def parse_argv_options
     options = {}
     if (separator_index = ARGV.index("--"))
-      ARGV.slice(separator_index + 1, ARGV.length)
-        .map { |pair| pair.split("=") }
-        .each { |opt| options[opt[0]] = opt[1] || nil }
+      awaiting_value = nil
+      ARGV.drop(separator_index + 1).each do |arg|
+        if arg.start_with?("-")
+          key, value = arg.split("=", 2)
+          options[key] = value
+          awaiting_value = value.nil? ? key : nil
+        elsif awaiting_value
+          options[awaiting_value] = arg
+          awaiting_value = nil
+          Rake::Task.define_task(arg)
+        end
+      end
     end
     options.symbolize_keys!
   end

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: litestream
 version: !ruby/object:Gem::Version
-  version: 0.14.0
+  version: 0.16.0
 platform: x86_64-linux
 authors:
 - Stephen Margheim
 bindir: exe
 cert_chain: []
-date: 2025-06-13 00:00:00.000000000 Z
+date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: sqlite3
@@ -23,6 +23,7 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+  force_ruby_platform: false
 - !ruby/object:Gem::Dependency
   name: actionpack
   requirement: !ruby/object:Gem::Requirement
@@ -37,6 +38,7 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '7.0'
+  force_ruby_platform: false
 - !ruby/object:Gem::Dependency
   name: actionview
   requirement: !ruby/object:Gem::Requirement
@@ -51,6 +53,7 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '7.0'
+  force_ruby_platform: false
 - !ruby/object:Gem::Dependency
   name: activejob
   requirement: !ruby/object:Gem::Requirement
@@ -65,6 +68,7 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '7.0'
+  force_ruby_platform: false
 - !ruby/object:Gem::Dependency
   name: activesupport
   requirement: !ruby/object:Gem::Requirement
@@ -79,6 +83,7 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '7.0'
+  force_ruby_platform: false
 - !ruby/object:Gem::Dependency
   name: railties
   requirement: !ruby/object:Gem::Requirement
@@ -93,6 +98,7 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '7.0'
+  force_ruby_platform: false
 - !ruby/object:Gem::Dependency
   name: rails
   requirement: !ruby/object:Gem::Requirement
@@ -107,20 +113,6 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-- !ruby/object:Gem::Dependency
-  name: rubyzip
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
 email:
 - stephen.margheim@gmail.com
 executables:
@@ -142,6 +134,7 @@ files:
 - config/routes.rb
 - exe/litestream
 - exe/x86_64-linux/litestream
+- exe/x86_64-linux/litestream.sha256
 - lib/litestream.rb
 - lib/litestream/commands.rb
 - lib/litestream/engine.rb
@@ -174,7 +167,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.6.3
+rubygems_version: 3.6.7
 specification_version: 4
 summary: Integrate Litestream with the RubyGems infrastructure.
 test_files: []