ruborg 0.9.3 → 0.9.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8f3c9f239a72ec2f321419eb6f4349c25fe4cec4ca5174c156e2f1d3fe84ec68
-  data.tar.gz: 4930dbbc8dc2d4e2040628a8c57e62d7bf3ac20ca58a0e7e363a649f6f8d6546
+  metadata.gz: b10d4c2ab697830a9ac1b3e4ce1e13de22728577b2c5f7326465dd9ac877d688
+  data.tar.gz: 488993366f00681563d850468c28e50bbdeffae07aa2da7e1925451d813ab135
 SHA512:
-  metadata.gz: ba9bd867f83d24c88abf435f4b64368b9c414080290747f107a6b5e94db6e82ddba0a59b0f700a4625e689b54724aefceecccc56f5e51f3fc8271be74601bc5e
-  data.tar.gz: e95dd232d73c7c63968c0105170928d0ab28d689342ad875a81fbc30ac56d508179a1866b476b79125fe43a85fc69cc68baef14a45cd220019be5612ce458bea
+  metadata.gz: 33792f380eb30ecbeb8d1036d807b11077fd3ddd55ab22abedcd4e7b7ee8d8aaffe3552e796ca816f396fc947e6c338a3f68a9d2330db041c1608cf2d1ac020a
+  data.tar.gz: 23cdd2f090b1a7c860a1568edc266088c59c39ecc54424523fbad3f8e260cb54da33e6400d645ea4df5d22906414a72cc6c7f02b5fae07d3906a8f914e9a10a2

data/CHANGELOG.md

@@ -7,6 +7,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.9.4] - 2026-05-09
+
+### Fixed
+- **`ruborg lock` now detects Borg cache locks** — previously, a stale cache lock (`~/.cache/borg/<id>/lock.exclusive`) was invisible to `ruborg lock`, which only checked repository-level lock files. The command now reports both the repository lock and the cache lock state independently, so a cache-only stale lock no longer produces a misleading "No lock found" result.
+  - `Repository#cache_locked?` — pure filesystem check on the cache lock; reads the repo ID from `<repo>/config` without invoking Borg or requiring a passphrase
+  - `Repository#force_break_lock` — now also removes the cache lock and includes `"cache:lock.exclusive"` in its returned list when removed
+  - `borg break-lock` (used by `--break`) already handles both locks atomically — no change needed there
+  - Respects `BORG_CACHE_DIR` when set
+  - Fixes [#10](https://github.com/mpantel/ruborg/issues/10)
+- **Per-file backup: O(N) `borg info` calls replaced with a single `borg list --format` pass** — previously, building the archive inventory called `borg info` once per archive not yet in the `ArchiveCache`. On large repositories this produced hundreds of sequential Borg subprocesses per run, each briefly acquiring the Borg cache lock, causing cumulative lock timeout errors. The inventory now uses a single `borg list --format '{name} {comment}\n'` call which retrieves name and metadata comment for all archives at once, warming the cache transparently in the same pass (combined warm-and-continue). Subsequent runs hit the cache with zero additional Borg calls.
+  - Fixes [#12](https://github.com/mpantel/ruborg/issues/12)
+- **Progress display improvements** — backup runs now give richer real-time feedback (fixes [#14](https://github.com/mpantel/ruborg/issues/14)):
+  - **Elapsed time in all spinners**: any step that takes more than 3 seconds shows a live `(Ns)` counter so long-running stages are visually distinguishable from fast ones
+  - **Stage 1 is always animated**: the "Verifying repository" stage now spins immediately (passphrase fetch, auto-init, lock-wait) rather than flashing and disappearing
+  - **Standard backup file count**: `borg create` runs with `--list --filter AM`; the spinner updates in real time with the count of new/changed files being archived, and the completion line reports the final count (e.g. `✓ Archive created: my-repo-… — 42 new/changed file(s)`)
+  - **`Progress#update_spin`**: new method for updating a spinner's label mid-operation without restarting it
+
 ## [0.9.3] - 2026-05-09
 
 ### Added

data/README.md

@@ -479,11 +479,20 @@ Type: regular file
 
 Borg uses lock files to prevent concurrent access. If a backup crashes, stale locks can block all subsequent operations. Use `ruborg lock` to inspect and clear them.
 
+Borg maintains **two independent locks** — one on the repository itself and one on the local cache (`~/.cache/borg/<id>/lock.exclusive`). `ruborg lock` checks and reports both:
+
+```
+Lock detected on repository 'documents' (/mnt/backups/documents):
+  Repository lock : clear
+  Cache lock      : LOCKED
+  Run with --break --yes (via borg) or --force --yes (direct removal).
+```
+
 ```bash
-# Check if a repository is locked (exits 0 = no lock, 1 = locked)
+# Check both repository and cache locks (exits 0 = no lock, 1 = locked)
 ruborg lock --repository documents
 
-# Break the lock via borg break-lock (requires Borg >= 1.4.0)
+# Break both locks via borg break-lock (requires Borg >= 1.4.0)
 ruborg lock --repository documents --break --yes
 
 # Force-remove lock files directly (no Borg required, last resort)
@@ -857,6 +866,13 @@ repositories:
 
 **Performance Note:** Per-file mode creates many archives (one per file). Borg handles this efficiently due to deduplication, but it's best suited for directories with hundreds to thousands of files rather than millions.
 
+**Progress feedback:** During backup, ruborg shows real-time progress on stderr:
+- All spinner stages display an elapsed-time counter after 3 seconds (`Preparing... (12s)`) so long-running steps are always visible
+- Standard backup mode (`borg create`) streams file-level output and shows a running count of new/changed files in the spinner, with a final summary in the completion line (e.g. `✓ Archive created: my-repo-… — 42 new/changed file(s)`)
+- Per-file mode shows a progress bar with current/total file count
+
+**Inventory performance:** At the start of each per-file backup run, ruborg builds an inventory of existing archives using a single `borg list` call. Metadata is cached locally in a `.ruborg_cache.json` file beside the repository — subsequent runs serve the inventory entirely from cache with no additional Borg calls. The first run after introducing a large existing repository will be slower as it warms the cache, but all subsequent runs are fast regardless of how many archives exist.
+
 **Backup vs Retention:** The per-file `retention_mode` only affects how archives are created and pruned. Traditional backup commands still work normally - you can list, restore, and check per-file archives just like standard archives.
 
 ### Skip Hash Check for Faster Backups

data/lib/ruborg/backup.rb

@@ -32,10 +32,15 @@ module Ruborg
       print_repository_header
       @progress&.spin("Creating archive: #{archive_name}")
 
-      cmd = build_create_command(archive_name)
-      execute_borg_command(cmd)
+      count = if @progress
+                stream_borg_create(archive_name)
+              else
+                execute_borg_command(build_create_command(archive_name))
+                0
+              end
 
-      @progress&.done("Archive created: #{archive_name}")
+      summary = count.positive? ? " — #{count} new/changed file(s)" : ""
+      @progress&.done("Archive created: #{archive_name}#{summary}")
       @logger&.info("[#{@repo_name}] Created archive #{archive_name} with #{@config.backup_paths.size} source(s)")
 
       remove_source_files if remove_source
@@ -342,19 +347,43 @@ module Ruborg
       cmd
     end
 
-    def execute_borg_command(cmd)
+    def borg_env
       env = {}
       passphrase = @repository.instance_variable_get(:@passphrase)
       env["BORG_PASSPHRASE"] = passphrase if passphrase
       env["BORG_RELOCATED_REPO_ACCESS_IS_OK"] = "yes"
       env["BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK"] = "yes"
+      env
+    end
 
-      result = system(env, *cmd, in: "/dev/null")
+    def execute_borg_command(cmd)
+      result = system(borg_env, *cmd, in: "/dev/null")
       raise BorgError, "Borg command failed: #{cmd.join(" ")}" unless result
 
       result
     end
 
+    # Run borg create with --list --filter AM, streaming stderr to count
+    # new/changed files and update the spinner label in real time.
+    def stream_borg_create(archive_name)
+      require "open3"
+
+      cmd = build_create_command(archive_name) + ["--list", "--filter", "AM"]
+      file_count = 0
+
+      Open3.popen3(borg_env, *cmd, in: "/dev/null") do |_sin, _sout, stderr, wait_thr|
+        stderr.each_line do |line|
+          next unless line.match?(/\A[AM] /)
+
+          file_count += 1
+          @progress.update_spin("Creating archive: #{archive_name} — #{file_count} new/changed file(s)")
+        end
+        raise BorgError, "Borg command failed: #{cmd.join(" ")}" unless wait_thr.value.success?
+      end
+
+      file_count
+    end
+
     def remove_single_file(file_path)
       require "fileutils"
 
@@ -458,51 +487,41 @@ module Ruborg
       puts "=" * 60
     end
 
-    # rubocop:disable Metrics/AbcSize, Metrics/MethodLength
     def get_existing_archive_names
-      require "json"
       require "open3"
 
-      cmd = [@repository.borg_path, "list", @repository.path, "--json"]
       env = {}
       passphrase = @repository.instance_variable_get(:@passphrase)
       env["BORG_PASSPHRASE"] = passphrase if passphrase
       env["BORG_RELOCATED_REPO_ACCESS_IS_OK"] = "yes"
       env["BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK"] = "yes"
 
+      # Single borg call: fetch name + comment for every archive at once.
+      # Replaces the previous O(N) borg info loop and warms the cache in the same pass.
+      cmd = [@repository.borg_path, "list", @repository.path, "--format", "{name} {comment}\n"]
       stdout, stderr, status = Open3.capture3(env, *cmd)
       raise BorgError, "Failed to list archives: #{stderr}" unless status.success?
 
-      archives = JSON.parse(stdout)["archives"] || []
       cache = ArchiveCache.new(@repository.path).fetch
+      result = {}
 
-      result = archives.each_with_object({}) do |archive, hash|
-        archive_name = archive["name"]
+      stdout.each_line do |line|
+        archive_name, comment = line.chomp.split(" ", 2)
+        next unless archive_name
 
         if (cached = cache[archive_name])
-          hash[archive_name] = cached
+          result[archive_name] = cached
           next
         end
 
-        info_cmd = [@repository.borg_path, "info", "#{@repository.path}::#{archive_name}", "--json"]
-        info_stdout, _, info_status = Open3.capture3(env, *info_cmd)
-
-        metadata = if info_status.success?
-                     parse_archive_comment(JSON.parse(info_stdout).dig("archives", 0, "comment") || "")
-                   else
-                     { path: "", size: 0, hash: "", source_dir: "" }
-                   end
-
+        metadata = parse_archive_comment(comment || "")
         cache.store(archive_name, metadata)
-        hash[archive_name] = metadata
+        result[archive_name] = metadata
       end
 
       cache.save_if_changed
       result
-    rescue JSON::ParserError => e
-      raise BorgError, "Failed to parse archive info: #{e.message}"
     end
-    # rubocop:enable Metrics/AbcSize, Metrics/MethodLength
 
     def parse_archive_comment(comment)
       if comment.include?("|||")

data/lib/ruborg/cli.rb

@@ -395,14 +395,19 @@ module Ruborg
       passphrase = fetch_passphrase_for_repo(merged_config)
       repo = build_repo(repo_config["path"], merged_config, passphrase)
 
-      unless repo.locked?
+      repo_locked   = repo.locked?
+      cache_locked  = repo.cache_locked?
+
+      unless repo_locked || cache_locked
         puts "No lock found for repository '#{repo_config["name"]}'"
         @logger.info("Lock check: no lock found for '#{repo_config["name"]}'")
         return
       end
 
-      warn "Lock detected on repository '#{repo_config["name"]}' (#{repo_config["path"]})"
-      @logger.warn("Lock detected on repository '#{repo_config["name"]}'")
+      warn "Lock detected on repository '#{repo_config["name"]}' (#{repo_config["path"]}):"
+      warn "  Repository lock : #{repo_locked  ? "LOCKED" : "clear"}"
+      warn "  Cache lock      : #{cache_locked ? "LOCKED" : "clear"}"
+      @logger.warn("Lock detected on '#{repo_config["name"]}' — repo=#{repo_locked}, cache=#{cache_locked}")
 
       unless options[:break] || options[:force]
         warn "  Run with --break --yes (via borg) or --force --yes (direct removal)."
@@ -708,6 +713,7 @@ module Ruborg
 
       progress = Progress.new
       progress.stage(1, stage_total, "Verifying repository: #{repo_name}")
+      progress.spin("Preparing...")
 
       passphrase = fetch_passphrase_for_repo(merged_config)
       lock_wait = (merged_config["lock_wait"] || DEFAULT_LOCK_WAIT).to_i

data/lib/ruborg/progress.rb

@@ -13,6 +13,8 @@ module Ruborg
       @output = output
       @tty = output.respond_to?(:isatty) && output.isatty
       @spinner_thread = nil
+      @spin_label = nil
+      @spin_start = nil
     end
 
     # Print a numbered stage header: "[2/3] Label"
@@ -23,21 +25,32 @@ module Ruborg
     end
 
     # Start a spinner on the current line for an indeterminate operation.
+    # Appends elapsed time after 3 seconds so long-running steps are visible.
     # Call stop_spin (or done) to halt it.
     def spin(label)
       stop_spin
       return unless @tty
 
+      @spin_label = label
+      @spin_start = Time.now
       frame = 0
       @spinner_thread = Thread.new do
         loop do
-          @output.print "\r  #{SPINNER_FRAMES[frame % SPINNER_FRAMES.size]}  #{label}"
+          elapsed = (Time.now - @spin_start).to_i
+          time_str = elapsed >= 3 ? " (#{elapsed}s)" : ""
+          @output.print "\r  #{SPINNER_FRAMES[frame % SPINNER_FRAMES.size]}  #{@spin_label}#{time_str}"
           frame += 1
           sleep 0.1
         end
       end
     end
 
+    # Update the label on a running spinner without restarting it.
+    # Safe to call from the main thread while the spinner runs in the background.
+    def update_spin(label)
+      @spin_label = label
+    end
+
     # Stop the spinner and erase its line.
     def stop_spin
       return unless @spinner_thread

data/lib/ruborg/repository.rb

@@ -27,6 +27,10 @@ module Ruborg
         File.exist?(File.join(@path, "lock.roster"))
     end
 
+    def cache_locked?
+      File.exist?(cache_lock_path)
+    end
+
     def break_lock
       raise BorgError, "Repository does not exist at #{@path}" unless exists?
 
@@ -47,6 +51,13 @@ module Ruborg
         FileUtils.rm_rf(target)
         true
       end
+
+      cache_lock = cache_lock_path
+      if File.exist?(cache_lock)
+        FileUtils.rm_f(cache_lock)
+        removed << "cache:lock.exclusive"
+      end
+
       @logger&.info("Force-removed lock files at #{@path}: #{removed.join(", ")}")
       removed
     end
@@ -645,6 +656,19 @@ module Ruborg
       (actual_parts <=> minimum_parts) >= 0
     end
 
+    def cache_lock_path
+      cache_dir = ENV.fetch("BORG_CACHE_DIR", File.join(Dir.home, ".cache", "borg"))
+      repo_id = read_repo_id
+      File.join(cache_dir, repo_id, "lock.exclusive")
+    end
+
+    def read_repo_id
+      config_file = File.join(@path, "config")
+      return "" unless File.exist?(config_file)
+
+      File.read(config_file).match(/^\s*id\s*=\s*([0-9a-f]+)/)&.captures&.first || ""
+    end
+
     def find_in_path(command)
       ENV["PATH"].split(File::PATH_SEPARATOR).each do |directory|
         path = File.join(directory, command)

data/lib/ruborg/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Ruborg
-  VERSION = "0.9.3"
+  VERSION = "0.9.4"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ruborg
 version: !ruby/object:Gem::Version
-  version: 0.9.3
+  version: 0.9.4
 platform: ruby
 authors:
 - Michail Pantelelis