ruborg 0.9.0 → 0.9.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a4d69dff5edaf281b1014e64653462df7d7af84be0f341db0c4fe389978f25b1
-  data.tar.gz: b206422e04dab022cd19a8d4ea6f9bd399988fc27b6e810fd673dfb7de0fb9ba
+  metadata.gz: b10d4c2ab697830a9ac1b3e4ce1e13de22728577b2c5f7326465dd9ac877d688
+  data.tar.gz: 488993366f00681563d850468c28e50bbdeffae07aa2da7e1925451d813ab135
 SHA512:
-  metadata.gz: f881b5b0908afaa16729339263d1584d861cd3a3d6b613599cbdb120340a86a2dc69408dc3f630d347769b990ea6c36ced1b538d17ba1517bebfb347c5c9ef2b
-  data.tar.gz: ffd3ee5bd76b08ac9753d66ae95ac1aebaed2f4ba6db38ddc720e013970d2799a99202e4cf7295f969dd93ddf4f8b8b7df6d8730352a33f8ff86ce76eecbaff6
+  metadata.gz: 33792f380eb30ecbeb8d1036d807b11077fd3ddd55ab22abedcd4e7b7ee8d8aaffe3552e796ca816f396fc947e6c338a3f68a9d2330db041c1608cf2d1ac020a
+  data.tar.gz: 23cdd2f090b1a7c860a1568edc266088c59c39ecc54424523fbad3f8e260cb54da33e6400d645ea4df5d22906414a72cc6c7f02b5fae07d3906a8f914e9a10a2

data/CHANGELOG.md

@@ -7,6 +7,75 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.9.4] - 2026-05-09
+
+### Fixed
+- **`ruborg lock` now detects Borg cache locks** — previously, a stale cache lock (`~/.cache/borg/<id>/lock.exclusive`) was invisible to `ruborg lock`, which only checked repository-level lock files. The command now reports both the repository lock and the cache lock state independently, so a cache-only stale lock no longer produces a misleading "No lock found" result.
+  - `Repository#cache_locked?` — pure filesystem check on the cache lock; reads the repo ID from `<repo>/config` without invoking Borg or requiring a passphrase
+  - `Repository#force_break_lock` — now also removes the cache lock and includes `"cache:lock.exclusive"` in its returned list when removed
+  - `borg break-lock` (used by `--break`) already handles both locks atomically — no change needed there
+  - Respects `BORG_CACHE_DIR` when set
+  - Fixes [#10](https://github.com/mpantel/ruborg/issues/10)
+- **Per-file backup: O(N) `borg info` calls replaced with a single `borg list --format` pass** — previously, building the archive inventory called `borg info` once per archive not yet in the `ArchiveCache`. On large repositories this produced hundreds of sequential Borg subprocesses per run, each briefly acquiring the Borg cache lock, causing cumulative lock timeout errors. The inventory now uses a single `borg list --format '{name} {comment}\n'` call which retrieves name and metadata comment for all archives at once, warming the cache transparently in the same pass (combined warm-and-continue). Subsequent runs hit the cache with zero additional Borg calls.
+  - Fixes [#12](https://github.com/mpantel/ruborg/issues/12)
+- **Progress display improvements** — backup runs now give richer real-time feedback (fixes [#14](https://github.com/mpantel/ruborg/issues/14)):
+  - **Elapsed time in all spinners**: any step that takes more than 3 seconds shows a live `(Ns)` counter so long-running stages are visually distinguishable from fast ones
+  - **Stage 1 is always animated**: the "Verifying repository" stage now spins immediately (passphrase fetch, auto-init, lock-wait) rather than flashing and disappearing
+  - **Standard backup file count**: `borg create` runs with `--list --filter AM`; the spinner updates in real time with the count of new/changed files being archived, and the completion line reports the final count (e.g. `✓ Archive created: my-repo-… — 42 new/changed file(s)`)
+  - **`Progress#update_spin`**: new method for updating a spinner's label mid-operation without restarting it
+
+## [0.9.3] - 2026-05-09
+
+### Added
+- **`ruborg lock` command**: Check for and optionally break stale Borg repository locks
+  - `ruborg lock --repository NAME` — exits 0 if no lock, exits 1 if lock detected
+  - `ruborg lock --repository NAME --break --yes` — breaks the lock via `borg break-lock`
+  - `ruborg lock --repository NAME --force --yes` — force-removes lock files directly without invoking Borg (useful when Borg itself can't run)
+  - `--break` and `--force` are mutually exclusive; both require `--yes` as a safety guard
+  - `Repository#locked?` — pure filesystem check on `lock.exclusive` / `lock.roster`, no Borg invocation or passphrase required
+  - `Repository#break_lock` — delegates to `borg break-lock`; requires Borg >= 1.4.0
+  - `Repository#force_break_lock` — direct filesystem removal of lock files/dirs; no Borg needed
+  - Status output (lock present/absent) goes to stdout for scriptability; warning messages go to `$stderr`
+- **Pre-flight lock detection during backup**: If a repository is locked when `backup` starts, ruborg waits and retries instead of failing immediately
+  - Polls every 5 seconds, prints elapsed time via the spinner
+  - Aborts with a clear error message after `lock_wait` seconds (default 300), suggesting `ruborg lock` to inspect or clear
+- **`lock_wait` config key**: Optional integer (seconds). When set, also passed as `--lock-wait` to all Borg commands so Borg itself waits for mid-operation locks. Omitting the key leaves Borg at its own default (1 second)
+- **Minimum Borg version**: Raised to 1.4.0; `break_lock` verifies this before invoking `borg break-lock`
+- **`CLI::DEFAULT_LOCK_WAIT = 300`**: Named constant for the pre-flight wait timeout
+- Fixes [#8](https://github.com/mpantel/ruborg/issues/8)
+
+## [0.9.2] - 2026-05-09
+
+### Added
+- **CLI progress display**: Real-time feedback during backup operations
+  - Named stages printed to `$stderr`: `[1/3] Verifying repository`, `[2/3] Backing up files`, `[3/3] Pruning`
+  - Animated spinner for indeterminate operations (cache loading, `borg create`, pruning)
+  - Inline progress bar for per-file backup mode: `[=========>     ] 42/120  filename.jpg`
+  - Stage count adapts to the operation: 2 stages without pruning, 3 with
+  - Degrades gracefully to plain text lines when output is piped or redirected (non-TTY)
+  - All progress output goes to `$stderr` — `--json` stdout and piped output remain clean
+  - No external gem dependencies — pure Ruby with ANSI `\r` rewrite
+  - Fixes [#6](https://github.com/mpantel/ruborg/issues/6)
+
+## [0.9.1] - 2026-05-09
+
+### Added
+- **Archive metadata cache**: New `ArchiveCache` class eliminates N+1 `borg info` calls during per-file backup runs
+  - Metadata cached in `<repo_path>.ruborg_cache.json` — sibling file to the repository
+  - Cache is shared across machines: any host with access to the repo path shares the same cache
+  - Local repos use `File::LOCK_EX` for safe concurrent reads/writes with merge-on-conflict
+  - SSH repos (`user@host:/path`, `ssh://user@host/path`) fetch/push the cache via `scp` with optimistic locking (fetch-fresh → merge → push), avoiding deadlocks on process crashes
+  - Only archives not yet in cache trigger a `borg info` call; warm runs reduce subprocess overhead from O(n) to O(new archives)
+  - Fixes [#4](https://github.com/mpantel/ruborg/issues/4)
+- **Catalog command**: New `ruborg catalog` command for fast, offline browsing of backed-up files
+  - Reads the local cache file — no `borg` subprocess calls needed
+  - `--search PATTERN` — filter entries by file path using a regex
+  - `--stats` — show aggregate statistics (total archives, unique files, total size, source dirs)
+  - `--json` — machine-readable JSON output; default is a human-friendly text table
+  - Works per-repository like all other commands (`--repository`)
+  - Supports SSH repos transparently via the same `scp`-based cache fetch
+- **Bug fix**: `ArchiveCache` now normalises all loaded metadata to symbol keys, ensuring cache hits and cache misses return identical key types
+
 ## [0.9.0] - 2025-10-14
 
 ### Changed

data/README.md

@@ -25,13 +25,15 @@ A friendly Ruby frontend for [Borg Backup](https://www.borgbackup.org/). Ruborg
 - 📈 **Summary View** - Quick overview of all repositories and their configurations
 - 🔧 **Custom Borg Path** - Support for custom Borg executable paths per repository
 - 🏠 **Hostname Validation** - NEW! Restrict backups to specific hosts (global or per-repository)
-- ✅ **Well-tested** - Comprehensive test suite with RSpec (297 examples, 0 failures)
+- 🔒 **Lock Management** - Detect and break stale Borg repository locks with `ruborg lock`
+- ⏳ **Lock-aware Backups** - Pre-flight lock detection with configurable wait timeout before backup
+- ✅ **Well-tested** - Comprehensive test suite with RSpec (412 examples, 0 failures)
 - 🔒 **Security-focused** - Path validation, safe YAML loading, command injection protection
 
 ## Prerequisites
 
 - Ruby >= 3.2.0
-- [Borg Backup](https://www.borgbackup.org/) installed and available in PATH
+- [Borg Backup](https://www.borgbackup.org/) >= 1.4.0 installed and available in PATH
 - [Passbolt CLI](https://github.com/passbolt/go-passbolt-cli) (optional, for password management)
 
 ### Installing Borg Backup
@@ -169,8 +171,9 @@ repositories:
 - **Source Deletion Safety**: `allow_remove_source` flag to explicitly enable `--remove-source` option (default: disabled)
 - **Skip Hash Check**: Optional `skip_hash_check` flag to skip content hash verification for faster backups (per-file mode only)
 - **Type-Safe Booleans**: Strict boolean validation prevents configuration errors (must use `true`/`false`, not strings)
-- **Global Settings**: Hostname, compression, encryption, auto_init, allow_remove_source, skip_hash_check, log_file, borg_path, borg_options, and retention apply to all repositories
-- **Per-Repository Overrides**: Any global setting can be overridden at the repository level (including hostname, allow_remove_source, skip_hash_check, and custom borg_path)
+- **Lock Wait Timeout**: Optional `lock_wait` (integer, seconds) — how long ruborg waits for a locked repository before aborting. Also passed as `--lock-wait` to Borg when set. Default: 300 seconds (pre-flight), Borg default: 1 second (when not configured)
+- **Global Settings**: Hostname, compression, encryption, auto_init, allow_remove_source, skip_hash_check, lock_wait, log_file, borg_path, borg_options, and retention apply to all repositories
+- **Per-Repository Overrides**: Any global setting can be overridden at the repository level (including hostname, allow_remove_source, skip_hash_check, lock_wait, and custom borg_path)
 - **Custom Borg Path**: Specify a custom Borg executable path if borg is not in PATH or to use a specific version
 - **Retention Policies**: Define how many backups to keep (hourly, daily, weekly, monthly, yearly)
 - **Multiple Sources**: Each repository can have multiple backup sources with their own exclude patterns
@@ -472,6 +475,47 @@ Group: postgres
 Type: regular file
 ```
 
+### Manage Repository Locks
+
+Borg uses lock files to prevent concurrent access. If a backup crashes, stale locks can block all subsequent operations. Use `ruborg lock` to inspect and clear them.
+
+Borg maintains **two independent locks** — one on the repository itself and one on the local cache (`~/.cache/borg/<id>/lock.exclusive`). `ruborg lock` checks and reports both:
+
+```
+Lock detected on repository 'documents' (/mnt/backups/documents):
+  Repository lock : clear
+  Cache lock      : LOCKED
+  Run with --break --yes (via borg) or --force --yes (direct removal).
+```
+
+```bash
+# Check both repository and cache locks (exits 0 = no lock, 1 = locked)
+ruborg lock --repository documents
+
+# Break both locks via borg break-lock (requires Borg >= 1.4.0)
+ruborg lock --repository documents --break --yes
+
+# Force-remove lock files directly (no Borg required, last resort)
+ruborg lock --repository documents --force --yes
+```
+
+**Lock-aware backups:** When `ruborg backup` starts and detects a lock, it waits up to `lock_wait` seconds (default 300) for the lock to clear before aborting:
+
+```
+[1/2] Verifying repository: documents
+  Repository locked — waiting for lock to clear (5s / 300s)…
+  Repository locked — waiting for lock to clear (10s / 300s)…
+  ✓ Lock cleared
+[2/2] Creating archive
+```
+
+Configure the timeout in `ruborg.yml` (also passed as `--lock-wait` to Borg when set):
+
+```yaml
+# Wait up to 60s for a lock before aborting; also passes --lock-wait 60 to borg commands
+lock_wait: 60
+```
+
 ### Validate Repository Compatibility
 
 ```bash
@@ -659,6 +703,7 @@ See [SECURITY.md](SECURITY.md) for detailed security information and best practi
 | `list` | List archives or files in repository | `--config`, `--repository`, `--archive`, `--log` |
 | `restore ARCHIVE` | Restore files from archive | `--config`, `--repository`, `--destination`, `--path`, `--log` |
 | `metadata ARCHIVE` | Get file metadata from archive | `--config`, `--repository`, `--file`, `--log` |
+| `lock` | Check for and optionally break a repository lock | `--config`, `--repository`, `--break`, `--force`, `--yes`, `--log` |
 | `info` | Show repository information | `--config`, `--repository`, `--log` |
 | `version` | Show ruborg version | None |
 
@@ -821,6 +866,13 @@ repositories:
 
 **Performance Note:** Per-file mode creates many archives (one per file). Borg handles this efficiently due to deduplication, but it's best suited for directories with hundreds to thousands of files rather than millions.
 
+**Progress feedback:** During backup, ruborg shows real-time progress on stderr:
+- All spinner stages display an elapsed-time counter after 3 seconds (`Preparing... (12s)`) so long-running steps are always visible
+- Standard backup mode (`borg create`) streams file-level output and shows a running count of new/changed files in the spinner, with a final summary in the completion line (e.g. `✓ Archive created: my-repo-… — 42 new/changed file(s)`)
+- Per-file mode shows a progress bar with current/total file count
+
+**Inventory performance:** At the start of each per-file backup run, ruborg builds an inventory of existing archives using a single `borg list` call. Metadata is cached locally in a `.ruborg_cache.json` file beside the repository — subsequent runs serve the inventory entirely from cache with no additional Borg calls. The first run after introducing a large existing repository will be slower as it warms the cache, but all subsequent runs are fast regardless of how many archives exist.
+
 **Backup vs Retention:** The per-file `retention_mode` only affects how archives are created and pruned. Traditional backup commands still work normally - you can list, restore, and check per-file archives just like standard archives.
 
 ### Skip Hash Check for Faster Backups

data/lib/ruborg/archive_cache.rb

@@ -0,0 +1,189 @@
+# frozen_string_literal: true
+
+require "json"
+require "open3"
+require "tempfile"
+
+module Ruborg
+  # Persistent cache of per-archive metadata, stored as a JSON file sibling to the
+  # Borg repository. Eliminates repeated `borg info` calls across runs.
+  #
+  # Supports local paths (File::LOCK_EX) and SSH paths (optimistic merge via scp).
+  # All metadata is stored and returned with symbol keys (:path, :size, :hash, :source_dir).
+  class ArchiveCache
+    SSH_PATTERN = %r{\A(?:ssh://|[^\s/]+@[^\s:]+:)}
+
+    def initialize(repo_path)
+      @repo_path = repo_path
+      @data = {}
+      @snapshot = {}
+      @loaded = false
+    end
+
+    def fetch
+      return self if @loaded
+
+      if ssh?
+        load_remote
+      else
+        load_local
+      end
+
+      @snapshot = snapshot(@data)
+      @loaded = true
+      self
+    end
+
+    def [](archive_name)
+      @data[archive_name]
+    end
+
+    def store(archive_name, metadata)
+      @data[archive_name] = symbolize(metadata)
+    end
+
+    # Returns all cached entries as an array of hashes, each including :archive_name.
+    def entries
+      @data.map { |archive_name, metadata| metadata.merge(archive_name: archive_name) }
+    end
+
+    def save_if_changed
+      return unless dirty?
+
+      if ssh?
+        save_remote
+      else
+        save_local
+      end
+    end
+
+    private
+
+    def dirty?
+      @data != @snapshot
+    end
+
+    def snapshot(hash)
+      hash.transform_values(&:dup)
+    end
+
+    def ssh?
+      SSH_PATTERN.match?(@repo_path)
+    end
+
+    def cache_path_for(path)
+      "#{path}.ruborg_cache.json"
+    end
+
+    def symbolize(metadata)
+      metadata.transform_keys(&:to_sym)
+    end
+
+    def normalize_archives(raw)
+      (raw || {}).transform_values { |v| symbolize(v) }
+    end
+
+    def load_local
+      path = cache_path_for(@repo_path)
+      return unless File.exist?(path)
+
+      File.open(path, "r") do |f|
+        f.flock(File::LOCK_SH)
+        parsed = JSON.parse(f.read)
+        @data = normalize_archives(parsed["archives"])
+      end
+    rescue JSON::ParserError
+      @data = {}
+    end
+
+    def save_local
+      path = cache_path_for(@repo_path)
+      File.open(path, File::RDWR | File::CREAT, 0o600) do |f|
+        f.flock(File::LOCK_EX)
+        existing = read_existing_local(f)
+        merged = existing.merge(@data)
+        f.rewind
+        f.write(JSON.generate({ "version" => 1, "archives" => stringify_for_storage(merged) }))
+        f.truncate(f.pos)
+      end
+    end
+
+    def read_existing_local(file)
+      content = file.read
+      return {} if content.empty?
+
+      normalize_archives(JSON.parse(content)["archives"])
+    rescue JSON::ParserError
+      {}
+    end
+
+    # JSON requires string keys; convert symbol keys back before writing.
+    def stringify_for_storage(data)
+      data.transform_values { |v| v.transform_keys(&:to_s) }
+    end
+
+    def parse_ssh
+      if @repo_path.start_with?("ssh://")
+        require "uri"
+        uri = URI.parse(@repo_path)
+        host = uri.user ? "#{uri.user}@#{uri.host}" : uri.host
+        host = "#{host}:#{uri.port}" if uri.port && uri.port != 22
+        [host, uri.path]
+      else
+        match = @repo_path.match(%r{\A([^\s/]+@[^\s:]+):(.+)\z})
+        return [nil, nil] unless match
+
+        [match[1], match[2]]
+      end
+    end
+
+    def load_remote
+      host, path = parse_ssh
+      return unless host
+
+      remote = "#{host}:#{cache_path_for(path)}"
+      loaded = nil
+      Tempfile.create(["ruborg_cache", ".json"]) do |tmp|
+        _, status = Open3.capture2e("scp", "-q", "-B", remote, tmp.path)
+        next unless status.success?
+
+        begin
+          loaded = normalize_archives(JSON.parse(File.read(tmp.path))["archives"])
+        rescue JSON::ParserError
+          loaded = {}
+        end
+      end
+      @data = loaded if loaded
+    end
+
+    def save_remote
+      host, path = parse_ssh
+      return unless host
+
+      remote = "#{host}:#{cache_path_for(path)}"
+      fresh = fetch_remote_fresh(remote)
+      merged = fresh.merge(@data)
+
+      Tempfile.create(["ruborg_cache_upload", ".json"]) do |tmp|
+        tmp.write(JSON.generate({ "version" => 1, "archives" => stringify_for_storage(merged) }))
+        tmp.flush
+        Open3.capture2e("scp", "-q", "-B", tmp.path, remote)
+      end
+    end
+
+    def fetch_remote_fresh(remote)
+      result = {}
+      Tempfile.create(["ruborg_cache_fresh", ".json"]) do |tmp|
+        _, status = Open3.capture2e("scp", "-q", "-B", remote, tmp.path)
+        next unless status.success?
+
+        begin
+          result = normalize_archives(JSON.parse(File.read(tmp.path))["archives"])
+        rescue JSON::ParserError
+          result = {}
+        end
+      end
+      result
+    end
+  end
+end

data/lib/ruborg/backup.rb

@@ -3,13 +3,15 @@
 module Ruborg
   # Backup operations using Borg
   class Backup
-    def initialize(repository, config:, retention_mode: "standard", repo_name: nil, logger: nil, skip_hash_check: false)
+    def initialize(repository, config:, retention_mode: "standard", repo_name: nil, logger: nil,
+                   skip_hash_check: false, progress: nil)
       @repository = repository
       @config = config
       @retention_mode = retention_mode
       @repo_name = repo_name
       @logger = logger
       @skip_hash_check = skip_hash_check
+      @progress = progress
     end
 
     def create(name: nil, remove_source: false)
@@ -27,36 +29,36 @@ module Ruborg
     def create_standard_archive(name, remove_source)
       archive_name = name || Time.now.strftime("%Y-%m-%d_%H-%M-%S")
 
-      # Show repository header in console only
       print_repository_header
+      @progress&.spin("Creating archive: #{archive_name}")
 
-      # Show progress in console
-      puts "Creating archive: #{archive_name}"
-
-      cmd = build_create_command(archive_name)
-
-      execute_borg_command(cmd)
+      count = if @progress
+                stream_borg_create(archive_name)
+              else
+                execute_borg_command(build_create_command(archive_name))
+                0
+              end
 
-      # Log successful action
+      summary = count.positive? ? " — #{count} new/changed file(s)" : ""
+      @progress&.done("Archive created: #{archive_name}#{summary}")
       @logger&.info("[#{@repo_name}] Created archive #{archive_name} with #{@config.backup_paths.size} source(s)")
-      puts "✓ Archive created successfully"
 
       remove_source_files if remove_source
     end
 
     # rubocop:disable Metrics/AbcSize, Metrics/MethodLength, Metrics/PerceivedComplexity, Metrics/BlockNesting
     def create_per_file_archives(name_prefix, remove_source)
-      # Collect all files from backup paths
+      @progress&.spin("Collecting files...")
       files_to_backup = collect_files_from_paths(@config.backup_paths, @config.exclude_patterns)
+      @progress&.stop_spin
 
       raise BorgError, "No files found to backup" if files_to_backup.empty?
 
-      # Get list of existing archives for duplicate detection
+      @progress&.spin("Loading archive catalog...")
       existing_archives = get_existing_archive_names
+      @progress&.done("Catalog loaded — #{existing_archives.size} archive(s) known")
 
-      # Show repository header in console only
       print_repository_header
-
       puts "Found #{files_to_backup.size} file(s) to backup"
 
       backed_up_count = 0
@@ -78,8 +80,8 @@ module Ruborg
         # Ensure archive name doesn't exceed 255 characters (filesystem limit)
         archive_name = name_prefix || build_archive_name(@repo_name, sanitized_filename, path_hash, file_mtime)
 
-        # Show progress in console
-        print "  [#{index + 1}/#{files_to_backup.size}] Backing up: #{file_path}"
+        @progress&.bar(index + 1, files_to_backup.size, File.basename(file_path))
+        $stderr.print "  [#{index + 1}/#{files_to_backup.size}] Backing up: #{file_path}" unless @progress
 
         # Check if archive already exists AND contains this exact file
         if existing_archives.key?(archive_name)
@@ -149,7 +151,7 @@ module Ruborg
         cmd = build_per_file_create_command(archive_name, file_path, source_dir)
 
         execute_borg_command(cmd)
-        puts ""
+        puts "" unless @progress
 
         # Log successful action with details
         @logger&.info("[#{@repo_name}] Archived #{file_path} in archive #{archive_name}")
@@ -160,11 +162,13 @@ module Ruborg
       end
       # rubocop:enable Metrics/BlockLength
 
-      if skipped_count.positive?
-        puts "✓ Per-file backup completed: #{backed_up_count} file(s) backed up, #{skipped_count} skipped (unchanged)"
-      else
-        puts "✓ Per-file backup completed: #{backed_up_count} file(s) backed up"
-      end
+      summary = if skipped_count.positive?
+                  "#{backed_up_count} file(s) backed up, #{skipped_count} skipped (unchanged)"
+                else
+                  "#{backed_up_count} file(s) backed up"
+                end
+      @progress&.done(summary)
+      puts "✓ Per-file backup completed: #{summary}" unless @progress
     end
     # rubocop:enable Metrics/AbcSize, Metrics/MethodLength, Metrics/PerceivedComplexity, Metrics/BlockNesting
 
@@ -343,19 +347,43 @@ module Ruborg
       cmd
     end
 
-    def execute_borg_command(cmd)
+    def borg_env
       env = {}
       passphrase = @repository.instance_variable_get(:@passphrase)
       env["BORG_PASSPHRASE"] = passphrase if passphrase
       env["BORG_RELOCATED_REPO_ACCESS_IS_OK"] = "yes"
       env["BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK"] = "yes"
+      env
+    end
 
-      result = system(env, *cmd, in: "/dev/null")
+    def execute_borg_command(cmd)
+      result = system(borg_env, *cmd, in: "/dev/null")
       raise BorgError, "Borg command failed: #{cmd.join(" ")}" unless result
 
       result
     end
 
+    # Run borg create with --list --filter AM, streaming stderr to count
+    # new/changed files and update the spinner label in real time.
+    def stream_borg_create(archive_name)
+      require "open3"
+
+      cmd = build_create_command(archive_name) + ["--list", "--filter", "AM"]
+      file_count = 0
+
+      Open3.popen3(borg_env, *cmd, in: "/dev/null") do |_sin, _sout, stderr, wait_thr|
+        stderr.each_line do |line|
+          next unless line.match?(/\A[AM] /)
+
+          file_count += 1
+          @progress.update_spin("Creating archive: #{archive_name} — #{file_count} new/changed file(s)")
+        end
+        raise BorgError, "Borg command failed: #{cmd.join(" ")}" unless wait_thr.value.success?
+      end
+
+      file_count
+    end
+
     def remove_single_file(file_path)
       require "fileutils"
 
@@ -459,89 +487,57 @@ module Ruborg
       puts "=" * 60
     end
 
-    # rubocop:disable Metrics/AbcSize, Metrics/MethodLength, Metrics/PerceivedComplexity
     def get_existing_archive_names
-      require "json"
       require "open3"
 
-      # First get list of archives
-      cmd = [@repository.borg_path, "list", @repository.path, "--json"]
       env = {}
       passphrase = @repository.instance_variable_get(:@passphrase)
       env["BORG_PASSPHRASE"] = passphrase if passphrase
       env["BORG_RELOCATED_REPO_ACCESS_IS_OK"] = "yes"
       env["BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK"] = "yes"
 
+      # Single borg call: fetch name + comment for every archive at once.
+      # Replaces the previous O(N) borg info loop and warms the cache in the same pass.
+      cmd = [@repository.borg_path, "list", @repository.path, "--format", "{name} {comment}\n"]
       stdout, stderr, status = Open3.capture3(env, *cmd)
       raise BorgError, "Failed to list archives: #{stderr}" unless status.success?
 
-      json_data = JSON.parse(stdout)
-      archives = json_data["archives"] || []
+      cache = ArchiveCache.new(@repository.path).fetch
+      result = {}
 
-      # Build hash by querying each archive individually for comment
-      # This is necessary because 'borg list' doesn't include comments
-      archives.each_with_object({}) do |archive, hash|
-        archive_name = archive["name"]
+      stdout.each_line do |line|
+        archive_name, comment = line.chomp.split(" ", 2)
+        next unless archive_name
 
-        # Query this specific archive to get the comment
-        info_cmd = [@repository.borg_path, "info", "#{@repository.path}::#{archive_name}", "--json"]
-        info_stdout, _, info_status = Open3.capture3(env, *info_cmd)
-
-        unless info_status.success?
-          # If we can't get info for this archive, skip it with defaults
-          hash[archive_name] = { path: "", size: 0, hash: "", source_dir: "" }
+        if (cached = cache[archive_name])
+          result[archive_name] = cached
           next
         end
 
-        info_data = JSON.parse(info_stdout)
-        archive_info = info_data["archives"]&.first || {}
-        comment = archive_info["comment"] || ""
-
-        # Parse comment based on format
-        # The comment field stores metadata as: path|||size|||hash|||source_dir (using ||| as delimiter)
-        # For backward compatibility, handle old formats:
-        #   - Old format 1: plain path (no |||)
-        #   - Old format 2: path|||hash (2 parts)
-        #   - Old format 3: path|||size|||hash (3 parts)
-        #   - New format: path|||size|||hash|||source_dir (4 parts)
-        if comment.include?("|||")
-          parts = comment.split("|||")
-          file_path = parts[0]
-          if parts.length >= 4
-            # New format: path|||size|||hash|||source_dir
-            file_size = parts[1].to_i
-            file_hash = parts[2] || ""
-            source_dir = parts[3] || ""
-          elsif parts.length >= 3
-            # Format 3: path|||size|||hash (no source_dir)
-            file_size = parts[1].to_i
-            file_hash = parts[2] || ""
-            source_dir = ""
-          else
-            # Old format: path|||hash (size and source_dir not available)
-            file_size = 0
-            file_hash = parts[1] || ""
-            source_dir = ""
-          end
+        metadata = parse_archive_comment(comment || "")
+        cache.store(archive_name, metadata)
+        result[archive_name] = metadata
+      end
+
+      cache.save_if_changed
+      result
+    end
+
+    def parse_archive_comment(comment)
+      if comment.include?("|||")
+        parts = comment.split("|||")
+        file_path = parts[0]
+        if parts.length >= 4
+          { path: file_path, size: parts[1].to_i, hash: parts[2] || "", source_dir: parts[3] || "" }
+        elsif parts.length >= 3
+          { path: file_path, size: parts[1].to_i, hash: parts[2] || "", source_dir: "" }
         else
-          # Oldest format: comment is just the path string
-          file_path = comment
-          file_size = 0
-          file_hash = ""
-          source_dir = ""
+          { path: file_path, size: 0, hash: parts[1] || "", source_dir: "" }
         end
-
-        hash[archive_name] = {
-          path: file_path,
-          size: file_size,
-          hash: file_hash,
-          source_dir: source_dir
-        }
+      else
+        { path: comment, size: 0, hash: "", source_dir: "" }
       end
-    rescue JSON::ParserError => e
-      raise BorgError, "Failed to parse archive info: #{e.message}"
     end
-    # rubocop:enable Metrics/AbcSize, Metrics/MethodLength, Metrics/PerceivedComplexity
 
     def find_next_version_name(base_name, existing_archives)
       version = 2