archive_storage 0.1.0 → 0.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 32099fdbd406f02b06d31a562372c848c0d520aa6d4da0278793aecb4ed6cfe5
-  data.tar.gz: e28f3f328bcefdf82382b854a7dc7841367f9b6f42e83c5a65fb62082a6985a7
+  metadata.gz: 6441fbc9f1b8de1aa81f70d435a4be4c6355844160e23da06c65501273c8be63
+  data.tar.gz: 7949fa9dc2bd082d290a806ecf6858b6acf1643f50669db176bd99ce250dfef9
 SHA512:
-  metadata.gz: 2db3a3be0b2d4300e53d00c6ae1e60f450e9201dd37efe62b2c703f1f50a075e212e9ac09265c1b1814a567b0b43ed4e142aa873eac32c20f3d240dab3860146
-  data.tar.gz: 3fd152d37cb0f5e4f4f0566f9c1b0577fc877cddf728abde9b6a35fcbce5fa0834c1c2dbfa0c686f431e8537126a7aef8fe20606aa2b88694108cedff895a431
+  metadata.gz: 8c0a6a7d3b6e50582e0f9a115809139710f75b60fe77ce0f6ee74d83e55cfa5f7832e46bba109f9e1d32d878bbaaf1fce7f4621127c055f7074205cb3c026152
+  data.tar.gz: c462624a3b0b1b59c1b41ba9bec587ca6fc35f8de67c74f3b33c92b34637dec07491365ace8794fdc587e506e2a26a7324a02b1773d1005728604ff4c7987dac

data/README.md

@@ -1,42 +1,50 @@
 # archive_storage
 
-Zero-downtime archival storage for CarrierWave uploads.
+Archival storage for Rails uploaders.
 
-`archive_storage` moves older uploaded files from one storage backend to another, keeps a registry of the current file location, and routes reads to the right backend. It currently integrates with CarrierWave; support for other uploader libraries can be added later without changing the registry model.
+`archive_storage` moves older uploaded files from a primary storage backend to one or more archive backends, records the current file location in a registry table, and keeps reads routed through the uploader.
 
-Supported storage adapters:
+The gem currently supports CarrierWave. The storage, registry, and migration layers are intentionally not tied to CarrierWave, so support for other Rails uploader libraries can be added later.
 
-- S3-compatible object storage, including MinIO and AWS S3
-- filesystem/NFS
-- memory adapter for tests
+## Contents
 
-Typical use cases:
-
-- `main` S3/MinIO bucket -> `archive_001` cold bucket
-- `archive_001` -> `archive_002` when the first archive fills up
-- NFS/local disk -> S3-compatible archive storage
+- [Features](#features)
+- [Installation](#installation)
+- [Getting Started](#getting-started)
+- [Configuration](#configuration)
+- [CarrierWave](#carrierwave)
+- [Policies](#policies)
+- [Scheduled Jobs](#scheduled-jobs)
+- [Command Line](#command-line)
+- [Migration Flow](#migration-flow)
+- [Verification](#verification)
+- [Cleanup](#cleanup)
+- [Registry](#registry)
+- [Development](#development)
 
 ## Features
 
-- model-first DSL: `archive_storage_for :file`
-- automatic CarrierWave storage wiring
-- ActiveRecord registry table: `archive_storage_files`
-- dry-run planning
-- scheduled enqueueing
-- background migration jobs
-- copy, verify, read switch, fallback read, delayed source cleanup
-- optional CarrierWave versions/thumbs migration
+- Model-level archive policy with `archive_storage_for :file`
+- CarrierWave integration without changing shared base uploaders globally
+- Multiple archive storages, for example `archive_001`, then `archive_002`
+- S3-compatible storage, filesystem/NFS storage, and a memory adapter for tests
+- ActiveRecord registry table for file location and migration state
+- Dry-run planning
+- Scheduled enqueueing
+- Background migration jobs
+- Copy, verify, read switch, fallback read, and delayed source cleanup
+- Optional CarrierWave version/thumb migration
 - GoodJob, ActiveJob, Sidekiq, `sidekiq-cron`, and `sidekiq-scheduler` support
 
 ## Installation
 
-Add the gem:
+Add the gem to your Rails application:
 
 ```ruby
 gem "archive_storage"
 ```
 
-For S3-compatible storage:
+For S3-compatible storage, also add:
 
 ```ruby
 gem "aws-sdk-s3"
@@ -49,9 +57,9 @@ bin/rails generate archive_storage:install
 bin/rails db:migrate
 ```
 
-## Configuration
+## Getting Started
 
-Define the storage backends and scheduled archive jobs.
+Configure storages:
 
 ```ruby
 # config/initializers/archive_storage.rb
@@ -86,6 +94,85 @@ ArchiveStorage.configure do |config|
     s.region = "us-east-1"
     s.path_style = true
   end
+end
+```
+
+Add a policy to the model that owns the upload:
+
+```ruby
+class ProjectDocument < ApplicationRecord
+  scope :ready_for_archive, -> { where("created_at <= ?", 90.days.ago) }
+
+  mount_uploader :file, DocumentUploader
+
+  archive_storage_for :file do
+    primary :main
+
+    archive :archive_001,
+            after: 90.days,
+            scope: :ready_for_archive,
+            max_byte_size: 3.gigabytes,
+            if: ->(record) { record.closed? }
+
+    archive :archive_002,
+            after: 2.years,
+            scope: ->(records) { records.where(priority: "low") },
+            if: ->(record) { record.closed? }
+
+    read_fallbacks :main, :archive_001, :archive_002
+  end
+end
+```
+
+Keep the uploader focused on paths and filenames:
+
+```ruby
+class DocumentUploader < CarrierWave::Uploader::Base
+  def store_dir
+    "uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
+  end
+end
+```
+
+Run a dry plan:
+
+```bash
+bin/rails archive_storage:plan MODEL=ProjectDocument MOUNT=file
+```
+
+Enqueue migrations:
+
+```bash
+bin/rails archive_storage:enqueue MODEL=ProjectDocument MOUNT=file LIMIT=10000
+```
+
+## Configuration
+
+`archive_storage` needs storage definitions and, optionally, schedules and runtime defaults.
+
+```ruby
+# config/initializers/archive_storage.rb
+
+ArchiveStorage.configure do |config|
+  config.storage :main do |s|
+    s.provider = :s3
+    s.endpoint = ENV.fetch("MAIN_STORAGE_ENDPOINT")
+    s.bucket = "production-main"
+    s.access_key_id = ENV.fetch("MAIN_STORAGE_ACCESS_KEY")
+    s.secret_access_key = ENV.fetch("MAIN_STORAGE_SECRET_KEY")
+    s.region = "us-east-1"
+    s.path_style = true
+  end
+
+  config.storage :archive_001 do |s|
+    s.provider = :s3
+    s.endpoint = ENV.fetch("ARCHIVE_001_ENDPOINT")
+    s.bucket = "production-archive-001"
+    s.access_key_id = ENV.fetch("ARCHIVE_001_ACCESS_KEY")
+    s.secret_access_key = ENV.fetch("ARCHIVE_001_SECRET_KEY")
+    s.region = "us-east-1"
+    s.path_style = true
+  end
 
   config.schedule :archive_documents,
                   cron: "0 0-6,22,23 * * 1-5",
@@ -93,7 +180,7 @@ ArchiveStorage.configure do |config|
                   mounted_as: :file,
                   migration_rate: 10_000
 
-  # Optional defaults:
+  # Defaults:
   #
   # config.job_backend = :active_job # :active_job, :good_job, :sidekiq, or :inline
   # config.migration_queue = :default
@@ -105,18 +192,20 @@ ArchiveStorage.configure do |config|
 end
 ```
 
-Filesystem/NFS storage can be mixed with S3-compatible storage:
+Filesystem or NFS storage can be used as either source or archive storage:
 
 ```ruby
-config.storage :nfs_main do |s|
-  s.provider = :filesystem
-  s.root_path = "/mnt/uploads"
+ArchiveStorage.configure do |config|
+  config.storage :nfs_main do |s|
+    s.provider = :filesystem
+    s.root_path = "/mnt/uploads"
+  end
 end
 ```
 
-## Model Policy
+## CarrierWave
 
-Put archive policy next to the model that owns the file.
+`archive_storage_for` automatically wires the mounted CarrierWave uploader to `storage :archive_storage`.
 
 ```ruby
 class ProjectDocument < ApplicationRecord
@@ -124,62 +213,107 @@ class ProjectDocument < ApplicationRecord
 
   archive_storage_for :file do
     primary :main
+    archive :archive_001, after: 90.days, scope: :ready_for_archive
+  end
+end
+```
 
-    archive :archive_001,
-      after: 90.days,
-      scope: :ready_for_archive,
-      if: ->(record) { record.closed? }
+The gem creates a per-model/per-mount uploader subclass under the model and uses that subclass internally. This avoids changing a shared uploader class globally when the same uploader is mounted by many models.
 
-    archive :archive_002,
-      after: 2.years,
-      scope: ->(records) { records.where(priority: "low") },
-      if: ->(record) { record.closed? }
+CarrierWave versions are not migrated by default. Enable them only when the generated files should follow the same archive policy:
 
-    read_fallbacks :main, :archive_001, :archive_002
+```ruby
+archive_storage_for :file do
+  include_versions true
+end
+```
 
-    # Optional:
-    #
-    # delete_source_after verification: true, delay: 7.days
-    # include_versions true
-    # versions :thumb, :preview
-    # timestamp_attribute :created_at
-  end
+To migrate only selected versions:
+
+```ruby
+archive_storage_for :file do
+  versions :thumb, :preview
 end
 ```
 
-`archive_storage_for` automatically wires the mounted CarrierWave uploader to `storage :archive_storage`. The uploader can stay focused on path, filename, and version behavior:
+## Policies
+
+Policies are declared on the model:
 
 ```ruby
-class DocumentUploader < CarrierWave::Uploader::Base
-  def store_dir
-    "uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
+archive_storage_for :file do
+  primary :main
+
+  archive :archive_001,
+          after: 90.days,
+          scope: :ready_for_archive,
+          max_byte_size: 3.gigabytes,
+          if: ->(record) { record.closed? }
+
+  read_fallbacks :main, :archive_001
+
+  # delete_source_after verification: true, delay: 7.days
+  # include_versions true
+  # versions :thumb, :preview
+  # timestamp_attribute :created_at
+end
+```
+
+Policy options:
+
+- `primary` sets the storage used for new uploads.
+- `archive` adds an archive destination rule.
+- `after` is checked in Ruby after records are loaded.
+- `scope` narrows the ActiveRecord relation before scanning records.
+- `if` applies a per-record Ruby predicate.
+- `max_byte_size` skips oversized files using storage metadata and checks again before copy.
+- `read_fallbacks` sets the read recovery order.
+- `delete_source_after` configures the per-mount cleanup delay.
+- `include_versions` and `versions` control CarrierWave versions.
+- `timestamp_attribute` changes the attribute used by `after`.
+
+For large tables, keep heavy filters in SQL:
+
+```ruby
+class ProjectDocument < ApplicationRecord
+  scope :ready_for_archive, -> {
+    where("created_at <= ?", 90.days.ago).where(status: "closed")
+  }
+
+  archive_storage_for :file do
+    primary :main
+    archive :archive_001, after: 90.days, scope: :ready_for_archive
   end
 end
 ```
 
-Policy notes:
+`after` is useful as a safety check, but it should not replace a selective SQL scope on large production tables.
 
-- `primary` is where new uploads are stored.
-- `archive` rules are checked in order; the last eligible rule wins.
-- `scope` narrows the model relation before records are scanned. It can be a model scope name, a relation, or a callable that receives the current relation.
-- `read_fallbacks` is the read-recovery order when registry metadata is missing or a configured fallback error is raised.
-- By default only the original CarrierWave file is planned. Use `include_versions true` or `versions ...` when thumbnails/previews must move too.
+Archive rules are checked in order. The last eligible rule wins, which allows progressive archives:
+
+```ruby
+archive_storage_for :file do
+  primary :main
+  archive :archive_001, after: 90.days, scope: :ready_for_archive
+  archive :archive_002, after: 2.years, scope: :ready_for_archive
+end
+```
 
 ## Scheduled Jobs
 
-Schedules are declared in global configuration:
+Schedules are declared in `ArchiveStorage.configure`:
 
 ```ruby
 ArchiveStorage.configure do |config|
   config.schedule :archive_documents,
-                  cron: "0 0-6,22,23 * * 1-5",
+                  cron: "*/10 * * * *",
                   model: "ProjectDocument",
                   mounted_as: :file,
                   migration_rate: 10_000
 end
 ```
 
-`migration_rate` means at most this many files are enqueued by one scheduled run.
+`migration_rate` is the maximum number of files enqueued by one scheduled run. If the cron runs every 10 minutes, `migration_rate: 10_000` means up to 10,000 files per run, not per hour.
 
 `archive_storage` registers scheduler entries automatically. You do not need to merge `ArchiveStorage.good_job_cron` or `ArchiveStorage.sidekiq_cron` into your application config.
 
@@ -187,7 +321,7 @@ end
 
 When `good_job` is present, `archive_storage` appends its entries to `config.good_job.cron` after Rails initialization. Existing GoodJob cron entries are preserved.
 
-Enable GoodJob cron in the app environment where the scheduler should run:
+Enable GoodJob cron in the environment where scheduling should run:
 
 ```ruby
 # config/environments/production.rb
@@ -199,7 +333,7 @@ end
 
 ### Sidekiq
 
-Use Sidekiq for migration jobs:
+Use Sidekiq workers for archive jobs:
 
 ```ruby
 # config/initializers/archive_storage.rb
@@ -217,14 +351,14 @@ gem "sidekiq-cron"
 gem "sidekiq-scheduler"
 ```
 
-On Sidekiq server startup, `archive_storage` adds its own schedules without deleting existing jobs:
+On Sidekiq server startup, `archive_storage` adds its schedules without deleting existing schedules:
 
-- with `sidekiq-cron`, it uses non-destructive `Sidekiq::Cron::Job.load_from_hash`
+- with `sidekiq-cron`, it uses `Sidekiq::Cron::Job.load_from_hash`
 - with `sidekiq-scheduler`, it uses `Sidekiq.set_schedule` and reloads the scheduler
 
-Existing jobs from `sidekiq.yml`, `config/schedule.yml`, or custom initializers remain in place.
+Existing jobs from `sidekiq.yml`, `config/schedule.yml`, and custom initializers remain in place.
 
-## Commands
+## Command Line
 
 ```bash
 bin/rails archive_storage:plan MODEL=ProjectDocument MOUNT=file
@@ -235,66 +369,52 @@ bin/rails archive_storage:cleanup_source
 bin/rails archive_storage:status
 ```
 
-Options:
+Supported environment options:
 
 ```bash
 MODEL=ProjectDocument
 MOUNT=file
+UPLOADER=DocumentUploader
 OLDER_THAN=90d
 LIMIT=10000
 INLINE=true
 ESTIMATE_SIZES=false
 ```
 
-`UPLOADER=DocumentUploader` is still accepted for advanced/legacy uploader-level configurations.
-
 Command behavior:
 
 - `plan` prints a dry-run plan.
-- `enqueue` and `migrate` enqueue migration jobs by default.
+- `enqueue` enqueues migration jobs.
+- `migrate` enqueues migration jobs by default.
 - `migrate INLINE=true` runs migration inline.
-- `verify` re-checks already migrated files.
-- `cleanup_source` deletes verified source copies that are past the cleanup delay.
+- `verify` rechecks already migrated files.
+- `cleanup_source` deletes verified source copies after the cleanup delay.
 - `status` prints registry counters.
 
+`MODEL` and `MOUNT` are recommended for model-level policies. `UPLOADER` is still accepted for advanced or legacy uploader-level configurations.
+
 ## Migration Flow
 
+The migration process is intentionally staged:
+
 ```text
 source only
-source + destination copied
-destination verified
-registry points reads to destination
+source + archive copied
+archive verified
+registry points reads to archive
 reads can fallback to source
 source deleted later when cleanup is enabled
 ```
 
-Source deletion is disabled by default:
-
-```ruby
-config.delete_source_enabled = false
-```
-
-Turn it on only after the migration path has been verified in production:
-
-```ruby
-config.delete_source_enabled = true
-```
-
-Per-mount cleanup delay:
-
-```ruby
-archive_storage_for :file do
-  delete_source_after verification: true, delay: 7.days
-end
-```
+This keeps the application reading through the uploader while files are being copied and verified.
 
 ## Verification
 
-The default strategy is `:auto`.
+The default verification strategy is `:auto`.
 
 `archive_storage` does not blindly trust S3 ETags. Multipart S3 uploads can have ETags like `hash-3`, and uploading the same bytes to another storage can produce a different ETag.
 
-Strategies:
+Available strategies:
 
 - `:auto` - size check, then checksum when available, then non-multipart ETag, otherwise size-only
 - `:checksum` - require matching checksums
@@ -309,40 +429,86 @@ ArchiveStorage.configure do |config|
 end
 ```
 
-## Registry
+For filesystem/NFS sources, checksums are based on the bytes read from disk. For S3-compatible sources, checksum and ETag metadata are used when available according to the configured strategy.
 
-The generated migration creates `archive_storage_files`.
+## Cleanup
 
-The registry stores:
+Source deletion is disabled by default:
 
-- model identity: `record_type`, `record_id`, `mounted_as`, `uploader`
-- object identity: `identifier`, `storage_key`, source/target keys
-- storage state: `current_storage`, `source_storage`, `target_storage`
-- migration state: enqueue, migration, verification, cleanup timestamps
-- metadata: byte size, checksum, content type, attempts, last error
+```ruby
+ArchiveStorage.configure do |config|
+  config.delete_source_enabled = false
+end
+```
 
-Business tables do not need extra columns for archive location.
+Enable it only after planning, migration, and reads have been verified in production:
 
-## CarrierWave Versions
+```ruby
+ArchiveStorage.configure do |config|
+  config.delete_source_enabled = true
+end
+```
 
-CarrierWave versions are disabled by default.
+It can also be a callable, which is useful for feature flags:
 
 ```ruby
-archive_storage_for :file do
-  include_versions true
+ArchiveStorage.configure do |config|
+  config.delete_source_enabled = -> { Unleash.enabled?(:archive_storage_delete_source) }
 end
 ```
 
-To migrate only selected versions:
+Configure cleanup delay per mount:
 
 ```ruby
 archive_storage_for :file do
-  versions :thumb, :preview
+  primary :main
+  archive :archive_001, after: 90.days, scope: :ready_for_archive
+  delete_source_after verification: true, delay: 7.days
 end
 ```
 
-Use this only when those files are stored and read as part of the same archival policy. It can multiply the number of objects planned for migration.
+Run cleanup:
+
+```bash
+bin/rails archive_storage:cleanup_source
+```
+
+## Registry
+
+The generated migration creates `archive_storage_files`.
+
+The registry stores:
+
+- model identity: `record_type`, `record_id`, `mounted_as`, `uploader`
+- object identity: `identifier`, `storage_key`, `source_storage_key`, `target_storage_key`
+- storage state: `current_storage`, `source_storage`, `target_storage`
+- migration state: `enqueued_at`, `migration_started_at`, `migrated_at`, `verified_at`, `source_deleted_at`
+- metadata: `byte_size`, `checksum`, `content_type`, `attempts`, `last_error`
+
+The registry has a unique identity index on:
+
+```text
+record_type, record_id, mounted_as, identifier, storage_key
+```
+
+Business tables do not need extra columns for archive location.
+
+If an application generated an older migration without the unique identity index, add a migration that replaces the old identity index with the unique one before relying on parallel enqueueing.
+
+## Development
+
+Run the test suite:
+
+```bash
+bundle exec rake test
+```
+
+Build the gem:
+
+```bash
+bundle exec gem build archive_storage.gemspec
+```
 
-## Current Scope
+## License
 
-This MVP is focused on Rails, ActiveRecord, and CarrierWave. The storage and registry layers are not CarrierWave-specific, so other uploader integrations can be added later.
+MIT.

data/archive_storage.gemspec

@@ -8,8 +8,8 @@ Gem::Specification.new do |spec|
   spec.authors = ["E. Tashkovyan"]
   spec.email = []
 
-  spec.summary = "Policy-based archive storage and zero-downtime file migration."
-  spec.description = "Move uploads across storage backends such as filesystem, NFS, MinIO, and S3 without downtime."
+  spec.summary = "Archival storage for Rails uploaders."
+  spec.description = "Move older Rails uploads across storage backends such as filesystem, NFS, MinIO, and S3."
   spec.homepage = "https://github.com/estashkovyan/archive_storage"
   spec.license = "MIT"
   spec.required_ruby_version = ">= 3.1.0"

data/lib/archive_storage/configuration.rb

@@ -44,6 +44,10 @@ module ArchiveStorage
       @verification_strategy = :checksum if value
     end
 
+    def delete_source_enabled?
+      delete_source_enabled.respond_to?(:call) ? !!delete_source_enabled.call : !!delete_source_enabled
+    end
+
     def storage(name, &block)
       config = (@storages[name.to_sym] ||= StorageConfig.new(name))
       block.call(config) if block

data/lib/archive_storage/errors.rb

@@ -1,9 +1,10 @@
 # frozen_string_literal: true
 
 module ArchiveStorage
-    Error = Class.new(StandardError)
-    ConfigurationError = Class.new(Error)
-    NotFoundError = Class.new(Error)
-    VerificationError = Class.new(Error)
-    RegistryUnavailableError = Class.new(Error)
+  Error = Class.new(StandardError)
+  ConfigurationError = Class.new(Error)
+  NotFoundError = Class.new(Error)
+  VerificationError = Class.new(Error)
+  RegistryUnavailableError = Class.new(Error)
+  MaxByteSizeExceededError = Class.new(Error)
 end

data/lib/archive_storage/migrator.rb

@@ -48,6 +48,8 @@ module ArchiveStorage
         source = ArchiveStorage.adapter(source_storage)
         target = ArchiveStorage.adapter(target_storage)
 
+        validate_max_byte_size!(file_record, source, source_key, target_storage)
+
         target.copy_from(source, source_key, target_key)
         verification = Verifier.new.verify!(
           source_adapter: source,
@@ -105,7 +107,7 @@ module ArchiveStorage
     attr_reader :planner
 
     def cleanup_ready?(file_record)
-      return false unless ArchiveStorage.configuration.delete_source_enabled
+      return false unless ArchiveStorage.configuration.delete_source_enabled?
       return false unless file_record.source_storage
       return false if file_record.source_deleted_at
 
@@ -130,6 +132,17 @@ module ArchiveStorage
       nil
     end
 
+    def validate_max_byte_size!(file_record, source_adapter, source_key, target_storage)
+      rule = policy_for(file_record)&.rule_for_storage(target_storage)
+      return unless rule&.max_byte_size?
+
+      metadata = source_adapter.head(source_key)
+      return if rule.byte_size_allowed?(metadata.byte_size)
+
+      raise MaxByteSizeExceededError,
+            "object #{source_key.inspect} is #{metadata.byte_size} bytes; max is #{rule.max_byte_size}"
+    end
+
     def safe_update_error(file_record, error)
       file_record.update!(last_error: "#{error.class}: #{error.message}") if file_record.respond_to?(:update!)
     rescue StandardError

data/lib/archive_storage/model.rb

@@ -7,9 +7,14 @@ module ArchiveStorage
 
       policy = PolicyBuilder.build(&block)
       uploader_class = archive_storage_uploader_for(mounted_as)
-
-      ArchiveStorage.wire_carrierwave_uploader!(uploader_class)
-      ArchiveStorage.register_mount(self, mounted_as, uploader: uploader_class, policy: policy)
+      archive_uploader_class = ArchiveStorage.build_mount_uploader!(
+        self,
+        mounted_as,
+        uploader_class
+      )
+
+      ArchiveStorage.wire_carrierwave_uploader!(archive_uploader_class)
+      ArchiveStorage.register_mount(self, mounted_as, uploader: archive_uploader_class, policy: policy)
 
       archive_storage_policies[mounted_as.to_sym] = policy
     end

data/lib/archive_storage/planner.rb

@@ -126,11 +126,13 @@ module ArchiveStorage
         storage_key: storage_key,
         default: policy.primary_storage_key
       )
-      target_storage = policy.target_storage_for(record)
-      return nil unless target_storage
-      return nil if current_storage.to_sym == target_storage.to_sym
+      now = Time.now
+      metadata = source_metadata_for(policy, record, current_storage, storage_key, now: now)
+      target_rule = policy.target_rule_for(record, now: now, byte_size: metadata&.byte_size)
+      return nil unless target_rule
 
-      metadata = estimate_metadata(target_storage, policy.primary_storage_key, storage_key)
+      target_storage = target_rule.storage_key
+      return nil if current_storage.to_sym == target_storage.to_sym
 
       Candidate.new(
         record: record,
@@ -179,12 +181,21 @@ module ArchiveStorage
       nil
     end
 
-    def estimate_metadata(_target_storage, source_storage, storage_key)
+    def estimate_metadata(source_storage, storage_key)
       return nil unless estimate_sizes
 
       ArchiveStorage.adapter(source_storage).head(storage_key)
     rescue StandardError
       nil
     end
+
+    def source_metadata_for(policy, record, source_storage, storage_key, now:)
+      return estimate_metadata(source_storage, storage_key) if estimate_sizes
+      return nil unless policy.requires_byte_size_for?(record, now: now)
+
+      ArchiveStorage.adapter(source_storage).head(storage_key)
+    rescue StandardError
+      nil
+    end
   end
 end

data/lib/archive_storage/policy.rb

@@ -27,12 +27,39 @@ module ArchiveStorage
       primary_storage&.storage_key
     end
 
-    def target_storage_for(record, now: Time.now)
+    def target_storage_for(record, now: Time.now, byte_size: nil)
+      target_rule_for(record, now: now, byte_size: byte_size)&.storage_key
+    end
+
+    def target_rule_for(record, now: Time.now, byte_size: nil)
       eligible_rules = rules.select do |rule|
-        rule.eligible?(record, now: now, timestamp_attribute: timestamp_attribute)
+        rule.eligible?(
+          record,
+          now: now,
+          timestamp_attribute: timestamp_attribute,
+          byte_size: byte_size
+        )
       end
 
-      eligible_rules.last&.storage_key
+      eligible_rules.last
+    end
+
+    def rule_for_storage(storage_key)
+      rules.reverse.find { |rule| rule.storage_key == storage_key.to_sym }
+    end
+
+    def requires_byte_size?
+      rules.any?(&:max_byte_size?)
+    end
+
+    def requires_byte_size_for?(record, now: Time.now)
+      rules.any? do |rule|
+        rule.requires_byte_size_for?(
+          record,
+          now: now,
+          timestamp_attribute: timestamp_attribute
+        )
+      end
     end
 
     def apply_rule_scopes(scope)

data/lib/archive_storage/policy_builder.rb

@@ -44,7 +44,8 @@ module ArchiveStorage
         name,
         after: options[:after],
         condition: options[:if],
-        scope: options[:scope]
+        scope: options[:scope],
+        max_byte_size: options[:max_byte_size]
       )
     end
 

data/lib/archive_storage/registry.rb

@@ -4,46 +4,39 @@ require_relative "errors"
 require_relative "models/file_record"
 
 module ArchiveStorage
-    class Registry
-      def available?
-        defined?(::ActiveRecord::Base) &&
-          ::ActiveRecord::Base.connected? &&
-          record_class.table_exists?
-      rescue StandardError
-        false
-      end
+  class Registry
+    def available?
+      defined?(::ActiveRecord::Base) &&
+        ::ActiveRecord::Base.connected? &&
+        record_class.table_exists?
+    rescue StandardError
+      false
+    end
 
-      def find_for_uploader(uploader, identifier:, storage_key:)
-        return nil unless available?
-        return nil unless uploader_identity_available?(uploader)
+    def find_for_uploader(uploader, identifier:, storage_key:)
+      return nil unless available?
+      return nil unless uploader_identity_available?(uploader)
 
-        record_class.find_by(
-          record_type: uploader.model.class.name,
-          record_id: uploader.model.id,
-          mounted_as: uploader.mounted_as.to_s,
-          identifier: identifier.to_s,
-          storage_key: storage_key.to_s
-        )
-      end
+      record_class.find_by(
+        identity_for_uploader(uploader, identifier: identifier, storage_key: storage_key)
+      )
+    end
 
-      def current_storage_for(uploader, identifier:, storage_key:, default:)
-        find_for_uploader(
-          uploader,
-          identifier: identifier,
-          storage_key: storage_key
-        )&.current_storage&.to_sym || default
-      end
+    def current_storage_for(uploader, identifier:, storage_key:, default:)
+      find_for_uploader(
+        uploader,
+        identifier: identifier,
+        storage_key: storage_key
+      )&.current_storage&.to_sym || default
+    end
 
-      def upsert_for_uploader(uploader, identifier:, storage_key:, current_storage:, metadata: {})
-        return nil unless available?
-        return nil unless uploader_identity_available?(uploader)
+    def upsert_for_uploader(uploader, identifier:, storage_key:, current_storage:, metadata: {})
+      return nil unless available?
+      return nil unless uploader_identity_available?(uploader)
 
+      with_unique_retry do
         record = record_class.find_or_initialize_by(
-          record_type: uploader.model.class.name,
-          record_id: uploader.model.id,
-          mounted_as: uploader.mounted_as.to_s,
-          identifier: identifier.to_s,
-          storage_key: storage_key.to_s
+          identity_for_uploader(uploader, identifier: identifier, storage_key: storage_key)
         )
 
         record.uploader = uploader.class.name
@@ -54,16 +47,14 @@ module ArchiveStorage
         record.save!
         record
       end
+    end
 
-      def claim_candidate(candidate)
-        raise RegistryUnavailableError, "archive_storage_files table is not available" unless available?
+    def claim_candidate(candidate)
+      raise RegistryUnavailableError, "archive_storage_files table is not available" unless available?
 
+      with_unique_retry do
         record = record_class.find_or_initialize_by(
-          record_type: candidate.record.class.name,
-          record_id: candidate.record.id,
-          mounted_as: candidate.mounted_as.to_s,
-          identifier: candidate.identifier.to_s,
-          storage_key: candidate.storage_key.to_s
+          identity_for_candidate(candidate)
         )
         return nil unless claimable?(record)
 
@@ -80,30 +71,64 @@ module ArchiveStorage
         record.save!
         record
       end
+    end
 
-      alias ensure_for_candidate claim_candidate
+    alias ensure_for_candidate claim_candidate
 
-      private
+    private
 
-      def record_class
-        ArchiveStorage.configuration.registry_class
-      end
+    def record_class
+      ArchiveStorage.configuration.registry_class
+    end
 
-      def claimable?(record)
-        return false if record.respond_to?(:migrated_at) && record.migrated_at
-        return true unless record.respond_to?(:enqueued_at)
-        return true unless record.enqueued_at
+    def claimable?(record)
+      return false if record.respond_to?(:migrated_at) && record.migrated_at
+      return true unless record.respond_to?(:enqueued_at)
+      return true unless record.enqueued_at
 
-        record.enqueued_at <= Time.now - ArchiveStorage.configuration.enqueue_claim_ttl
-      end
+      record.enqueued_at <= Time.now - ArchiveStorage.configuration.enqueue_claim_ttl
+    end
 
-      def uploader_identity_available?(uploader)
-        uploader.respond_to?(:model) &&
-          uploader.model &&
-          uploader.model.respond_to?(:id) &&
-          uploader.model.id &&
-          uploader.respond_to?(:mounted_as) &&
-          uploader.mounted_as
-      end
+    def uploader_identity_available?(uploader)
+      uploader.respond_to?(:model) &&
+        uploader.model &&
+        uploader.model.respond_to?(:id) &&
+        uploader.model.id &&
+        uploader.respond_to?(:mounted_as) &&
+        uploader.mounted_as
+    end
+
+    def identity_for_uploader(uploader, identifier:, storage_key:)
+      {
+        record_type: uploader.model.class.name,
+        record_id: uploader.model.id,
+        mounted_as: uploader.mounted_as.to_s,
+        identifier: identifier.to_s,
+        storage_key: storage_key.to_s
+      }
+    end
+
+    def identity_for_candidate(candidate)
+      {
+        record_type: candidate.record.class.name,
+        record_id: candidate.record.id,
+        mounted_as: candidate.mounted_as.to_s,
+        identifier: candidate.identifier.to_s,
+        storage_key: candidate.storage_key.to_s
+      }
+    end
+
+    def with_unique_retry
+      yield
+    rescue StandardError => error
+      raise unless unique_violation?(error)
+
+      yield
+    end
+
+    def unique_violation?(error)
+      defined?(::ActiveRecord::RecordNotUnique) &&
+        error.is_a?(::ActiveRecord::RecordNotUnique)
     end
+  end
 end

data/lib/archive_storage/storage_rule.rb

@@ -2,19 +2,21 @@
 
 module ArchiveStorage
   class StorageRule
-    attr_reader :role, :storage_key, :after, :condition, :scope
+    attr_reader :role, :storage_key, :after, :condition, :scope, :max_byte_size
 
-    def initialize(role, storage_key, after: nil, condition: nil, scope: nil)
+    def initialize(role, storage_key, after: nil, condition: nil, scope: nil, max_byte_size: nil)
       @role = role.to_sym
       @storage_key = storage_key.to_sym
       @after = after
       @condition = condition
       @scope = scope
+      @max_byte_size = normalize_byte_size(max_byte_size)
     end
 
-    def eligible?(record, now:, timestamp_attribute:)
+    def eligible?(record, now:, timestamp_attribute:, byte_size: nil)
       old_enough?(record, now: now, timestamp_attribute: timestamp_attribute) &&
-        condition_matches?(record)
+        condition_matches?(record) &&
+        byte_size_allowed?(byte_size)
     end
 
     def scoped?
@@ -32,8 +34,31 @@ module ArchiveStorage
       end
     end
 
+    def max_byte_size?
+      !max_byte_size.nil?
+    end
+
+    def requires_byte_size_for?(record, now:, timestamp_attribute:)
+      max_byte_size? &&
+        old_enough?(record, now: now, timestamp_attribute: timestamp_attribute) &&
+        condition_matches?(record)
+    end
+
+    def byte_size_allowed?(byte_size)
+      return true unless max_byte_size?
+      return false if byte_size.nil?
+
+      byte_size <= max_byte_size
+    end
+
     private
 
+    def normalize_byte_size(value)
+      return nil if value.nil?
+
+      Integer(value)
+    end
+
     def old_enough?(record, now:, timestamp_attribute:)
       return true unless after
       return false unless record

data/lib/archive_storage/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module ArchiveStorage
-  VERSION = "0.1.0"
+  VERSION = "0.1.1"
 end

data/lib/archive_storage.rb

@@ -53,6 +53,15 @@ module ArchiveStorage
       configuration.mount(model, mounted_as, uploader: uploader, policy: policy)
     end
 
+    def build_mount_uploader!(model_class, mounted_as, uploader_class)
+      return uploader_class unless model_class.respond_to?(:uploaders)
+      return uploader_class unless model_class.uploaders.respond_to?(:[]=)
+
+      subclass = mount_uploader_subclass(model_class, mounted_as, uploader_class)
+      model_class.uploaders[mounted_as.to_sym] = subclass
+      subclass
+    end
+
     def wire_carrierwave_uploader!(uploader_class)
       return unless uploader_class
 
@@ -101,6 +110,17 @@ module ArchiveStorage
       nil
     end
 
+    def mount_uploader_subclass(model_class, mounted_as, uploader_class)
+      const_name = "ArchiveStorage#{camelize(mounted_as)}Uploader"
+      return model_class.const_get(const_name, false) if model_class.const_defined?(const_name, false)
+
+      model_class.const_set(const_name, Class.new(uploader_class))
+    end
+
+    def camelize(value)
+      value.to_s.split("_").map(&:capitalize).join
+    end
+
     def mount_policy_for_uploader(uploader)
       return nil unless uploader.respond_to?(:model) && uploader.model
       return nil unless uploader.respond_to?(:mounted_as) && uploader.mounted_as

data/lib/generators/archive_storage/templates/create_archive_storage_files.rb

@@ -35,7 +35,8 @@ class CreateArchiveStorageFiles < ActiveRecord::Migration[7.0]
     end
 
     add_index :archive_storage_files,
-              [:record_type, :record_id, :mounted_as, :identifier],
+              [:record_type, :record_id, :mounted_as, :identifier, :storage_key],
+              unique: true,
               name: "idx_archive_storage_identity"
 
     add_index :archive_storage_files,

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: archive_storage
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.1
 platform: ruby
 authors:
 - E. Tashkovyan
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2026-05-26 00:00:00.000000000 Z
+date: 2026-06-08 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -152,8 +152,8 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '13.0'
-description: Move uploads across storage backends such as filesystem, NFS, MinIO,
-  and S3 without downtime.
+description: Move older Rails uploads across storage backends such as filesystem,
+  NFS, MinIO, and S3.
 email: []
 executables: []
 extensions: []
@@ -223,5 +223,5 @@ requirements: []
 rubygems_version: 3.5.22
 signing_key:
 specification_version: 4
-summary: Policy-based archive storage and zero-downtime file migration.
+summary: Archival storage for Rails uploaders.
 test_files: []