file_hutch 0.1.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e2b2054f9f101d16f7ea61d375dfcbb9715838b5988c517bd547da595759f454
-  data.tar.gz: b81dc343e6ee21e17d396502b0700ff7e9ed253e5729f928eca28673248c2801
+  metadata.gz: 4b59ec3bdd7aeb52b1e3b135698091716fcb3a38ac5ce686ec648631fe7b7087
+  data.tar.gz: 17485d436dbdcf4ddd85cd8c8c4706343281e13c433b0ba0544df81293ba0858
 SHA512:
-  metadata.gz: 48a44a32ed534d38c731ab9cb9be31c856b635a642035ae5942cdcfffca73743c973f8fe11cb7a2dcccf7206e503b650d1997dd98c17f85dbcee705aebb20adc
-  data.tar.gz: 13cfab0ce82a0afa0594818e1508b994ee066790319fb621a18c7bc3033bc787a2a1afa22699c29a92bc1e783b101a1bb9e7bb9ae0e6e08465c813286d18a98a
+  metadata.gz: c07c3d0cc5169acbb0ee860d8374be31128d62ba6fd9e03fbca8defed997b90313e706eb13e0f84f7dff5b2d12d0e6f036fd34c289eaa1f12f6f4523d7a8e088
+  data.tar.gz: e3b329321814427ffee8e657ae512e01201b9b5db1a505c64513f47f29e8f23e7db6d19671643ad7cdfb5632b5f1738f354274cdd351dc97530fdd6960d25f57

data/CHANGELOG.md

@@ -1,5 +1,31 @@
 # Changelog
 
+## 0.3.0 — 2026-09-24
+
+### Removed
+
+- `has_file_hutch_file`, deprecated in 0.2.0 as promised. Rename it to `has_hutch`; the options
+  are the same.
+
+### Rails
+
+- `bin/rails file_hutch:backfill MODEL=… FROM=…` moves an Active Storage attachment onto its
+  `has_hutch` column: streams each blob through FileHutch, or with `ADOPT_FROM=conn_…` registers
+  blobs in place when the Active Storage bucket is connected. Restartable; fills only blank rows.
+  Backed by `FileHutch::Backfill`.
+
+### Client
+
+- `adopt(key:, storage_connection:)` registers an object already in a connected bucket as a file,
+  without copying it. Idempotent: adopting the same key again returns the same file.
+
+## 0.2.0 — 2026-09-16
+
+### Rails
+
+- `has_file_hutch_file` is now `has_hutch`, with the same options. The old name still works and
+  logs a deprecation warning; it will be removed in 0.3.
+
 ## 0.1.0 — 2026-09-15
 
 First release.

data/README.md

@@ -161,8 +161,8 @@ bin/rails generate file_hutch:attachment User avatar    # adds users.avatar_file
 
 ```ruby
 class User < ApplicationRecord
-  has_file_hutch_file :avatar, policy: "avatars"
-  has_file_hutch_file :contract, policy: "documents", dependent: false
+  has_hutch :avatar, policy: "avatars"
+  has_hutch :contract, policy: "documents", dependent: false
 end
 
 user.avatar = params[:avatar]           # uploaded IO → uploaded to storage on save
@@ -227,14 +227,14 @@ Submit buttons are disabled while uploading. The element dispatches `filehutch:s
 `filehutch:progress`, `filehutch:complete`, and `filehutch:error`. `directUpload(file, { url,
 policy, onProgress })` is exported for use without Stimulus.
 
-On save, `has_file_hutch_file` verifies the submitted id is a ready file under the declared policy,
+On save, `has_hutch` verifies the submitted id is a ready file under the declared policy,
 so a client cannot attach someone else's upload to the wrong field.
 
 ### Coming from Active Storage
 
 | Active Storage | file_hutch |
 | --- | --- |
-| `has_one_attached :avatar` | `has_file_hutch_file :avatar, policy: "avatars"` |
+| `has_one_attached :avatar` | `has_hutch :avatar, policy: "avatars"` |
 | `active_storage_blobs` + `attachments` tables | `users.avatar_file_id` |
 | `url_for(user.avatar)` | `user.avatar_url` / `user.avatar_signed_url` |
 | `user.avatar.variant(resize_to_fill: [200, 200])` | `user.avatar_transform_url("avatar")`, defined once in the dashboard |
@@ -242,6 +242,26 @@ so a client cannot attach someone else's upload to the wrong field.
 | `DirectUpload` JS | `file_hutch/direct_upload_controller` |
 | service.yml, CORS, signed URL code | policies in the FileHutch dashboard |
 
+Moving existing attachments across is one task. Add the column and `has_hutch`
+beside the Active Storage attachment, then:
+
+```sh
+bin/rails file_hutch:backfill MODEL=User FROM=avatar
+```
+
+Each blob is read through Active Storage and uploaded under the attachment's
+policy, and its file ID goes into the column. Only rows whose column is still
+blank are touched, so a run that stops half way (or leaves failures) carries on
+where it was when you run it again. `TO=` names the `has_hutch` attachment when
+it differs from the Active Storage one.
+
+If the bucket Active Storage writes to is connected to your FileHutch project,
+add `ADOPT_FROM=conn_…` and nothing is copied: each blob becomes a file where it
+already is, by its key.
+
+The Active Storage attachment is left alone. Read from the new column with a
+fallback to the old one until the backfill has filled every row, then remove it.
+
 ## Development
 
 ```sh

data/lib/file_hutch/backfill.rb

@@ -0,0 +1,73 @@
+# frozen_string_literal: true
+
+module FileHutch
+  # Moves one Active Storage attachment on a model onto its has_hutch column:
+  #
+  #   FileHutch::Backfill.new(model: User, from: :photo, to: :avatar).call
+  #
+  # Two ways, per run:
+  #
+  # * Stream (the default). Each blob's bytes are read through Active Storage
+  #   and uploaded to FileHutch under the attachment's policy. Works whatever
+  #   bucket Active Storage uses, and costs a download and an upload per file.
+  #
+  # * Adopt (adopt_from: "conn_…"). When the bucket Active Storage writes to is
+  #   connected to FileHutch, each blob is registered where it already is, by
+  #   its key. Nothing is copied or downloaded.
+  #
+  # Restartable. Only records whose column is still blank are touched, and the
+  # column is written as soon as each file exists, so a run that stops half way
+  # carries on from where it was. Adopting is idempotent on the server too.
+  #
+  # The Active Storage attachment is left alone: keep reading from it until the
+  # new column is filled everywhere, then remove it.
+  class Backfill
+    Result = Struct.new(:done, :skipped, :failed, keyword_init: true)
+
+    def initialize(model:, from:, to: from, adopt_from: nil, client: FileHutch.client, logger: FileHutch.config.logger)
+      @model, @from, @to, @adopt_from, @client, @logger = model, from.to_sym, to.to_sym, adopt_from.presence, client, logger
+      @options = model.file_hutch_files.fetch(@to) do
+        raise ArgumentError, "#{model.name} has no has_hutch :#{@to}"
+      end
+    end
+
+    def call
+      result = Result.new(done: 0, skipped: 0, failed: 0)
+
+      @model.where(column => [ nil, "" ]).find_each do |record|
+        attachment = record.public_send(@from)
+        unless attachment.respond_to?(:attached?) && attachment.attached?
+          result.skipped += 1
+          next
+        end
+
+        file = @adopt_from ? adopt(attachment.blob) : stream(attachment.blob)
+        # update_column: the file exists already, so has_hutch's check on save
+        # would only ask the server again, and callbacks have no business here.
+        record.update_column(column, file.id)
+        result.done += 1
+      rescue FileHutch::Error => e
+        result.failed += 1
+        @logger&.warn("[file_hutch] backfill #{@model.name}##{record.id} #{@from}: #{e.message}")
+      end
+
+      result
+    end
+
+    private
+
+    def column = @options.fetch(:column)
+    def policy = @options.fetch(:policy)
+
+    def adopt(blob)
+      @client.adopt(storage_connection: @adopt_from, key: blob.key, filename: blob.filename.to_s,
+                    content_type: blob.content_type, policy: policy)
+    end
+
+    def stream(blob)
+      blob.open do |io|
+        @client.upload(io, policy: policy, filename: blob.filename.to_s, content_type: blob.content_type)
+      end
+    end
+  end
+end

data/lib/file_hutch/client.rb

@@ -35,6 +35,15 @@ module FileHutch
       request(:get, "/api/v1/transforms").fetch("transforms").map { |t| Transform.new(t, client: self) }
     end
 
+    # Registers an object that is already in a connected bucket as a file, in
+    # place: nothing is copied and the object keeps its key. Adopting the same
+    # key again returns the same file, so a backfill can be re-run.
+    def adopt(key:, storage_connection:, filename: nil, content_type: nil, visibility: nil, policy: nil)
+      body = { storage_connection: storage_connection, key: key, filename: filename,
+               content_type: content_type, visibility: visibility, policy: policy }.compact
+      File.new(request(:post, "/api/v1/files/adopt", body).fetch("file"), client: self)
+    end
+
     # -- Declarative configuration (see `file_hutch plan|apply`) -----------
 
     # The project as a config hash: {"uploads" => {...}, "transforms" => {...}, "environments" => [...]}.

data/lib/file_hutch/rails/attachable.rb

@@ -3,11 +3,11 @@
 require "active_support/concern"
 
 module FileHutch
-  # `has_file_hutch_file` for Active Record. The model stores one string column,
+  # `has_hutch` for Active Record. The model stores one string column,
   # `<name>_file_id`, holding an FileHutch file id. Nothing about storage leaks in.
   #
   #   class User < ApplicationRecord
-  #     has_file_hutch_file :avatar, policy: "avatars"
+  #     has_hutch :avatar, policy: "avatars"
   #   end
   #
   #   user.avatar = params[:avatar]            # uploaded IO: uploaded to storage on save
@@ -25,7 +25,7 @@ module FileHutch
     extend ActiveSupport::Concern
 
     class_methods do
-      def has_file_hutch_file(name, policy:, column: "#{name}_file_id", dependent: :delete, verify: true)
+      def has_hutch(name, policy:, column: "#{name}_file_id", dependent: :delete, verify: true)
         include Attachable unless include?(Attachable)
         name = name.to_sym
         column = column.to_s
@@ -45,6 +45,7 @@ module FileHutch
         define_method(:"purge_#{name}") { file_hutch_purge(name) }
       end
 
+
       def file_hutch_files
         @file_hutch_files ||= superclass.respond_to?(:file_hutch_files) ? superclass.file_hutch_files.dup : {}
       end

data/lib/file_hutch/rails/engine.rb

@@ -13,6 +13,10 @@ module FileHutch
   class Engine < ::Rails::Engine
     isolate_namespace FileHutch
 
+    initializer "file_hutch.deprecator" do |app|
+      app.deprecators[:file_hutch] = FileHutch.deprecator if app.respond_to?(:deprecators)
+    end
+
     initializer "file_hutch.active_record" do
       ActiveSupport.on_load(:active_record) { extend FileHutch::Attachable::ClassMethods }
     end

data/lib/file_hutch/rails.rb

@@ -2,4 +2,10 @@
 
 require "rails"
 require_relative "rails/attachable"
+require_relative "backfill"
+
+module FileHutch
+  def self.deprecator = @deprecator ||= ActiveSupport::Deprecation.new("0.4", "file_hutch")
+end
+
 require_relative "rails/engine"

data/lib/file_hutch/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module FileHutch
-  VERSION = "0.1.0"
+  VERSION = "0.3.0"
 end

data/lib/generators/file_hutch/attachment/attachment_generator.rb

@@ -6,7 +6,7 @@ require "rails/generators/active_record"
 module FileHutch
   module Generators
     # bin/rails generate file_hutch:attachment User avatar
-    # Adds users.avatar_file_id (string). Pair with `has_file_hutch_file :avatar, policy: "…"`.
+    # Adds users.avatar_file_id (string). Pair with `has_hutch :avatar, policy: "…"`.
     class AttachmentGenerator < ::Rails::Generators::NamedBase
       include ::ActiveRecord::Generators::Migration
 
@@ -18,7 +18,7 @@ module FileHutch
       end
 
       def show_macro
-        say %(Add to #{class_name}:  has_file_hutch_file :#{attachment}, policy: "#{attachment.pluralize}"), :green
+        say %(Add to #{class_name}:  has_hutch :#{attachment}, policy: "#{attachment.pluralize}"), :green
       end
 
       private

data/lib/generators/file_hutch/install/install_generator.rb

@@ -25,7 +25,7 @@ module FileHutch
         say ""
         say "FileHutch installed.", :green
         say "  1. Set FILE_HUTCH_API_KEY (Dashboard → API keys) and, if not production, FILE_HUTCH_URL."
-        say "  2. Add a column and macro:  bin/rails g file_hutch:attachment User avatar  then  has_file_hutch_file :avatar, policy: \"avatars\""
+        say "  2. Add a column and macro:  bin/rails g file_hutch:attachment User avatar  then  has_hutch :avatar, policy: \"avatars\""
         say "  3. For browser uploads, set FileHutch.config.authorize_direct_upload in the initializer and register the"
         say "     Stimulus controller:  application.register(\"filehutch-direct-upload\", DirectUploadController)"
       end

data/lib/tasks/file_hutch.rake

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+namespace :file_hutch do
+  desc "Copy an Active Storage attachment onto a has_hutch column. " \
+       "MODEL=User FROM=photo [TO=avatar] [ADOPT_FROM=conn_… to register blobs in place instead of copying]"
+  task backfill: :environment do
+    model = ENV.fetch("MODEL") { abort "MODEL= is required, e.g. MODEL=User" }.constantize
+    from = ENV.fetch("FROM") { abort "FROM= is required: the Active Storage attachment name" }
+    to = ENV.fetch("TO", from)
+
+    result = FileHutch::Backfill.new(model: model, from: from, to: to, adopt_from: ENV["ADOPT_FROM"]).call
+    puts "#{model.name}##{to}: #{result.done} filled, #{result.skipped} with nothing attached, #{result.failed} failed"
+    puts "Run it again to retry the failures; filled rows are skipped." if result.failed.positive?
+  end
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: file_hutch
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Andy Leverenz
@@ -10,8 +10,8 @@ cert_chain: []
 date: 1980-01-02 00:00:00.000000000 Z
 dependencies: []
 description: Talk to the FileHutch file control plane from Ruby. Server-side and browser-direct
-  uploads, signed URLs, and a has_file_hutch_file macro for Active Record that stores
-  only opaque file ids.
+  uploads, signed URLs, and a has_hutch macro for Active Record that stores only opaque
+  file ids.
 email:
 - andy@justalever.com
 executables:
@@ -27,6 +27,7 @@ files:
 - config/routes.rb
 - exe/file_hutch
 - lib/file_hutch.rb
+- lib/file_hutch/backfill.rb
 - lib/file_hutch/cli.rb
 - lib/file_hutch/client.rb
 - lib/file_hutch/configuration.rb
@@ -44,6 +45,7 @@ files:
 - lib/generators/file_hutch/attachment/templates/migration.rb.tt
 - lib/generators/file_hutch/install/install_generator.rb
 - lib/generators/file_hutch/install/templates/initializer.rb
+- lib/tasks/file_hutch.rake
 homepage: https://github.com/filehutch/filehutch-ruby
 licenses:
 - MIT