activestorage-multi_node_disk 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: d354671f0910267b48241ec1225870aa5ac97a5b3fd1e484f4f8fe7523a4be43
+  data.tar.gz: ccbe3a767a0cc540727d421e4779bf8014194ea7746d14731552872f54a0f96d
+SHA512:
+  metadata.gz: 8bc81e5ecbbe8b81c8b0a9e1bee4c8112644e366fc0f68632d0a8ccf6cdf23745215b34374e202fcf9dd7cb7f107bce780ee7ba60a24da1a420bff4f4a5db507
+  data.tar.gz: 90c49f600de770e09b48ac0fe78bba4c6514a4cf8f72d018f6b2a223325b03ddb5651e14ad72635302cabd62357bf9e7ea932f810debea9e5e107d439367dee4

data/CHANGELOG.md

@@ -0,0 +1,43 @@
+# Changelog
+
+All notable changes to activestorage-multi_node_disk are listed here. The
+format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and
+the project uses [Semantic Versioning](https://semver.org/). Before 1.0, a
+minor version may change public APIs; each entry says how.
+
+## [Unreleased]
+
+## [0.1.0] - 2026-10-20
+
+The first release.
+
+### Added
+
+- `ActiveStorage::Service::MultiNodeDiskService` (`service: MultiNodeDisk`), a
+  Disk service for several nodes: uploads stay on the receiving node's disk;
+  reads, `exist?` and deletes ask the other nodes (`nodes:`) in parallel under
+  one deadline, and the first copy that matches the blob's size and checksum
+  wins. No migration, no metadata, no backfill.
+- A node endpoint (`/rails/active_storage/multi_node_disk/blobs/:key`, `GET`,
+  `HEAD`, `DELETE`) in an isolated engine, mounted under Active Storage's
+  routes prefix ahead of the app's routes (`draw_routes`). It serves its own
+  disk only, so a hop never fans out.
+- Request authentication: `auth` `:none` (default, with a boot warning in
+  production), `:token` (Bearer, `secure_compare`), `:ip` (the nodes' DNS
+  addresses with a 60-second cache, or IPs/CIDRs) and `[:ip, :token]`.
+- Errors that subclass Active Storage's: `NotHere`, `Unavailable`,
+  `Mismatch`; `error_mapper` for host classes.
+- Delete modes: `:sync` (raises `Unavailable` for a node that did not
+  confirm) and `:job` (`DeleteJob`, retried, idempotent; Active Job optional).
+- Audit: one versioned JSON document per answered request and per remote
+  fetch or delete, to a JSON Lines file (default), a table, nowhere or a
+  callable; `audit_requester`, `audit_context`, `with_requester` and the
+  requester header; `audit.multi_node_disk` events.
+- `bin/rails g active_storage:multi_node_disk:install [--audit]`: the
+  initializer and the audit table's migration (PostgreSQL, MySQL 8, Oracle,
+  SQLite; names within 30 characters; `table_name_prefix`).
+- `ActiveStorage::MultiNodeDisk::TestHelper`: fake nodes answered by the real
+  endpoint in process.
+
+[Unreleased]: https://github.com/clearstackio/activestorage-multi_node_disk/compare/v0.1.0...HEAD
+[0.1.0]: https://github.com/clearstackio/activestorage-multi_node_disk/releases/tag/v0.1.0

data/CONTRIBUTING.md

@@ -0,0 +1,89 @@
+# Contributing
+
+Thanks for working on activestorage-multi_node_disk. Read
+[AGENTS.md](AGENTS.md) first: it is the source of truth for the gem's scope,
+shape and rules, for people and coding agents alike.
+
+Open an issue before a large change, so we can agree on the shape first.
+Pull requests need a green CI and a CHANGELOG line for anything a host would
+notice.
+
+## Setup
+
+Tools are pinned in `mise.toml` (Ruby 4.0.7; the gem supports 3.3+).
+
+```sh
+bin/setup           # mise install, bundle install, rake
+```
+
+## Running the checks
+
+```sh
+bundle exec rake                       # specs, StandardRB, RBS syntax
+bundle exec rspec spec/requests        # one folder
+DATABASE_URL=postgres://user:pw@127.0.0.1/scratch bundle exec rspec   # on PostgreSQL
+DATABASE_URL=trilogy://root:pw@127.0.0.1:3306/scratch bundle exec rspec  # on MySQL 8
+```
+
+Without `DATABASE_URL` the dummy app (`spec/dummy`) uses SQLite. Use a scratch
+database only: the suite creates and drops tables.
+
+## Gem conventions
+
+Current RubyGems and Bundler practice, as applied here:
+
+- **gemspec:** name, version from `lib/active_storage/multi_node_disk/version.rb`,
+  summary, description, homepage, MIT, `required_ruby_version >= 3.3`;
+  metadata `homepage_uri`, `source_code_uri`, `changelog_uri`,
+  `bug_tracker_uri`, `documentation_uri`, `rubygems_mfa_required` and
+  `allowed_push_host` (rubygems.org); `spec.files` from `git ls-files`
+  without specs, sigs, `bin/` and dot files.
+- **Dependencies:** runtime only `activestorage`, `actionpack`, `railties`
+  (`>= 8.1, < 9`); development gems in the Gemfile. `Gemfile.lock` is
+  committed (Bundler's current guidance). Dependabot proposes updates weekly.
+- **Code:** `lib/activestorage-multi_node_disk.rb` (Bundler's auto-require)
+  requires `active_storage/multi_node_disk`, which sets up
+  `Zeitwerk::Loader.for_gem_extension(ActiveStorage)`; the service file is
+  left to Active Storage's Configurator. `# frozen_string_literal: true`
+  everywhere; one concern per file; a header comment on every class; YARD on
+  public methods. RBS signatures in `sig/` (`rake rbs` checks the syntax; a
+  full type check needs Rails signatures).
+- **Style:** StandardRB, targeting Ruby 3.3 (`.standard.yml`); no
+  `.rubocop.yml`.
+- **Versions:** SemVer from 0.1.0; [Keep a Changelog](CHANGELOG.md): add a
+  line under `[Unreleased]` for anything a host would notice.
+- **CI:** GitHub Actions (`.github/workflows/ci.yml`): `rake` on Ruby 3.3,
+  3.4 and 4.0, then the suite on PostgreSQL 17 and MySQL 8. Actions are
+  pinned to commit SHAs. Oracle is not run in CI: the audit migration is
+  rendered and its names checked by the suite.
+- **Security:** no secrets in the repo; see [SECURITY.md](SECURITY.md).
+
+## Releasing
+
+Releases go to rubygems.org from GitHub Actions with
+[trusted publishing](https://guides.rubygems.org/trusted-publishing/) (no
+API key) and a Sigstore attestation (`.github/workflows/release.yml`, in the
+`rubygems` environment).
+
+Before the first release, an owner of the gem adds a pending trusted
+publisher on rubygems.org (Profile → Trusted publishers): gem
+`activestorage-multi_node_disk`, repository owner `clearstackio`, repository
+`activestorage-multi_node_disk`, workflow `release.yml`, environment
+`rubygems`. After the first push it becomes the gem's trusted publisher.
+
+1. Update `VERSION` in `lib/active_storage/multi_node_disk/version.rb` and
+   run `bundle install` so `Gemfile.lock` follows.
+2. Move the `[Unreleased]` entries in `CHANGELOG.md` under the new version
+   and date, and update the compare links at the bottom.
+3. Commit, then tag and push: `git tag v0.1.0 && git push origin master v0.1.0`.
+
+The workflow checks the tag matches `VERSION`, runs `rake`, then builds,
+pushes and attests the gem.
+
+## Tests
+
+Every behaviour gets a spec: the service against fake nodes
+(`spec/active_storage/service`), the endpoint and routing (`spec/requests`),
+configuration, audit and the generator (`spec/active_storage/multi_node_disk`,
+`spec/generators`). Specs test behaviour, not layout. Multi-node behaviour
+uses `ActiveStorage::MultiNodeDisk::TestHelper`, never the network.

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Ajaya Agrawalla
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,249 @@
+# activestorage-multi_node_disk
+
+An Active Storage Disk service for Rails apps that run on several nodes, each with its own disk.
+
+A file stays on the disk of the node that received it. Any node can read, check or delete it: the service looks on its own disk first, then asks the other nodes and keeps the first copy that matches the blob. Zero migration by default: nothing is recorded, so existing blobs work as they are.
+
+[![Build Status](https://github.com/clearstackio/activestorage-multi_node_disk/actions/workflows/ci.yml/badge.svg)](https://github.com/clearstackio/activestorage-multi_node_disk/actions)
+
+## Installation
+
+Add this line to your application’s Gemfile:
+
+```ruby
+gem "activestorage-multi_node_disk"
+```
+
+## Getting Started
+
+Replace `service: Disk` with `service: MultiNodeDisk` and list the nodes:
+
+```yml
+local:
+  service: MultiNodeDisk
+  root: <%= Rails.root.join("storage") %>
+  nodes: [app1.internal, app2.internal, app3.internal]
+```
+
+Give the nodes a shared token and turn on token checks:
+
+```sh
+bin/rails credentials:edit   # multi_node_disk: { token: <the same random value on every node> }
+```
+
+```ruby
+# config/initializers/active_storage_multi_node_disk.rb
+ActiveStorage::MultiNodeDisk.configure do |config|
+  config.auth = :token
+end
+```
+
+That’s it. `bin/rails g active_storage:multi_node_disk:install` writes that initializer for you.
+
+> **Warning:** the endpoint checks nothing by default (`auth = :none`): anyone who can reach a node can read and delete its files. Use `:token` (or `[:ip, :token]`) anywhere but a closed network. The gem logs a warning at boot in production while it is `:none`.
+
+## How It Works
+
+- **Upload** - written to this node’s disk, exactly as Disk
+- **Read** (`download`, `download_chunk`, `open`) - this disk first; else every other node is asked at once, under one deadline (`open_timeout + read_timeout`), and the first copy whose size and checksum match the blob wins. A copy that does not match is skipped
+- **`exist?`** - this disk, else whether any node has the file (presence, as Disk)
+- **Delete** - this disk, then every other node; a 404 counts as done
+- **The endpoint** - each node answers other nodes at `/rails/active_storage/multi_node_disk/blobs/:key` (`GET`, `HEAD`, `DELETE`), from its own disk only. A request that arrives from another node never fans out again
+
+A whole-file `download` is also checked against the blob’s size and checksum when the file is local, and `open` keeps Rails’ own check.
+
+Local only, as Disk: `url` (use Active Storage’s proxy mode, which reads through `download`), `compose` and `delete_prefixed` (variants, which any node can regenerate). Keys with no blob row (variants) are never looked for elsewhere.
+
+### Cost
+
+A local hit costs a `File.exist?` (plus one indexed blob lookup for a whole-file `download`). A miss on N nodes sends N - 1 requests, at most 8 at a time per call; only the node with the file sends it, the others answer 404 after a `File.exist?`. A delete always sends N - 1 requests.
+
+## Errors
+
+The service raises subclasses of Active Storage’s own errors, so Rails keeps handling them:
+
+Error | Superclass | When
+--- | --- | ---
+`ActiveStorage::MultiNodeDisk::NotHere` | `ActiveStorage::FileNotFoundError` | every node answered "not here"
+`ActiveStorage::MultiNodeDisk::Unavailable` | `ActiveStorage::Error` | a node that may have it did not answer (retry)
+`ActiveStorage::MultiNodeDisk::Mismatch` | `ActiveStorage::IntegrityError` | the only copies found do not match the blob
+
+Each has `nodes`, `key` and `filename`. When no copy wins: any node failed → `Unavailable`; else a wrong copy → `Mismatch`; else `NotHere`.
+
+## Deletes
+
+`delete_mode` decides how a delete reaches the other nodes; the local copy is always deleted at once.
+
+- `:sync` (default) - inline, after which `Unavailable` names any node that did not confirm (as other services raise on a failed delete). Rails’ `purge` removes the rows first, so the rows are gone either way
+- `:job` - `ActiveStorage::MultiNodeDisk::DeleteJob`, retried while a node does not confirm, idempotent. Needs Active Job (checked at boot)
+
+Rails’ `purge_later` already runs `purge` in `ActiveStorage::PurgeJob`; with `:job`, that job deletes the local copy and queues `DeleteJob` for the rest.
+
+```ruby
+config.delete_mode = :job
+config.delete_job_queue = :maintenance        # default: Active Storage's purge queue
+config.delete_job_class = "FileDeleteJob"     # a subclass, for other retries
+```
+
+## Request Authentication
+
+`auth` | The endpoint needs
+--- | ---
+`:none` (default) | nothing
+`:token` | `Authorization: Bearer <token>`; else 401 with `WWW-Authenticate: Bearer realm="Multi-node disk"`
+`:ip` | the caller in `allowed_ips`; else 403
+`[:ip, :token]` | both (the address is checked first)
+
+Outgoing calls send the Bearer token only under `:token`.
+
+`allowed_ips` defaults to every node’s addresses, resolved by DNS and resolved again every 60 seconds (a node that fails to resolve keeps its last addresses). Set it to IPs or CIDRs to override. The caller is the socket’s peer; `X-Forwarded-For` counts only when that peer is listed in `config.action_dispatch.trusted_proxies`, so set it to your proxies when one sits in front of the app.
+
+The gem checks who is calling, never who may see a file: that is your app’s business, on its own pages.
+
+## Audit
+
+Every answered request (served, refused, deleted) and every remote fetch or delete is one JSON document:
+
+```json
+{"v":1,"at":"2026-10-17T12:00:00.000Z","action":"read","result":"served","status":200,"key":"abc","filename":"members.csv",
+ "byte_size":22,"node":"app2","from_node":"app1","remote_ip":"10.0.0.1","auth":["token"],"identity":"node","hop":true,
+ "duration_ms":3.1,"requester":{"id":42,"label":"hr@example.com","type":"hr_admin"},"requester_verified":true,"context":{"source_id":7}}
+```
+
+`action` is `read`, `head`, `delete`, `fetch` or `remote_delete`; `result` is `served`, `deleted`, `not_found`, `mismatch`, `unauthorized`, `forbidden` or `error`. It never holds a token, file contents or a signed URL. The same document is instrumented as `audit.multi_node_disk`.
+
+`audit` | Where
+--- | ---
+`:file` (default) | a JSON line appended to `audit_path` (`tmp/multi_node_disk/audit.log`), whole lines across threads and processes. Rotate it with logrotate (`copytruncate`), or rename it: the next write starts a new file
+`:table` | a row in `multi_node_disk_audits` (`id`, `created_at`, `payload`); run `bin/rails g active_storage:multi_node_disk:install --audit` and migrate
+`:none` | nowhere
+a callable | `->(entry) { ... }`
+
+A sink that fails is logged and never fails the request.
+
+### Who it is for
+
+```ruby
+config.audit_requester = ->(request) { {id: Current.user&.id, label: Current.user&.email, type: "staff"} }
+config.audit_context = ->(request, entry) { {record: Upload.find_by_blob_key(entry[:key])&.id} }
+```
+
+Work that serves no request (a job) names the requester for a block:
+
+```ruby
+ActiveStorage::MultiNodeDisk.with_requester(id: user.id, label: user.email) do
+  blob.open { |file| import(file) }
+end
+```
+
+The requester is sent to the node that answers (`X-Multi-Node-Disk-Requester`, JSON, at most 1 KB; larger is dropped) and recorded there; that node marks it `requester_verified: true` only when a check (`:token` or `:ip`) passed. Never put a secret in it.
+
+### The audit table
+
+The generator writes a migration named `CreateActiveStorageMultiNodeDiskAudits`; nothing runs on its own. It is safe to run twice and reversible.
+
+Database | `payload` | Tested
+--- | --- | ---
+PostgreSQL | `jsonb` | suite
+MySQL 8 | `json` | suite
+SQLite | `text` | suite
+Oracle | CLOB (`text`) | migration rendered and names checked; not run
+
+The only index is on `created_at` (`i_mnd_audits_created_at`), for pruning. Every name fits Oracle’s 30 characters. `table_name_prefix` (default `multi_node_disk_`) changes the table and index names.
+
+## Testing Your App
+
+```ruby
+RSpec.configure { |config| config.include ActiveStorage::MultiNodeDisk::TestHelper }
+
+it "reads a file another node holds" do
+  node_b = multi_node_disk_node("app2")
+  multi_node_disk_move(blob, to: node_b)
+
+  multi_node_disk_cluster("app2" => node_b, "app3" => :missing) do |requests|
+    expect(blob.download).to eq(bytes)
+  end
+end
+```
+
+A node in the cluster is a service (answered by the gem’s real endpoint, in process), `:missing`, `:wrong`, `:timeout` or `:down`. `requests` lists method, node, URL and header names, never values.
+
+## Advanced
+
+Set hooks once, in an initializer. A hook is a value or anything that responds to `call`. The configuration is validated and frozen after boot.
+
+Option | Default | Notes
+--- | --- | ---
+`nodes` | `nodes:` in storage.yml | `Array` or `-> Array`; this node may be in it
+`current_node` | `Socket.gethostname` | `node:` in storage.yml wins
+`url_for` | `->(node, key) { "https://#{node}/rails/active_storage/multi_node_disk/blobs/#{key}" }` | must return `https:`
+`auth` | `:none` | per service in storage.yml too
+`allowed_ips` | nodes’ DNS addresses | per service in storage.yml too
+`token` | credentials `multi_node_disk.token` | what `:token` compares against
+`token_for` | `->(node, blob, action) { token }` | the token sent; `blob` is nil for a delete after the rows went
+`authenticate` | `->(token, request) { :node if token matches }` | returns an identity label, or nil to refuse
+`http_client` | `ActiveStorage::MultiNodeDisk::NetHttpClient` | `call(method, url, headers:, node:, open_timeout:, read_timeout:)` → `Response(status, body, headers)`; called from several threads at once
+`open_timeout`, `read_timeout` | 3, 20 | seconds; per service in storage.yml too
+`hop_header` | `X-Multi-Node-Disk-Hop` | carries the calling node’s name
+`requester_header` | `X-Multi-Node-Disk-Requester` |
+`error_mapper` | `->(error) { error }` | raise your own classes (subclass the gem’s)
+`delete_mode`, `delete_job_queue`, `delete_job_class` | `:sync`, purge queue, `DeleteJob` |
+`audit`, `audit_path`, `audit_context`, `audit_requester` | `:file`, `tmp/multi_node_disk/audit.log`, nil, nil | `audit_context(request or nil, entry)`, `audit_requester(request or nil)`
+`table_name_prefix` | `multi_node_disk_` |
+`draw_routes` | `true` | mounts the endpoint under Active Storage’s routes prefix, ahead of your routes; independent of `config.active_storage.draw_routes`
+
+Events (`ActiveSupport::Notifications`): `fetch`, `exist`, `delete`, `mismatch`, `serve`, `denied`, `audit` and `delete_abandoned`, each `.multi_node_disk`, plus Active Storage’s own `service_*.active_storage`.
+
+### Security
+
+- Keep the token in credentials or your secrets store; `filter_parameters` does not cover headers, so keep `Authorization` out of request logs and APM
+- Under `:token`, `url_for` must point at HTTPS (checked before a token is sent)
+- See [SECURITY.md](SECURITY.md)
+
+### A full example
+
+The host this gem was written for runs three nodes behind per-node proxies, with its own HTTP client, named API tokens and audit table:
+
+```ruby
+ActiveStorage::MultiNodeDisk.configure do |config|
+  config.auth = :token
+  config.url_for = ->(node, key) { "https://#{App.public_host}#{App.prefix}#{ActiveStorage::MultiNodeDisk.mount_path}/blobs/#{key}" }
+  config.token_for = ->(node, blob, action) { NodeFiles.token_for(blob) }
+  config.authenticate = ->(token, request) { NodeFiles.caller_for(token) }
+  config.http_client = NodeFiles::Client.new   # HTTPX, pinned to each node's IPv4 addresses
+  config.hop_header = "X-App-File-Hop"
+  config.error_mapper = ->(error) { NodeFiles.app_error(error) }
+  config.delete_mode = :job
+  config.delete_job_queue = :maintenance
+  config.audit = :table
+  config.audit_requester = ->(request) { NodeFiles.requester(request) }
+  config.audit_context = ->(request, entry) { NodeFiles.context(entry) }
+end
+```
+
+## Requirements
+
+Ruby | Rails | Databases (audit table)
+--- | --- | ---
+3.3+ | 8.1 | PostgreSQL, MySQL 8, Oracle, SQLite
+
+No runtime dependencies beyond Rails.
+
+## History
+
+View the [changelog](CHANGELOG.md)
+
+## Contributing
+
+Everyone is encouraged to help improve this project. Here are a few ways you can help:
+
+- [Report bugs](https://github.com/clearstackio/activestorage-multi_node_disk/issues)
+- Fix bugs and [submit pull requests](https://github.com/clearstackio/activestorage-multi_node_disk/pulls)
+- Write, clarify, or fix documentation
+- Suggest or add new features
+
+To get started with development, see [CONTRIBUTING.md](CONTRIBUTING.md). Report security issues privately, as [SECURITY.md](SECURITY.md) explains.
+
+## License
+
+MIT. See [LICENSE](LICENSE).

data/SECURITY.md

@@ -0,0 +1,33 @@
+# Security policy
+
+## Reporting a vulnerability
+
+Report it privately through
+[GitHub's private vulnerability reporting](https://github.com/clearstackio/activestorage-multi_node_disk/security/advisories/new),
+or email ajaya@clearstack.io. Please do not open a public issue.
+
+We aim to acknowledge a report within three working days, and to publish a
+fix and a GitHub security advisory once it is confirmed.
+
+Include the activestorage-multi_node_disk version (or commit), how to
+reproduce the problem, and what an attacker could read or delete.
+
+## Supported versions
+
+activestorage-multi_node_disk is pre-1.0. Only the latest release is
+supported.
+
+## The model
+
+- The node endpoint checks **who is calling** (`auth`), never who may see a
+  file: that stays with the host app. The default, `:none`, checks nothing;
+  use `:token` (or `[:ip, :token]`) anywhere but a closed network.
+- Tokens are compared in constant time, never logged and never part of an
+  event or audit entry. Under `:token`, node URLs must be HTTPS.
+- `:ip` trusts `X-Forwarded-For` only from proxies the app lists in
+  `config.action_dispatch.trusted_proxies`; otherwise the socket's peer counts.
+- A requester sent by another node is recorded as verified only after a check
+  passed. It must never carry a secret.
+- Releases are published to rubygems.org from GitHub Actions with trusted
+  publishing (no long-lived API key) and carry a Sigstore attestation. Any
+  other push needs MFA (`rubygems_mfa_required`).

data/app/controllers/active_storage/multi_node_disk/blobs_controller.rb

@@ -0,0 +1,166 @@
+# frozen_string_literal: true
+
+module ActiveStorage
+  module MultiNodeDisk
+    # One node's endpoint for the others: a file from this node's disk only
+    # (never asking further, so a hop cannot fan out), or its deletion.
+    #
+    # 1. The service's auth modes: :none checks nothing; :ip needs
+    #    request.remote_ip in the allowlist (else 403); :token needs a Bearer
+    #    token that config.authenticate turns into an identity (else 401 with
+    #    a Bearer challenge). [:ip, :token] needs both, IP first.
+    # 2. Answer from this disk; every response names this node. Request
+    #    auth only: who may see which file is the app's business, on its own
+    #    pages.
+    # 3. Audit every answer (AuditLog), with the requester another node sent
+    #    (trusted only once a check passed; unverified under :none).
+    class BlobsController < ActionController::API
+      include ActionController::HttpAuthentication::Token::ControllerMethods
+
+      REALM = "Multi-node disk"
+      BEARER = /\ABearer\s+\S/
+
+      before_action :name_this_node
+      before_action :authenticate_caller!
+      before_action :set_blob
+
+      # GET (the file) and HEAD (whether it is here).
+      def show
+        return not_here unless service&.local?(key)
+        return answer { head :ok } if request.head?
+
+        bytes = File.binread(service.path_for(key))
+        answer { send_data bytes, filename: @blob&.filename.to_s.presence || key, type: "application/octet-stream" }
+      rescue Errno::ENOENT, ActiveStorage::InvalidKeyError
+        not_here
+      end
+
+      def destroy
+        return not_here unless service
+
+        begin
+          File.delete(service.path_for(key))
+        rescue Errno::ENOENT
+          # Not here, or already deleted: nothing left on this node.
+        end
+        answer { head :no_content }
+      rescue ActiveStorage::InvalidKeyError
+        not_here
+      end
+
+      private
+
+      def key = params[:key].to_s
+
+      def name_this_node
+        @started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        response.headers[NODE_HEADER] = this_node
+      end
+
+      def auth_modes
+        @auth_modes ||= endpoint_service&.auth_modes || MultiNodeDisk::Configuration.auth_modes(MultiNodeDisk.config.auth)
+      end
+
+      def authenticate_caller!
+        modes = auth_modes
+        return deny(:forbidden) if modes.include?(:ip) && !allowed_ip?
+        return @checked = modes != [:none] unless modes.include?(:token)
+
+        if BEARER.match?(request.authorization.to_s)
+          @identity = authenticate_with_http_token { |token, _options| MultiNodeDisk.config.authenticate.call(token, request) }
+        end
+        return @checked = true if @identity
+
+        response.headers["WWW-Authenticate"] = %(Bearer realm="#{REALM}")
+        deny(:unauthorized)
+      end
+
+      # The socket's peer. Only when that peer is one of the proxies the app
+      # names in config.action_dispatch.trusted_proxies does request.remote_ip
+      # (X-Forwarded-For) count: Rails' own remote_ip would otherwise take a
+      # forwarded address any caller can write.
+      def allowed_ip?
+        caller_ip = IPAddr.new(caller_address.to_s)
+        caller_ip = caller_ip.native if caller_ip.ipv4_mapped?
+        endpoint_service.allowed_ips.any? { |allowed| allowed.include?(caller_ip) }
+      rescue IPAddr::Error
+        false
+      end
+
+      def caller_address
+        proxies = Rails.application.config.action_dispatch.trusted_proxies
+        peer = request.remote_addr.to_s
+        return peer if proxies.blank?
+
+        (Array(proxies).any? { |proxy| IPAddr.new(proxy.to_s).include?(peer) }) ? request.remote_ip : peer
+      rescue IPAddr::Error
+        peer
+      end
+
+      def set_blob
+        @blob = ActiveStorage::Blob.find_by(key: key)
+      end
+
+      def action = request.delete? ? :delete : :read
+
+      def audit_action
+        return :delete if request.delete?
+
+        request.head? ? :head : :read
+      end
+
+      # This node's MultiNodeDisk service (an in-process test may name
+      # another); nil for a blob of a different service.
+      def service
+        return @service if defined?(@service)
+
+        found = endpoint_service
+        @service = found if found && (@blob.nil? || @blob.service_name == found.name.to_s)
+      end
+
+      def endpoint_service
+        found = request.env[SERVICE_ENV_KEY] || MultiNodeDisk.service
+        found if found.is_a?(ActiveStorage::Service::MultiNodeDiskService)
+      end
+
+      def this_node
+        endpoint_service&.node || MultiNodeDisk.config.resolve(:current_node).to_s
+      end
+
+      def not_here
+        answer(:not_found) { render json: {message: "not here", node: this_node}, status: :not_found }
+      end
+
+      def answer(result = nil)
+        yield
+        response.headers[NODE_HEADER] = this_node
+        MultiNodeDisk.instrument(:serve, key: key, node: this_node, action: action, identity: @identity, status: response.status)
+        audit(result || (request.delete? ? :deleted : :served))
+      end
+
+      def deny(status)
+        MultiNodeDisk.instrument(:denied, key: key, node: this_node, action: action, status: Rack::Utils.status_code(status))
+        head status
+        audit(status)
+      end
+
+      def audit(result)
+        hop = request.headers[MultiNodeDisk.config.hop_header].presence
+        sent = MultiNodeDisk::Requester.decode(request.headers[MultiNodeDisk.config.requester_header])
+        trusted = @checked == true
+        entry = {
+          action: audit_action, result: result, status: response.status, key: key,
+          filename: @blob&.filename&.to_s, byte_size: @blob&.byte_size,
+          node: this_node, from_node: hop, remote_ip: caller_address, auth: auth_modes,
+          identity: @identity&.to_s, hop: !hop.nil?, duration_ms: duration_ms,
+          requester_verified: (trusted if sent)
+        }
+        MultiNodeDisk::AuditLog.record(entry, request: request, requester: sent || MultiNodeDisk.current_requester)
+      end
+
+      def duration_ms
+        ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - (@started || 0)) * 1000).round(1)
+      end
+    end
+  end
+end

data/config/routes.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+# The engine's own routes, under its mount path (Engine). GET also answers
+# HEAD. Nothing at the engine's root.
+ActiveStorage::MultiNodeDisk::Engine.routes.draw do
+  scope format: false, constraints: {key: %r{[^/]+}} do
+    get "blobs/:key" => "blobs#show", :as => :blob
+    delete "blobs/:key" => "blobs#destroy"
+  end
+end

data/lib/active_storage/multi_node_disk/audit.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+module ActiveStorage
+  module MultiNodeDisk
+    # The :table audit sink: one row per entry, the whole entry in payload
+    # (JSON on PostgreSQL, a CLOB on Oracle). The table comes from the
+    # host's migration (`bin/rails g active_storage:multi_node_disk:install
+    # --audit`); the gem never creates it.
+    class Audit < ActiveRecord::Base
+      self.table_name = "#{MultiNodeDisk.config.table_name_prefix}audits"
+
+      attribute :payload, :json
+    end
+  end
+end