mailbox-kit 0.1.0 → 0.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 186fcaea489075063aeb25ee886080ff243289eb3ec02fdfac110f9c7e5c9081
-  data.tar.gz: 81071f06cf54c2de754bf16a056e6be93ae7dcbbca6604c3636a8e3ed19e3169
+  metadata.gz: 1b8650dea4b54dbd598376075edb422437abd1f64fb88aaae1b1d342f0bdd35d
+  data.tar.gz: 96f6170e145f22e14469cec036bdd4858ed10c2f518429fa72ef866fe197a345
 SHA512:
-  metadata.gz: 6df3f4be7cd2b1b07eebbd164a609c4fa488572b2bed5c65312e1363e4c21c04312ad11d3b6cf73cf636ff0893dc7d4b066a31f7f842beb1d1a829819db783ef
-  data.tar.gz: ba69a55126f0a291dc21296230532c4c575a183d45f49c37dbeeb4295762847a43c47b7fa091c8e835b2b0cd3bdbf87fd697baebd9244ff52345d46795997df5
+  metadata.gz: 1f29ae48e734ffbdfb5ef3074188c9189eb666eecc84644094439c698efe507753d0af829b41f2df7b7ee399f1cec50ed164bfc4e236bcfe484e02cac847948a
+  data.tar.gz: fbb0c980266f89fa4e96550de66982afa323a8958d3cfce1c4b950d0248385dc07144f0d52c16c98355f35ee367479add6de70351766e179a33100698dcf4043

data/README.md

@@ -7,22 +7,14 @@ and access control. SQLite works out of the box; database tenancy is opt-in.
 Rails already stores raw email, parses it, routes it to handlers, and runs
 processing jobs. Mailbox Kit adds the inbox your users interact with.
 
-## Choose your setup
+## Install
 
-- **Cloudflare receiving or sending:** follow the [Cloudflare Rails guide](https://github.com/cole-robertson/cloudflare-email/blob/main/docs/getting-started.md).
-  It includes this core; use the Cloudflare installer.
-- **An existing Action Mailbox integration:** install this core and attach received
-  email to inboxes. Follow the example below.
-- **Only processing incoming email:** Action Mailbox alone may be enough.
-
-Add Mailbox Kit 0.1 to your Gemfile:
+For a Rails app using another inbound provider:
 
 ```ruby
 gem "mailbox-kit", "~> 0.1.0"
 ```
 
-For a new Rails 7.2–8.1 application:
-
 ```sh
 bundle install
 bin/rails action_mailbox:install
@@ -30,8 +22,10 @@ bin/rails generate mailbox_kit:install
 bin/rails db:migrate
 ```
 
-Already using Cloudflare mailboxes? Follow the [upgrade guide](docs/upgrading.md)
-instead of creating the tables again.
+Skip `action_mailbox:install` if Action Mailbox is already configured.
+
+For Cloudflare, use the [Cloudflare mailbox guide](https://github.com/cole-robertson/cloudflare-email/blob/main/docs/mailboxes.md)
+instead. Cloudflare Email installs Mailbox Kit automatically.
 
 ## Create an inbox in code
 
@@ -81,24 +75,11 @@ The optional management engine provides a server-rendered UI with no frontend
 build. Your application supplies authentication and authorizes each action;
 access is denied until configured. See [management setup](docs/integration.md#management-interface).
 
-## The split
-
-| Layer | Owns |
-| --- | --- |
-| Rails | Raw MIME, Active Storage, parsing, routing, processing jobs/status, cleanup policy, Action Mailer |
-| Mailbox Kit | Inbox identities, addresses/aliases, memberships, read/archive state, selective retention, optional tenant context, management UI |
-| Cloudflare | Request verification, authoritative envelope metadata, Worker retries/storage, sending, delivery feedback, provisioning |
-| Your app | Users/organizations/sites, ownership lifecycle, authorization, sender acceptance, business effects |
-
-Inbound and outbound can use different providers through Rails. The core does
-not send email or supply a universal outbound adapter registry. `owner_ref` is an
-application-managed reference; it is not an automatic `has_mailbox` association
-or permission grant.
+## More features
 
-Inbox-associated messages survive Rails' automatic incineration. For retaining
-all inbound messages, Rails already offers `config.action_mailbox.incinerate = false`.
-Raw storage and processing remain Rails responsibilities.
+Use the [integration guide](docs/integration.md) for aliases, catch-all addresses,
+retention, authentication, and separate tenant databases.
 
-See the [integration guide](docs/integration.md) for aliases, catch-all addresses,
-verified source ingestion, duplicates, retention, authorization and optional
-database tenancy. See [upgrading](docs/upgrading.md) for existing installations.
+Your app supplies users, organizations, sender policy, and mailbox permissions.
+`owner_ref` stores your reference to an application record; it does not grant access.
+Inbound and outbound email can use different providers through Rails.

data/docs/integration.md

@@ -6,29 +6,9 @@ configurable retention. Mailbox Kit adds inbox identities, addresses/aliases,
 membership, read/archive state, scoped access and a server-rendered management UI.
 SQLite works; separate tenant databases are opt-in.
 
-This package is maintained alongside `cloudflare-email` so changes to the core and
-its first integration can be tested together. It is not yet published. In this
-checkout use `gem "mailbox-kit", path: "mailbox-kit"`. Once released, applications
-can use `gem "mailbox-kit", "~> 0.1"`.
-
-## What belongs where?
-
-| Layer | Responsibility |
-| --- | --- |
-| Mailbox Kit | Inbox identities, addresses, recipient lookup, membership, read/archive/purge, selective retention, tenant context, management UI |
-| Rails | InboundEmail records, original MIME/ActiveStorage, parsing, processing callbacks/status, routing jobs, configurable retention, and ActionMailer |
-| Provider integration | Verify incoming requests and envelope recipients, deliver outgoing messages, authenticate delivery feedback, configure DNS/routes |
-| Your application | Users, organizations, sites, permissions, sender acceptance and business workflows |
-
-Mailbox Kit does not require Cloudflare, configure DNS, or send messages by itself.
-`cloudflare-email` supplies its existing Worker ingress, sending/outbox, feedback,
-and provisioning integration. Other providers can call the core receiving APIs;
-this release does not claim complete SES, Postmark, or generic outbound adapters.
-The tests exercise Rails' stock Postmark HTTP ingress followed by explicit kit
-attachment in one database; provider setup, dynamic envelope authorization and
-provider outage behavior are separate integration responsibilities.
-Inbound and outbound need not use the same service. A mailbox has no `provider`
-attribute: receiving through one provider does not authorize sending through it.
+Install `gem "mailbox-kit", "~> 0.1.0"`, or use the
+[Cloudflare mailbox setup](https://github.com/cole-robertson/cloudflare-email/blob/main/docs/mailboxes.md)
+when Cloudflare is your provider.
 
 ## Create an inbox
 
@@ -63,20 +43,13 @@ end
 tenancy. Domain/address activation records evidence supplied by your trusted
 application. It does not verify DNS or grant permission to send from that address.
 
-`owner_ref` is currently an application-managed reference, not a foreign key or an
-authorization grant. Use stable identities, rather than reusable numeric IDs.
-Automatic `has_mailbox` owner bindings and deletion reconciliation are a separate
-follow-up; this extraction does not introduce a callback that could silently
-reassign an old address to a new owner.
+`owner_ref` stores your application's reference to a user, team, or organization.
+Use a stable identity and check access in your application.
 
-## Reuse Action Mailbox
+## Attach an Action Mailbox record
 
-For a receive-and-process application, Action Mailbox alone may be enough. It
-already supports dynamic handlers through regex/callable routes, processing
-callbacks, test helpers, and the development conductor at
-`/rails/conductor/action_mailbox/inbound_emails`. The kit does not replace those.
-Rails' `delivered` status means an inbound handler finished processing; read and
-archive state belong to the kit's inbox membership instead.
+Use ordinary `ApplicationMailbox` handlers for processing. Rails' `delivered`
+status means processing finished; read and archive state belong to the inbox.
 
 If an existing Rails ingress has already stored an email, attach that record:
 
@@ -127,7 +100,7 @@ The adapter remains responsible for authentication, authoritative envelope
 recipients, delivery-specific identity where necessary, and retrying failed
 requests. ActionMailbox still owns processing and its `ApplicationMailbox` routes.
 
-For application checks before persistence, the existing block form remains:
+To check application policy before saving, use a block:
 
 ```ruby
 require "mailbox_kit/inbound_email"
@@ -140,12 +113,11 @@ end
 Blocks must return the existing Rails record on duplicates if membership should
 be attached. Rails' bare `create_and_extract_message_id!` returns `nil` for a
 duplicate, so using it directly in that block can omit a second inbox membership.
-Returning `nil` intentionally skips membership, preserving the older block API.
+Returning `nil` skips membership.
 Neither a database transaction nor source deduplication guarantees exactly-once
 business effects, external blob cleanup on rollback, or recovery of lost jobs.
 
-With Cloudflare, the adapter preserves its existing authenticated envelope and
-metadata-based delivery identity, then uses the same Rails persistence bridge:
+A custom Cloudflare endpoint can verify and receive in two steps:
 
 ```ruby
 verified = Cloudflare::Email::Ingress.verify(
@@ -158,8 +130,8 @@ verified.message.receive_into_mailbox! if verified.status == :ok
 The shipped Cloudflare ingress controller already does this, including HTTP error
 handling and size limits. Installing the core does not add another HTTP endpoint
 or SMTP server. `persist_action_mailbox!` remains available for applications that
-only need Rails storage; its existing return convention is new record or `nil`
-on duplicate. The new inbox bridge can repair membership on duplicate delivery
+only need Rails storage; it returns the new record or `nil`
+on duplicate. The inbox bridge can repair membership on duplicate delivery
 without rerouting the email or replacing stored authentication metadata.
 
 For an application with its own persistence, use
@@ -254,30 +226,9 @@ Never choose the tenant from request parameters. Rails framework jobs preserve t
 explicit context; your own tenant jobs can `prepend MailboxKit::TenantJobContext`
 after requiring `mailbox_kit/tenant_job_context`.
 
-## Existing Cloudflare Email applications
-
-Keep your existing requires, initializer, migrations, Worker and mounted engine.
-Updating the Cloudflare gem brings in this core as a dependency. Public Cloudflare
-constants resolve to the same core models; tables, IDs and tenant job payload keys
-are retained. The `cloudflare_email_` table prefix is intentionally unchanged.
-Do **not** run `mailbox_kit:install` over an existing Cloudflare mailbox schema.
-
-For a fresh Cloudflare application, continue using the Cloudflare mailbox generator,
-which also installs its outbox and event tables. A core-only installation should
-not run both installers; adding Cloudflare outbound later needs its additional
-tables and explicit sending-account/domain configuration. The receiving-only core
-does not automatically become a Cloudflare sending account when the adapter loads.
-
 ## Sending from customer subdomains
 
 Cloudflare onboards each sending domain/subdomain separately. Parent-domain
 verification and wildcard receiving do not authorize arbitrary From subdomains.
 Use an explicitly verified sending domain and, where useful, a customer-specific
 inbound Reply-To. See [Cloudflare's subdomain rules](https://developers.cloudflare.com/email-service/configuration/subdomains/).
-
-## Development and release
-
-From the repository root, `bundle exec rake test` exercises both the compatibility
-API and standalone core. `bundle exec ruby script/verify_package.rb` builds both
-archives, installs a Rails-free consumer, and tests the packaged Rails integrations.
-Publish `mailbox-kit` before releasing a Cloudflare version that depends on it.

data/lib/mailbox_kit/active_record/base.rb

@@ -3,12 +3,31 @@ require "mailbox_kit/tenancy"
 
 module MailboxKit
   module ActiveRecord
-    # Rails may inspect every model's pool while preloading schema metadata
-    # before any request has selected a tenant. Report an unavailable
-    # connection using Rails' error hierarchy so boot can recover, while
-    # retaining the same fail-closed guard for every caller.
     class TenantConnectionUnavailable < ::ActiveRecord::ConnectionNotEstablished; end
 
+    # Returned instead of a pool while no valid tenant context is selected.
+    # Rails' production boot asks every model's pool for its schema reflection
+    # before any tenant exists (railtie "active_record.define_attribute_methods");
+    # raising there aborts that loop for every model still to come. This answers
+    # only that probe, with an empty reflection, and raises for anything else.
+    class UnavailablePool
+      def initialize(error)
+        @error = error
+      end
+
+      def schema_reflection
+        ::ActiveRecord::ConnectionAdapters::SchemaReflection.new(nil)
+      end
+
+      def method_missing(*)
+        raise TenantConnectionUnavailable, @error.message, cause: @error
+      end
+
+      def respond_to_missing?(*)
+        false
+      end
+    end
+
     # Uses the host's abstract tenant connection owner when explicitly configured.
     class Base < Tenancy.model_base(::ActiveRecord::Base)
       self.abstract_class = true
@@ -19,7 +38,7 @@ module MailboxKit
             begin
               Tenancy.require_context!
             rescue ConfigurationError => error
-              raise TenantConnectionUnavailable, error.message
+              return UnavailablePool.new(error)
             end
           end
           super

data/lib/mailbox_kit/inbound_email.rb

@@ -1,15 +1,30 @@
 require "mailbox-kit"
 require "digest"
+require "stringio"
 
 module MailboxKit
   # A small bridge to Rails' existing storage and routing lifecycle. It returns
   # the existing record on duplicate delivery so callers can attach memberships.
+  #
+  # The raw source is uploaded BEFORE any transaction this module opens. A
+  # database transaction can hold a write lock (SQLite's from BEGIN when
+  # transactions are immediate), so a storage round trip inside one stalls
+  # every other writer. Call .prepare outside the caller's transaction too
+  # (Mailboxes.receive does) and only rows are written while it is open.
   module InboundEmail
     Result = Struct.new(:record, :created, keyword_init: true) do
       def created? = created
     end
 
     def self.persist(source:, message_id: nil, message_checksum: nil, metadata: {})
+      prepare(source: source, message_id: message_id, message_checksum: message_checksum,
+        metadata: metadata).persist
+    end
+
+    # Resolves the identity and, unless a record already has it, uploads the raw
+    # email. Call in the tenant that will own the record, outside any
+    # transaction; then call #persist on the result.
+    def self.prepare(source:, message_id: nil, message_checksum: nil, metadata: {})
       unless defined?(::ActionMailbox::InboundEmail)
         raise ConfigurationError, "load ActionMailbox before persisting inbound mail"
       end
@@ -33,24 +48,72 @@ module MailboxKit
       end
       identity = { message_id: message_id || "#{checksum}@mailbox-kit.invalid",
                    message_checksum: checksum }
-      model = ::ActionMailbox::InboundEmail
-      model.transaction do
-        if (existing = model.find_by(identity))
-          next Result.new(record: existing, created: false).freeze
-        end
-        # Rails returns nil on a unique conflict. Roll back that savepoint before
-        # looking up the winner, including on PostgreSQL's aborted transactions.
-        record = model.transaction(requires_new: true) do
-          created = model.create_and_extract_message_id!(source, **identity)
-          raise ::ActiveRecord::Rollback unless created
-          unless metadata.empty?
-            blob = created.raw_email.blob
-            blob.update!(metadata: blob.metadata.merge(metadata))
+      Prepared.new(source, identity, metadata).tap(&:upload)
+    end
+
+    class Prepared
+      # The tenant the bytes were uploaded under, for a #discard run later.
+      attr_reader :tenant_key
+
+      def initialize(source, identity, metadata)
+        @source, @identity, @metadata = source, identity, metadata
+        @tenant_key = Tenancy.current_key if defined?(Tenancy)
+      end
+
+      # Uploads the bytes unless a record with this identity already exists.
+      def upload
+        return if @blob || model.uncached { model.exists?(@identity) }
+
+        blob = ::ActiveStorage::Blob.new(filename: "message.eml", content_type: "message/rfc822",
+          service_name: ::ActionMailbox.storage_service, metadata: @metadata.to_h)
+        io = StringIO.new(@source)
+        blob.unfurl(io)
+        blob.upload_without_unfurling(io)
+        @blob = blob
+      end
+
+      # Writes the record and its blob row, or returns the record a concurrent
+      # delivery already wrote. Uploaded bytes no committed row will name are
+      # deleted once no transaction is open.
+      def persist
+        result = model.transaction do
+          # Uncached: a request's query cache would repeat #upload's answer.
+          if (existing = model.uncached { model.find_by(@identity) })
+            next Result.new(record: existing, created: false).freeze
           end
-          created
+          # Rare: a record existed when #upload looked and is gone now.
+          upload
+          # Rails returns nil on a unique conflict. Roll back that savepoint before
+          # looking up the winner, including on PostgreSQL's aborted transactions.
+          record = model.transaction(requires_new: true) do
+            model.create!(raw_email: @blob, **@identity)
+          rescue ::ActiveRecord::RecordNotUnique
+            raise ::ActiveRecord::Rollback
+          end
+          Result.new(record: record || model.uncached { model.find_by!(@identity) }, created: !record.nil?).freeze
         end
-        Result.new(record: record || model.find_by!(identity), created: !record.nil?).freeze
+        discard unless result.created?
+        result
+      rescue StandardError
+        discard unless model.connection.transaction_open?
+        raise
+      end
+
+      # Deletes the uploaded bytes unless a committed row names them. Run after
+      # the transaction that would have committed that row has finished.
+      def discard
+        return unless @blob
+        return if ::ActiveStorage::Blob.uncached { ::ActiveStorage::Blob.exists?(key: @blob.key) }
+
+        @blob.service.delete(@blob.key)
+        @blob = nil
+      rescue StandardError => error
+        warn "mailbox-kit: raw email #{@blob&.key} not deleted: #{error.class}: #{error.message}"
       end
+
+      private
+
+      def model = ::ActionMailbox::InboundEmail
     end
   end
 end

data/lib/mailbox_kit/mailboxes/service.rb

@@ -39,15 +39,26 @@ module MailboxKit
         raise ArgumentError, "provide source or a persistence block, not both" if source && block
         raise ArgumentError, "source or persistence block required" unless source || block
         in_recipient_tenant(recipient) do |session, address, directory|
-          session.receive(address, directory) do |destination|
-            if block
-              # Legacy custom persistence blocks must return the existing Rails
-              # record on duplicates when membership attachment is desired.
-              block.lambda? && block.arity.zero? ? block.call : block.call(destination)
-            else
-              require "mailbox_kit/inbound_email"
-              InboundEmail.persist(source: source).record
+          if source
+            # Upload before session.receive opens the membership transaction,
+            # and only for a destination that can accept it.
+            session.with_recipient(address, directory) { nil }
+            require "mailbox_kit/inbound_email"
+            prepared = InboundEmail.prepare(source: source)
+          end
+          begin
+            session.receive(address, directory) do |destination|
+              if block
+                # Legacy custom persistence blocks must return the existing Rails
+                # record on duplicates when membership attachment is desired.
+                block.lambda? && block.arity.zero? ? block.call : block.call(destination)
+              else
+                prepared.persist.record
+              end
             end
+          ensure
+            # The transaction has finished: delete the upload if it rolled back.
+            prepared&.discard
           end
         end
       end

data/lib/mailbox_kit/version.rb

@@ -1,3 +1,3 @@
 module MailboxKit
-  VERSION = "0.1.0"
+  VERSION = "0.1.1"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: mailbox-kit
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.1
 platform: ruby
 authors:
 - Cole
@@ -27,7 +27,6 @@ files:
 - app/views/mailbox_kit/management/mailboxes/message.html.erb
 - app/views/mailbox_kit/management/mailboxes/show.html.erb
 - docs/integration.md
-- docs/upgrading.md
 - lib/generators/mailbox_kit/install/install_generator.rb
 - lib/generators/mailbox_kit/install/templates/create_mailbox_kit_mailboxes.rb
 - lib/generators/mailbox_kit/install/templates/create_mailbox_kit_receiving_domains.rb

data/docs/upgrading.md

@@ -1,60 +0,0 @@
-# Upgrading existing mailbox installations
-
-The extracted core keeps existing `cloudflare_email_` tables, record IDs, public
-Cloudflare constants and tenant job payloads. Keep your initializer, mounted
-engine, Worker and application processing code. Do not run `mailbox_kit:install`
-or the Cloudflare mailbox installer over existing mailbox tables.
-
-After updating the gems, generate the index and receiving-domain migrations:
-
-```sh
-bin/rails generate mailbox_kit:upgrade
-bin/rails db:migrate
-```
-
-For separate tenant databases:
-
-```sh
-bin/rails generate mailbox_kit:upgrade --tenant-migrations-path=db/tenant_migrate --directory-migrations-path=db/migrate
-```
-
-Apply the index migration to every database containing mailbox messages using your
-application's tenant migration runner. It does not belong in the shared domain
-directory database. The migration preserves messages and skips an existing
-single-column inbound index. New installers already include this index.
-
-Apply `AllowProviderNeutralReceivingDomains` to the shared directory database.
-It allows a null receiving-domain account ID, preserving existing account values.
-Receiving-only registration requires no Cloudflare account; sending still requires
-an explicit account and verified sending domain. On SQLite this schema change can
-rebuild the directory table, so use your normal migration window.
-
-Index creation can block writes on large tables. Plan its application according
-to your database's normal migration procedure; PostgreSQL installations needing
-online builds can adapt it to `algorithm: :concurrently` with
-`disable_ddl_transaction!`.
-
-## Verify the application upgrade
-
-1. Receive a message through the existing authenticated provider integration.
-2. Confirm it appears in the right inbox and runs the existing business handler.
-3. Replay the delivery and confirm no duplicate business effects.
-4. Check authorized and unauthorized UI access, read/archive actions, and retention.
-5. For separate databases, exercise tenant routing and processing jobs in two tenants.
-
-Keep your application's sender acceptance, ownership reconciliation and processing
-claims. Rails processing status does not replace these business policies.
-
-## Adding Cloudflare to a core-only application
-
-The core's mailbox schema alone does not supply Cloudflare's outbox and event
-tables. Add those provider tables and sending configuration explicitly using the
-Cloudflare integration guide; do not run both complete installers against the
-same database. Arbitrary subdomain receiving does not authorize sending from those
-subdomains: verify each Cloudflare sending domain separately.
-
-## Versions
-
-Cloudflare Email 0.4.0 depends on Mailbox Kit 0.1.x and installs it automatically.
-Core-only applications can install `gem "mailbox-kit", "~> 0.1.0"` directly.
-Upgrade and verify your application's email flows before a wider rollout.