hailer 0.1.0 → 0.3.0

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: hailer
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Abdelkader Boudih
@@ -149,17 +149,21 @@ files:
 - CHANGELOG.md
 - LICENSE.txt
 - README.md
-- docs/delivery-contract.md
-- docs/feedback.md
+- docs/durable-delivery.md
 - exe/hailer
 - hailer.example.toml
 - lib/hailer.rb
 - lib/hailer/binary.rb
 - lib/hailer/client.rb
+- lib/hailer/delivery_context.rb
 - lib/hailer/delivery_method.rb
 - lib/hailer/errors.rb
 - lib/hailer/feedback_client.rb
 - lib/hailer/feedback_consumer.rb
+- lib/hailer/lifecycle_client.rb
+- lib/hailer/mail_delivery_job.rb
+- lib/hailer/queued_client.rb
+- lib/hailer/queued_delivery_method.rb
 - lib/hailer/railtie.rb
 - lib/hailer/transport.rb
 - lib/hailer/version.rb

data/docs/delivery-contract.md

@@ -1,59 +0,0 @@
-# Delivery contract, v0.1
-
-The gem and service are released from this repository. Keep their versions in
-sync. Platform gems embed the compiled service, but it still runs as a
-separate Mothership-supervised process, never inside Rails.
-
-## Request
-
-`POST /internal/mail/send`, `Content-Type: application/json`.
-
-```json
-{
-  "raw_mime": "From: Alice <alice@example.com>\r\nTo: bob@example.com\r\n\r\nHello\r\n",
-  "to_addresses": ["bob@example.com"],
-  "cc_addresses": [],
-  "bcc_addresses": []
-}
-```
-
-- `raw_mime` is the full `Mail::Message#to_s`, not a rebuilt message.
-- `from_email_address` is omitted unless the SMTP envelope sender was
-  explicitly assigned. SES otherwise reads the MIME From header, including
-  display names.
-- `to_addresses` uses explicitly assigned `smtp_envelope_to`, otherwise `to`.
-- `cc_addresses` and `bcc_addresses` preserve their original recipient groups,
-  including when `smtp_envelope_to` is overridden. This matches the existing
-  CommonShare SESv2 adapter.
-- Absent recipient groups are arrays, never JSON `null`. The service also
-  defaults omitted recipient arrays to empty.
-- The service validates addresses and requires at least one recipient.
-
-Do not expose this endpoint to untrusted callers. Messages and addresses are
-sensitive; do not log raw request bodies.
-
-## Response
-
-SES acceptance: HTTP **200**, `{"message_id":"SES-message-id"}`.
-The adapter copies the ID to the original message's `ses_message_id` header
-after acceptance, so Rails delivery observers can read it.
-
-Failure: HTTP **400**, **422**, **502**, or **503**, `{"error":"description"}`.
-Other paths/methods currently receive **404**. See the README for Ruby error
-classes and the service's current AWS error-classification limitation.
-
-There is no redirect handling, fire-and-forget acceptance, automatic client
-retry, delivery guarantee, or idempotency key. `deliver_later` remains a Rails
-job. A timeout or lost response is an ambiguous outcome.
-
-## Compatibility tests
-
-`test/fixtures/send_request.json` is consumed by the Ruby HTTP client tests and
-Rust deserialization/exchange tests. Ruby tests separately render real
-`Mail::Message` instances, including binary attachments, display names, Bcc,
-and explicit envelope overrides. Rails boot tests check automatic configuration,
-`deliver_now`, `deliver_later` job scheduling, observer timing, failure
-propagation, and explicit `:test` delivery.
-
-Provider acceptance tests use a fake provider, not SES. This establishes the
-in-repository contract, not real AWS or Mothership interoperability.

data/docs/feedback.md

@@ -1,160 +0,0 @@
-# Feedback and the CommonShare cutover
-
-## Ownership
-
-Hailer owns SES/SNS parsing, SQS consumption, deduplication, suppression storage,
-and filtering before delivery. It has **no public SNS webhook**.
-
-Applications can consume normalized events to update business records. They
-do not need to know about SNS subscriptions or SES notification schemas.
-CommonShare's local generic history consumer has been wired and tested. Its
-webhook receivers have **not** been removed: deployed AWS subscriptions and
-the final suppression migration must still be verified before that is safe.
-
-## AWS prerequisites
-
-1. Configure SES identity notifications or configuration-set event publishing
-   to an SNS topic. At minimum, publish Bounce and Complaint; add Delivery and
-   other event types if application history needs them.
-2. Subscribe a **dedicated SQS queue** with `RawMessageDelivery=false`. Hailer
-   requires the SNS envelope and checks its `TopicArn` against the configured
-   topic. Both SES's `notificationType` and `eventType` payloads are supported.
-3. Restrict the queue policy to the intended SNS topic. Do not grant application
-   roles `sqs:SendMessage` on this queue: an authorized sender could forge an
-   SNS-looking envelope. Topic checking is not signature verification.
-4. Configure a dead-letter queue, a finite receive count, suitable retention,
-   and alarms for oldest-message age, backlog, and dead-letter messages.
-5. Give Hailer's role `sqs:ReceiveMessage` and `sqs:DeleteMessage` on that queue,
-   in addition to `ses:SendEmail`. For a customer-managed encrypted queue,
-   grant the necessary KMS decrypt permissions too.
-6. Set the queue URL, SNS topic ARN, and persistent state path in
-   `hailer.toml` (see `hailer.example.toml`). Region/credentials come from the
-   shared standard AWS credential chain, not the Rails adapter.
-
-SES identity feedback works automatically for those identities. If using an
-SES configuration set, make sure the sent message actually selects it, for
-example with SES's `X-SES-CONFIGURATION-SET` MIME header. Merely creating an SNS
-subscription does not enable SES event publishing.
-
-Hailer long-polls in batches of 10 with a 60-second visibility timeout. It
-commits first, then deletes each SQS receipt. Failed parsing, state writes,
-or deletes leave the message available for retry. Poison/unknown event types
-are not silently discarded: use and monitor the DLQ.
-
-## Suppression policy
-
-New Permanent bounces and complaints suppress recipients. Transient and
-Undetermined bounces are stored as events but do not permanently block mail.
-`not-spam` feedback does not add a suppression and does not automatically
-clear an existing one. There is no automatic unsuppression API yet.
-
-Matching trims/downcases the address and strips the local part's `+alias`,
-matching CommonShare's existing suppression policy. This is not a universal
-email-equivalence rule; it is an explicit migration policy. Only matching keys
-are normalized: allowed delivery addresses and the MIME message are unchanged.
-
-Imported suppressions are retained, including any old records that were
-originally created for transient bounces. Imports are atomic, repeatable, and
-cannot downgrade a complaint to a bounce. New feedback also cannot downgrade
-a complaint. The current service has one local state owner; separate replicas
-must not share a queue while maintaining separate state.
-
-## Import existing suppressions
-
-Subscribe SQS **before** the final export so new feedback accumulates while
-you migrate. Keep CommonShare's existing receiver active.
-
-Export with read-only application code to a protected file (the data is PII):
-
-```sh
-umask 077
-SUPPRESSION_EXPORT=/secure/suppressions.json bin/rails runner '
-  records = Platform::EmailSuppression.select(
-    :email_address, :reason, :suppressed_at, :originating_message_id, :metadata
-  )
-  File.write(ENV.fetch("SUPPRESSION_EXPORT"), records.map(&:attributes).to_json)
-'
-```
-
-The file is a JSON array. Each record needs `email_address`, `reason` (`bounce`
-or `complaint`), and an RFC 3339 `suppressed_at`. Message ID/metadata are optional.
-Extra ActiveRecord fields are ignored. See `test/fixtures/suppressions.json`.
-
-Stop Hailer before the import; redb permits one process to open its state file:
-
-```sh
-hailer --config /etc/hailer.toml import-suppressions --file /secure/suppressions.json
-```
-
-This command needs neither Mothership nor AWS credentials. An invalid address,
-reason, or timestamp rejects the whole batch without partially importing.
-Restart Hailer against the **same durable state path**, consume the backlog,
-and verify known suppressed recipients cannot reach SES. Protect and delete
-temporary exports according to your data-retention policy.
-
-## Provider-neutral event feed
-
-`POST /internal/mail/events` on Hailer's private Mothership bind:
-
-```json
-{"after":0,"limit":100}
-```
-
-`limit` is 1–100. A page looks like:
-
-```json
-{
-  "events": [{
-    "sequence": 1,
-    "event": {
-      "event_id": "arn:aws:sns:us-east-1:123456789012:mail-feedback:sns-id",
-      "kind": "bounced",
-      "provider": "ses",
-      "provider_message_id": "ses-id",
-      "message_id": "<original-rails-message-id@example.com>",
-      "occurred_at": "2026-10-20T12:00:00Z",
-      "recipients": ["recipient@example.com"],
-      "suppression": "bounce"
-    }
-  }],
-  "next_cursor": 1
-}
-```
-
-Kinds: `sent`, `delivered`, `bounced`, `complained`, `rejected`,
-`rendering_failed`, `delayed`, `opened`, `clicked`.
-
-The sequence is **ingestion order**, not event time. Delivery events can arrive
-late or out of order. Consumers should use `event_id` for deduplication, handle
-`occurred_at` deliberately, and persist their cursor with their domain changes.
-Retrying a page is safe only if the application handler is idempotent. There
-is no global acknowledgement or destructive read; each consumer owns its cursor.
-
-Use `message_id` to correlate the original RFC 5322 Message-ID; fall back to
-`provider_message_id` if your app stores the SES ID. `message_id` may be null
-when SES omitted headers. The feed contains recipient PII: keep it private.
-`Hailer::FeedbackClient#poll(after:, limit:)` validates the entire page before
-exposing typed events. `Hailer::FeedbackConsumer#call` dispatches them through
-`config.hailer.feedback_handler` and returns a cursor only after every callback
-succeeds. A missing handler fails before polling. Applications still own
-scheduling, idempotency, business-record updates, and durable cursor storage.
-
-## Safe removal checklist
-
-Before deleting CommonShare's AWS-specific email handling:
-
-- Verify a real SES mailbox-simulator bounce and complaint reach Hailer's queue.
-- Verify suppression persists across a service restart and blocks later sends.
-- Import the final old suppression snapshot with no gap in queue coverage.
-- Verify a generic event consumer updates `OutgoingEmail` without SES/SNS parsing.
-- Change the sender to Hailer and verify normal, CC/BCC, attachment, and
-  explicit-envelope delivery.
-- Unsubscribe the old **email-specific** HTTP subscriptions, then remove the
-  SES controllers/routes, parsing jobs/operations, and Ruby bounce interceptor.
-- Check other SNS consumers (including `sns_insights`) before removing shared
-  controllers, models, gems, admin history, or confirmation code.
-- Keep historical database records unless a separate retention migration is
-  approved. Removing a receiver is not a reason to drop historical tables.
-
-No AWS infrastructure was changed. CommonShare's local integration is on its
-own feature branch; its existing SES/SNS receivers are retained for cutover.