hailer 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 9438bee546359ce97506f8c5111ffb1eed0d1fe187a4632385e11f61e37c574a
4
+ data.tar.gz: 1945fc1ab7fec8409cdea1a9de9e82347d204afbeb98226c864ea27d46713011
5
+ SHA512:
6
+ metadata.gz: 91f53a1872615cca824ddaf029b073baa144f94dad7163b92fcb2c03ff1e687a24d8e65a4813516ff3a5339ef5d348103965c4719c68a18cae2c9a5aac4bfdca
7
+ data.tar.gz: 921b031e5486c6b6757e3ccb9d8673d0c3a6f7dd2c8aa4cb4be85b596f93572b167b8cc411ae01d8f33c0af9b72a6069b5af4f08d8257e2968092f7fef724a1d
data/CHANGELOG.md ADDED
@@ -0,0 +1,9 @@
1
+ # Changelog
2
+
3
+ ## 0.1.0 (unreleased)
4
+
5
+ - Rails 8.1+/Ruby 4.0+ ActionMailer delivery with automatic defaults.
6
+ - Typed provider-neutral feedback client and Rails callback consumer.
7
+ - SES sender, SQS feedback ingestion, durable suppressions, and import tooling.
8
+ - Platform gems carrying the standalone Rust service with a verified launcher.
9
+ - Local CommonShare integration tested without sending real email.
data/LICENSE.txt ADDED
@@ -0,0 +1,29 @@
1
+ BSD 3-Clause License
2
+
3
+ Copyright (c) 2026, Abdelkader Boudih
4
+ All rights reserved.
5
+
6
+ Redistribution and use in source and binary forms, with or without
7
+ modification, are permitted provided that the following conditions are met:
8
+
9
+ 1. Redistributions of source code must retain the above copyright notice, this
10
+ list of conditions and the following disclaimer.
11
+
12
+ 2. Redistributions in binary form must reproduce the above copyright notice,
13
+ this list of conditions and the following disclaimer in the documentation
14
+ and/or other materials provided with the distribution.
15
+
16
+ 3. Neither the name of the copyright holder nor the names of its
17
+ contributors may be used to endorse or promote products derived from
18
+ this software without specific prior written permission.
19
+
20
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
21
+ AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
22
+ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
23
+ DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
24
+ FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
25
+ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
26
+ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
27
+ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
28
+ OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
data/README.md ADDED
@@ -0,0 +1,241 @@
1
+ # Hailer
2
+
3
+ Rails writes the email. Hailer deals with sending it.
4
+
5
+ This repository ships two things at version **0.1.0**:
6
+
7
+ - **`hailer` gem** — an ActionMailer adapter, feedback client, and executable
8
+ launcher. Platform gems bundle the Rust binary. No Ruby AWS SDK or native extension.
9
+ - **`hailer` crate** — a standalone Rust mail ship, supervised by Mothership.
10
+ SESv2 is the first provider.
11
+
12
+ ```text
13
+ ActionMailer → hailer gem → private Mothership route → hailer binary → SES
14
+ ```
15
+
16
+ ## Rails
17
+
18
+ Requires Ruby 4.0+ and Rails 8.1 or newer within Rails 8.x.
19
+ Rails main (8.2 alpha) is also exercised in CommonShare. Older releases are not supported.
20
+
21
+ ```ruby
22
+ # Gemfile (use path: "/path/to/hailer" while developing)
23
+ gem "hailer"
24
+ ```
25
+
26
+ The Railtie registers and selects Hailer automatically. Set the private service
27
+ URL in the Rails process's environment:
28
+
29
+ ```sh
30
+ HAILER_URL=http://127.0.0.1:9293/internal/mail/send
31
+ ```
32
+
33
+ No application initializer is needed. To customize connection settings:
34
+
35
+ ```ruby
36
+ config.action_mailer.hailer_settings = {
37
+ endpoint: ENV.fetch("HAILER_URL"),
38
+ open_timeout: 5,
39
+ read_timeout: 30,
40
+ write_timeout: 30
41
+ }
42
+ ```
43
+
44
+ `HAILER_URL` is the complete private URL, for example
45
+ `http://127.0.0.1:9293/internal/mail/send`. It is also the default endpoint
46
+ when omitted from `hailer_settings`. There is no default host or fallback to SES.
47
+ The app can explicitly override the default, for example
48
+ `config.action_mailer.delivery_method = :test` in its test environment.
49
+
50
+ Keep using your mailers, interceptors, observers, `deliver_now`, and
51
+ `deliver_later`. Rails still owns job scheduling. The adapter sends rendered MIME
52
+ and separate envelope recipients, preserving explicit `smtp_envelope_from` and
53
+ `smtp_envelope_to` overrides. After SES accepts the message, it sets
54
+ `message.header[:ses_message_id]`.
55
+
56
+ ## Rust service
57
+
58
+ Platform gems install the `hailer` executable, ready for Mothership to launch.
59
+ Installing the gem does not start the service or send mail.
60
+
61
+ The source (`ruby`) gem has the same Rails client and launcher, but no binary.
62
+ For source development or an unsupported platform:
63
+
64
+ ```sh
65
+ cargo install --path . --locked
66
+ export HAILER_BINARY=/absolute/path/to/hailer
67
+ ```
68
+
69
+ Run it as a Mothership HTTP bay. Merge this example into your fleet manifest;
70
+ the bind must stay private and must not be routed on a public listener:
71
+
72
+ ```toml
73
+ [mothership.bind]
74
+ mail = "127.0.0.1:9293"
75
+
76
+ [[bays.http]]
77
+ name = "hailer"
78
+ command = "hailer"
79
+ workers = 0
80
+ threads = 2
81
+ routes = [{ bind = "mail", pattern = "^/internal/mail/send$" }]
82
+ ```
83
+
84
+ Mothership exports `MS_PID`, `MS_SHIP`, and `MS_SOCKET_DIR`. Hailer binds its
85
+ docking Unix socket; it does **not** run another HTTP server. For containers,
86
+ use a private network listener with network policy instead of a loopback address
87
+ if Rails runs in a different network namespace.
88
+
89
+ Give the **service**, not the Rails adapter, AWS region and credentials through
90
+ the standard AWS credential chain. Its identity needs `ses:SendEmail`, and the
91
+ SES account/identities must be configured for sending.
92
+
93
+ ## Bounces, complaints, and suppressions
94
+
95
+ ```text
96
+ SES → SNS → dedicated SQS queue → Hailer → durable events + suppressions
97
+ ```
98
+
99
+ Hailer atomically commits each event, deduplication key, and suppression change
100
+ to a local redb database **before** acknowledging SQS. Failed acknowledgements
101
+ can be retried without duplicating stored events. Invalid/untrusted messages
102
+ are left for SQS's dead-letter policy.
103
+
104
+ - Permanent bounces and complaints suppress recipients.
105
+ - Transient/undetermined bounces are recorded without permanent suppression.
106
+ - An explicit `not-spam` complaint does not create a new suppression.
107
+ - Suppression checks cover the actual To/Cc/Bcc envelope, including overrides.
108
+ Matching preserves CommonShare's case/plus-alias policy.
109
+ - Mixed messages keep allowed recipients and their original addresses; MIME
110
+ is unchanged. If every recipient is suppressed, Hailer returns HTTP 422
111
+ without calling SES. A suppression lookup failure fails closed with HTTP 503.
112
+
113
+ Copy [hailer.example.toml](hailer.example.toml), set the real queue/topic, and
114
+ pass `--config /path/to/hailer.toml` or `HAILER_CONFIG`. Without `[feedback]`,
115
+ the consumer is disabled, but previously stored suppressions are still enforced.
116
+ `HAILER_STATE_PATH` overrides `state_path`; the default is `data/hailer.redb`.
117
+ Use a persistent, private volume in production.
118
+
119
+ **One Hailer process owns one state file and one dedicated queue.** Run the
120
+ bay with `workers = 0`; this is not a distributed suppression database.
121
+ Do not run competing consumers with separate state files. Back up the state
122
+ file while Hailer is stopped; event history and deduplication currently have
123
+ no automatic retention/pruning.
124
+
125
+ Hailer also exposes a provider-neutral, nondestructive event feed:
126
+
127
+ ```http
128
+ POST /internal/mail/events
129
+ Content-Type: application/json
130
+
131
+ {"after":0,"limit":100}
132
+ ```
133
+
134
+ Route it on the **same private bind**, if application tracking needs it:
135
+
136
+ ```toml
137
+ routes = [{ bind = "mail", pattern = "^/internal/mail/(send|events)$" }]
138
+ ```
139
+
140
+ The gem validates the page and exposes immutable `FeedbackEvent` objects:
141
+
142
+ ```ruby
143
+ page = Hailer::FeedbackClient.new(endpoint: ENV.fetch("HAILER_URL")).poll(after: 0)
144
+ page.events.each { |event| puts [event.event_id, event.kind, event.message_id] }
145
+ ```
146
+
147
+ Configure a Rails business-history handler (a callable or reloadable class name):
148
+
149
+ ```ruby
150
+ Rails.application.config.hailer.feedback_handler = "Platform::MailFeedback"
151
+ ```
152
+
153
+ Then `Hailer::FeedbackConsumer.new.call(after: cursor)` calls that handler for
154
+ each validated event and returns the next cursor only after all callbacks
155
+ succeed. A callback must **raise** on failure. Persist the cursor and domain
156
+ updates in one application database transaction. Retried pages require
157
+ idempotent handlers; no hidden poller, scheduling, or checkpoint database is
158
+ started by the gem. Reading does not acknowledge or erase events.
159
+
160
+ See [feedback setup and migration](docs/feedback.md) before removing a receiver.
161
+
162
+ **Security:** the route currently has no application-level authentication.
163
+ `/internal/` is a name, not an access control. Keep the bind private. The client
164
+ does not follow redirects or use environment HTTP proxies.
165
+
166
+ ## Failure semantics
167
+
168
+ An HTTP 200 means **SES accepted the message**, not that the recipient received
169
+ it. A successful response must include a nonblank `message_id`.
170
+
171
+ | Failure | Ruby exception |
172
+ | --- | --- |
173
+ | HTTP 400 / 422 | `Hailer::ValidationError` |
174
+ | HTTP 502 | `Hailer::RejectedError` |
175
+ | HTTP 503 | `Hailer::UnavailableError` |
176
+ | Other non-200 status | `Hailer::DeliveryError` |
177
+ | Invalid/oversized response, missing message ID | `Hailer::ProtocolError` |
178
+ | Connection, TLS, or timeout failure | `Hailer::TransportError` |
179
+ | Missing/invalid endpoint or timeout settings | `Hailer::ConfigurationError` |
180
+
181
+ The gem does not retry. A lost response can happen **after** SES accepted the
182
+ message; retrying may duplicate it. There is no durable **outbound** queue or
183
+ send-idempotency store yet; feedback deduplication does not deduplicate sends.
184
+ Rails jobs can also retry, so configure them deliberately.
185
+ The current service maps all SES SDK send errors to HTTP 502; that status is
186
+ not yet a reliable distinction between permanent and transient AWS failures.
187
+
188
+ ## Status and migration
189
+
190
+ The delivery contract and Hailer-side feedback processing are implemented. CommonShare's
191
+ existing sender is the compatibility reference, not a runtime dependency.
192
+ See [the contract](docs/delivery-contract.md).
193
+
194
+ Before switching an existing app:
195
+
196
+ 1. Run Hailer on a private Mothership bind and verify real SES delivery.
197
+ 2. Add the gem, set `HAILER_URL`, and remove the app's old delivery-method
198
+ override so Hailer's default takes effect.
199
+ 3. Subscribe a dedicated SQS queue before exporting and importing existing
200
+ suppressions. Let Hailer catch up and verify the event feed.
201
+ 4. Keep existing receivers/interceptors until routing, suppression import,
202
+ and any provider-neutral app history consumer have been verified.
203
+ 5. Only then remove the old sender and AWS-specific feedback handling.
204
+
205
+ CommonShare's local integration is tested with its real application, database,
206
+ Mothership, packaged Hailer binary, and a loopback SES fixture. Its existing AWS
207
+ receivers remain until the real subscription and suppression cutover is verified.
208
+ No deployed AWS routing has been changed.
209
+ Account diagnostics, validation/behavior plugins,
210
+ distributed state, rate limiting, graceful draining, and reliable send-error
211
+ classification are later work. No plugin API is promised yet.
212
+
213
+ ## Development
214
+
215
+ ```sh
216
+ bundle install
217
+ bundle exec rake
218
+ cargo test --locked
219
+ cargo fmt --all -- --check
220
+ gem build hailer.gemspec --output /tmp/hailer-0.1.0.gem
221
+ bundle exec rake native:package
222
+ ```
223
+
224
+ The Ruby suite exercises real local HTTP exchanges and a booted Rails app.
225
+ The Rust suite checks the same JSON fixture and the docking exchange with a
226
+ fake provider. Feedback tests cover redelivery, acknowledgement ordering,
227
+ atomic rollback, restart persistence, imports, and suppression enforcement.
228
+ Neither suite sends real mail or requires AWS credentials.
229
+
230
+ `native:package` builds a locked release and writes
231
+ `pkg/hailer-0.1.0-<platform>.gem`. Each platform gem contains exactly its host
232
+ binary and a version/platform/SHA-256 manifest. The launcher verifies that
233
+ manifest before `exec`, preserving arguments, exit status, and signal handling.
234
+ `HAILER_BINARY` is an explicit, version-checked source-build override; there is
235
+ no download or PATH fallback.
236
+
237
+ Build on each supported release host before publishing its artifact. Only
238
+ `arm64-darwin` has been built and exercised in this checkout. Linux GNU builds
239
+ must use your intended minimum-glibc build environment; a native build on a
240
+ newer distro is not automatically portable to older distributions. Nothing in
241
+ the packaging task publishes a gem.
@@ -0,0 +1,59 @@
1
+ # Delivery contract, v0.1
2
+
3
+ The gem and service are released from this repository. Keep their versions in
4
+ sync. Platform gems embed the compiled service, but it still runs as a
5
+ separate Mothership-supervised process, never inside Rails.
6
+
7
+ ## Request
8
+
9
+ `POST /internal/mail/send`, `Content-Type: application/json`.
10
+
11
+ ```json
12
+ {
13
+ "raw_mime": "From: Alice <alice@example.com>\r\nTo: bob@example.com\r\n\r\nHello\r\n",
14
+ "to_addresses": ["bob@example.com"],
15
+ "cc_addresses": [],
16
+ "bcc_addresses": []
17
+ }
18
+ ```
19
+
20
+ - `raw_mime` is the full `Mail::Message#to_s`, not a rebuilt message.
21
+ - `from_email_address` is omitted unless the SMTP envelope sender was
22
+ explicitly assigned. SES otherwise reads the MIME From header, including
23
+ display names.
24
+ - `to_addresses` uses explicitly assigned `smtp_envelope_to`, otherwise `to`.
25
+ - `cc_addresses` and `bcc_addresses` preserve their original recipient groups,
26
+ including when `smtp_envelope_to` is overridden. This matches the existing
27
+ CommonShare SESv2 adapter.
28
+ - Absent recipient groups are arrays, never JSON `null`. The service also
29
+ defaults omitted recipient arrays to empty.
30
+ - The service validates addresses and requires at least one recipient.
31
+
32
+ Do not expose this endpoint to untrusted callers. Messages and addresses are
33
+ sensitive; do not log raw request bodies.
34
+
35
+ ## Response
36
+
37
+ SES acceptance: HTTP **200**, `{"message_id":"SES-message-id"}`.
38
+ The adapter copies the ID to the original message's `ses_message_id` header
39
+ after acceptance, so Rails delivery observers can read it.
40
+
41
+ Failure: HTTP **400**, **422**, **502**, or **503**, `{"error":"description"}`.
42
+ Other paths/methods currently receive **404**. See the README for Ruby error
43
+ classes and the service's current AWS error-classification limitation.
44
+
45
+ There is no redirect handling, fire-and-forget acceptance, automatic client
46
+ retry, delivery guarantee, or idempotency key. `deliver_later` remains a Rails
47
+ job. A timeout or lost response is an ambiguous outcome.
48
+
49
+ ## Compatibility tests
50
+
51
+ `test/fixtures/send_request.json` is consumed by the Ruby HTTP client tests and
52
+ Rust deserialization/exchange tests. Ruby tests separately render real
53
+ `Mail::Message` instances, including binary attachments, display names, Bcc,
54
+ and explicit envelope overrides. Rails boot tests check automatic configuration,
55
+ `deliver_now`, `deliver_later` job scheduling, observer timing, failure
56
+ propagation, and explicit `:test` delivery.
57
+
58
+ Provider acceptance tests use a fake provider, not SES. This establishes the
59
+ in-repository contract, not real AWS or Mothership interoperability.
data/docs/feedback.md ADDED
@@ -0,0 +1,160 @@
1
+ # Feedback and the CommonShare cutover
2
+
3
+ ## Ownership
4
+
5
+ Hailer owns SES/SNS parsing, SQS consumption, deduplication, suppression storage,
6
+ and filtering before delivery. It has **no public SNS webhook**.
7
+
8
+ Applications can consume normalized events to update business records. They
9
+ do not need to know about SNS subscriptions or SES notification schemas.
10
+ CommonShare's local generic history consumer has been wired and tested. Its
11
+ webhook receivers have **not** been removed: deployed AWS subscriptions and
12
+ the final suppression migration must still be verified before that is safe.
13
+
14
+ ## AWS prerequisites
15
+
16
+ 1. Configure SES identity notifications or configuration-set event publishing
17
+ to an SNS topic. At minimum, publish Bounce and Complaint; add Delivery and
18
+ other event types if application history needs them.
19
+ 2. Subscribe a **dedicated SQS queue** with `RawMessageDelivery=false`. Hailer
20
+ requires the SNS envelope and checks its `TopicArn` against the configured
21
+ topic. Both SES's `notificationType` and `eventType` payloads are supported.
22
+ 3. Restrict the queue policy to the intended SNS topic. Do not grant application
23
+ roles `sqs:SendMessage` on this queue: an authorized sender could forge an
24
+ SNS-looking envelope. Topic checking is not signature verification.
25
+ 4. Configure a dead-letter queue, a finite receive count, suitable retention,
26
+ and alarms for oldest-message age, backlog, and dead-letter messages.
27
+ 5. Give Hailer's role `sqs:ReceiveMessage` and `sqs:DeleteMessage` on that queue,
28
+ in addition to `ses:SendEmail`. For a customer-managed encrypted queue,
29
+ grant the necessary KMS decrypt permissions too.
30
+ 6. Set the queue URL, SNS topic ARN, and persistent state path in
31
+ `hailer.toml` (see `hailer.example.toml`). Region/credentials come from the
32
+ shared standard AWS credential chain, not the Rails adapter.
33
+
34
+ SES identity feedback works automatically for those identities. If using an
35
+ SES configuration set, make sure the sent message actually selects it, for
36
+ example with SES's `X-SES-CONFIGURATION-SET` MIME header. Merely creating an SNS
37
+ subscription does not enable SES event publishing.
38
+
39
+ Hailer long-polls in batches of 10 with a 60-second visibility timeout. It
40
+ commits first, then deletes each SQS receipt. Failed parsing, state writes,
41
+ or deletes leave the message available for retry. Poison/unknown event types
42
+ are not silently discarded: use and monitor the DLQ.
43
+
44
+ ## Suppression policy
45
+
46
+ New Permanent bounces and complaints suppress recipients. Transient and
47
+ Undetermined bounces are stored as events but do not permanently block mail.
48
+ `not-spam` feedback does not add a suppression and does not automatically
49
+ clear an existing one. There is no automatic unsuppression API yet.
50
+
51
+ Matching trims/downcases the address and strips the local part's `+alias`,
52
+ matching CommonShare's existing suppression policy. This is not a universal
53
+ email-equivalence rule; it is an explicit migration policy. Only matching keys
54
+ are normalized: allowed delivery addresses and the MIME message are unchanged.
55
+
56
+ Imported suppressions are retained, including any old records that were
57
+ originally created for transient bounces. Imports are atomic, repeatable, and
58
+ cannot downgrade a complaint to a bounce. New feedback also cannot downgrade
59
+ a complaint. The current service has one local state owner; separate replicas
60
+ must not share a queue while maintaining separate state.
61
+
62
+ ## Import existing suppressions
63
+
64
+ Subscribe SQS **before** the final export so new feedback accumulates while
65
+ you migrate. Keep CommonShare's existing receiver active.
66
+
67
+ Export with read-only application code to a protected file (the data is PII):
68
+
69
+ ```sh
70
+ umask 077
71
+ SUPPRESSION_EXPORT=/secure/suppressions.json bin/rails runner '
72
+ records = Platform::EmailSuppression.select(
73
+ :email_address, :reason, :suppressed_at, :originating_message_id, :metadata
74
+ )
75
+ File.write(ENV.fetch("SUPPRESSION_EXPORT"), records.map(&:attributes).to_json)
76
+ '
77
+ ```
78
+
79
+ The file is a JSON array. Each record needs `email_address`, `reason` (`bounce`
80
+ or `complaint`), and an RFC 3339 `suppressed_at`. Message ID/metadata are optional.
81
+ Extra ActiveRecord fields are ignored. See `test/fixtures/suppressions.json`.
82
+
83
+ Stop Hailer before the import; redb permits one process to open its state file:
84
+
85
+ ```sh
86
+ hailer --config /etc/hailer.toml import-suppressions --file /secure/suppressions.json
87
+ ```
88
+
89
+ This command needs neither Mothership nor AWS credentials. An invalid address,
90
+ reason, or timestamp rejects the whole batch without partially importing.
91
+ Restart Hailer against the **same durable state path**, consume the backlog,
92
+ and verify known suppressed recipients cannot reach SES. Protect and delete
93
+ temporary exports according to your data-retention policy.
94
+
95
+ ## Provider-neutral event feed
96
+
97
+ `POST /internal/mail/events` on Hailer's private Mothership bind:
98
+
99
+ ```json
100
+ {"after":0,"limit":100}
101
+ ```
102
+
103
+ `limit` is 1–100. A page looks like:
104
+
105
+ ```json
106
+ {
107
+ "events": [{
108
+ "sequence": 1,
109
+ "event": {
110
+ "event_id": "arn:aws:sns:us-east-1:123456789012:mail-feedback:sns-id",
111
+ "kind": "bounced",
112
+ "provider": "ses",
113
+ "provider_message_id": "ses-id",
114
+ "message_id": "<original-rails-message-id@example.com>",
115
+ "occurred_at": "2026-10-20T12:00:00Z",
116
+ "recipients": ["recipient@example.com"],
117
+ "suppression": "bounce"
118
+ }
119
+ }],
120
+ "next_cursor": 1
121
+ }
122
+ ```
123
+
124
+ Kinds: `sent`, `delivered`, `bounced`, `complained`, `rejected`,
125
+ `rendering_failed`, `delayed`, `opened`, `clicked`.
126
+
127
+ The sequence is **ingestion order**, not event time. Delivery events can arrive
128
+ late or out of order. Consumers should use `event_id` for deduplication, handle
129
+ `occurred_at` deliberately, and persist their cursor with their domain changes.
130
+ Retrying a page is safe only if the application handler is idempotent. There
131
+ is no global acknowledgement or destructive read; each consumer owns its cursor.
132
+
133
+ Use `message_id` to correlate the original RFC 5322 Message-ID; fall back to
134
+ `provider_message_id` if your app stores the SES ID. `message_id` may be null
135
+ when SES omitted headers. The feed contains recipient PII: keep it private.
136
+ `Hailer::FeedbackClient#poll(after:, limit:)` validates the entire page before
137
+ exposing typed events. `Hailer::FeedbackConsumer#call` dispatches them through
138
+ `config.hailer.feedback_handler` and returns a cursor only after every callback
139
+ succeeds. A missing handler fails before polling. Applications still own
140
+ scheduling, idempotency, business-record updates, and durable cursor storage.
141
+
142
+ ## Safe removal checklist
143
+
144
+ Before deleting CommonShare's AWS-specific email handling:
145
+
146
+ - Verify a real SES mailbox-simulator bounce and complaint reach Hailer's queue.
147
+ - Verify suppression persists across a service restart and blocks later sends.
148
+ - Import the final old suppression snapshot with no gap in queue coverage.
149
+ - Verify a generic event consumer updates `OutgoingEmail` without SES/SNS parsing.
150
+ - Change the sender to Hailer and verify normal, CC/BCC, attachment, and
151
+ explicit-envelope delivery.
152
+ - Unsubscribe the old **email-specific** HTTP subscriptions, then remove the
153
+ SES controllers/routes, parsing jobs/operations, and Ruby bounce interceptor.
154
+ - Check other SNS consumers (including `sns_insights`) before removing shared
155
+ controllers, models, gems, admin history, or confirmation code.
156
+ - Keep historical database records unless a separate retention migration is
157
+ approved. Removing a receiver is not a reason to drop historical tables.
158
+
159
+ No AWS infrastructure was changed. CommonShare's local integration is on its
160
+ own feature branch; its existing SES/SNS receivers are retained for cutover.
data/exe/hailer ADDED
@@ -0,0 +1,11 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ require 'hailer/binary'
5
+
6
+ begin
7
+ exec Hailer::Binary.path, *ARGV
8
+ rescue Hailer::Error => error
9
+ warn error.message
10
+ exit 1
11
+ end
@@ -0,0 +1,8 @@
1
+ # Keep this file and its state volume private. One Hailer process owns this file.
2
+ state_path = "/var/lib/hailer/state.redb"
3
+ threads = 2
4
+
5
+ [feedback]
6
+ # Dedicated queue; subscribe the SNS topic with RawMessageDelivery=false.
7
+ queue_url = "https://sqs.us-east-1.amazonaws.com/123456789012/hailer-feedback"
8
+ topic_arn = "arn:aws:sns:us-east-1:123456789012:mail-feedback"
@@ -0,0 +1,60 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'digest'
4
+ require 'json'
5
+ require 'open3'
6
+ require_relative 'errors'
7
+ require_relative 'version'
8
+
9
+ module Hailer
10
+ # Used by the executable without booting Rails. Never download a binary or
11
+ # select an unrelated executable from PATH.
12
+ module Binary
13
+ ROOT = File.expand_path('../..', __dir__)
14
+
15
+ module_function
16
+
17
+ def platform(local = Gem::Platform.local)
18
+ case local.os
19
+ when 'darwin'
20
+ cpu = %w[arm64 aarch64].include?(local.cpu) ? 'arm64' : local.cpu
21
+ "#{cpu}-darwin"
22
+ when 'linux'
23
+ cpu = %w[arm64 aarch64].include?(local.cpu) ? 'aarch64' : local.cpu
24
+ "#{cpu}-linux-#{local.version == 'musl' ? 'musl' : 'gnu'}"
25
+ else
26
+ raise BinaryError, "unsupported Hailer platform: #{local}"
27
+ end
28
+ end
29
+
30
+ def path(env: ENV, root: ROOT)
31
+ if env.key?('HAILER_BINARY')
32
+ candidate = File.expand_path(env.fetch('HAILER_BINARY'))
33
+ executable!(candidate)
34
+ output, status = Open3.capture2e(candidate, '--version')
35
+ unless status.success? && output.strip == "hailer #{VERSION}"
36
+ raise BinaryError, "HAILER_BINARY must be Hailer #{VERSION}"
37
+ end
38
+ return candidate
39
+ end
40
+
41
+ directory = File.join(root, 'vendor', platform)
42
+ candidate = File.join(directory, 'bin', 'hailer')
43
+ executable!(candidate)
44
+ manifest = JSON.parse(File.read(File.join(directory, 'manifest.json')))
45
+ unless manifest['version'] == VERSION && manifest['platform'] == platform &&
46
+ manifest['sha256'] == Digest::SHA256.file(candidate).hexdigest
47
+ raise BinaryError, 'bundled Hailer executable does not match its release manifest'
48
+ end
49
+ candidate
50
+ rescue Errno::ENOENT, JSON::ParserError
51
+ raise BinaryError, 'missing or invalid Hailer binary manifest; reinstall the platform gem'
52
+ end
53
+
54
+ def executable!(candidate)
55
+ return if File.file?(candidate) && File.executable?(candidate)
56
+
57
+ raise BinaryError, "no executable Hailer binary at #{candidate}; install the platform gem or set HAILER_BINARY"
58
+ end
59
+ end
60
+ end
@@ -0,0 +1,20 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Hailer
4
+ Response = Data.define(:message_id)
5
+
6
+ class Client
7
+ def initialize(**settings)
8
+ @transport = Transport.new(**settings)
9
+ end
10
+
11
+ def deliver(payload)
12
+ document = @transport.post('/internal/mail/send', payload)
13
+ message_id = document['message_id']
14
+ unless message_id.is_a?(String) && !message_id.strip.empty?
15
+ raise ProtocolError, 'Hailer accepted the request without a message_id'
16
+ end
17
+ Response.new(message_id: message_id)
18
+ end
19
+ end
20
+ end
@@ -0,0 +1,39 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'mail'
4
+
5
+ module Hailer
6
+ class DeliveryMethod
7
+ attr_reader :settings
8
+
9
+ def initialize(options = {})
10
+ @settings = { endpoint: ENV['HAILER_URL'], **Transport::DEFAULTS }.merge(options)
11
+ end
12
+
13
+ def deliver!(message)
14
+ payload = {
15
+ raw_mime: message.to_s,
16
+ to_addresses: Array(explicit_envelope?(message, :to) ? message.smtp_envelope_to : message.to),
17
+ cc_addresses: Array(message.cc),
18
+ bcc_addresses: Array(message.bcc)
19
+ }
20
+ # The default smtp_envelope_from drops display names. Let SES read the
21
+ # MIME From header unless the caller explicitly overrode the envelope.
22
+ if explicit_envelope?(message, :from)
23
+ payload[:from_email_address] = message.smtp_envelope_from
24
+ end
25
+
26
+ Client.new(**settings).deliver(payload).tap do |response|
27
+ message.header[:ses_message_id] = response.message_id
28
+ end
29
+ end
30
+
31
+ private
32
+
33
+ # Match the existing SESv2 adapter's distinction between Mail's computed
34
+ # defaults and explicitly assigned SMTP envelope addresses.
35
+ def explicit_envelope?(message, field)
36
+ message.instance_variable_get(:"@smtp_envelope_#{field}")
37
+ end
38
+ end
39
+ end
@@ -0,0 +1,24 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Hailer
4
+ class Error < StandardError; end
5
+ class ConfigurationError < Error; end
6
+ class ProtocolError < Error; end
7
+ class BinaryError < Error; end
8
+
9
+ # The request may already have reached SES. Retrying can send a duplicate.
10
+ class TransportError < Error; end
11
+
12
+ class DeliveryError < Error
13
+ attr_reader :status
14
+
15
+ def initialize(message, status:)
16
+ @status = status
17
+ super(message)
18
+ end
19
+ end
20
+
21
+ class ValidationError < DeliveryError; end
22
+ class RejectedError < DeliveryError; end
23
+ class UnavailableError < DeliveryError; end
24
+ end
@@ -0,0 +1,71 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'time'
4
+
5
+ module Hailer
6
+ FeedbackEvent = Data.define(
7
+ :sequence, :event_id, :kind, :provider, :provider_message_id,
8
+ :message_id, :occurred_at, :recipients, :suppression
9
+ )
10
+ FeedbackPage = Data.define(:events, :next_cursor)
11
+
12
+ class FeedbackClient
13
+ KINDS = %w[sent delivered bounced complained rejected rendering_failed delayed opened clicked].freeze
14
+ MAX_RESPONSE_BYTES = 2 * 1024 * 1024
15
+
16
+ def initialize(**settings)
17
+ @transport = Transport.new(**settings)
18
+ end
19
+
20
+ def poll(after:, limit: 100)
21
+ unless after.is_a?(Integer) && after >= 0 && limit.is_a?(Integer) && (1..100).cover?(limit)
22
+ raise ConfigurationError, 'feedback requires a nonnegative cursor and limit in 1..100'
23
+ end
24
+ document = @transport.post('/internal/mail/events', { after: after, limit: limit },
25
+ max_response_bytes: MAX_RESPONSE_BYTES)
26
+ entries = document['events']
27
+ cursor = document['next_cursor']
28
+ unless entries.is_a?(Array) && entries.size <= limit && cursor.is_a?(Integer) && cursor >= after
29
+ raise ProtocolError, 'invalid Hailer feedback page'
30
+ end
31
+ previous = after
32
+ events = entries.map do |entry|
33
+ event = decode_event(entry)
34
+ raise ProtocolError, 'feedback sequences must increase past the requested cursor' unless event.sequence > previous
35
+
36
+ previous = event.sequence
37
+ event
38
+ end
39
+ raise ProtocolError, 'feedback cursor does not match its page' unless cursor == previous
40
+
41
+ FeedbackPage.new(events: events.freeze, next_cursor: cursor)
42
+ end
43
+
44
+ private
45
+
46
+ def decode_event(entry)
47
+ raise ProtocolError, 'invalid feedback entry' unless entry.is_a?(Hash) && entry['sequence'].is_a?(Integer)
48
+
49
+ event = entry['event']
50
+ unless event.is_a?(Hash) && KINDS.include?(event['kind']) &&
51
+ %w[event_id provider provider_message_id occurred_at].all? { |key| text?(event[key]) } &&
52
+ (event['message_id'].nil? || text?(event['message_id'])) &&
53
+ [nil, 'bounce', 'complaint'].include?(event['suppression']) &&
54
+ event['recipients'].is_a?(Array) && !event['recipients'].empty? &&
55
+ event['recipients'].all? { |address| text?(address) }
56
+ raise ProtocolError, 'invalid normalized Hailer event'
57
+ end
58
+ FeedbackEvent.new(
59
+ sequence: entry['sequence'], event_id: event['event_id'].freeze,
60
+ kind: event['kind'].freeze, provider: event['provider'].freeze,
61
+ provider_message_id: event['provider_message_id'].freeze,
62
+ message_id: event['message_id']&.freeze, occurred_at: Time.iso8601(event['occurred_at']).freeze,
63
+ recipients: event['recipients'].map(&:freeze).freeze, suppression: event['suppression']&.freeze
64
+ )
65
+ rescue ArgumentError
66
+ raise ProtocolError, 'invalid feedback event timestamp'
67
+ end
68
+
69
+ def text?(value) = value.is_a?(String) && !value.strip.empty?
70
+ end
71
+ end
@@ -0,0 +1,22 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Hailer
4
+ # The caller persists the returned cursor with its domain changes. An
5
+ # exception returns no cursor; retrying the page requires idempotent handlers.
6
+ class FeedbackConsumer
7
+ def initialize(client: nil, handler: nil)
8
+ @client = client || FeedbackClient.new(**ActionMailer::Base.hailer_settings)
9
+ @handler = handler || Rails.application.config.hailer.feedback_handler
10
+ end
11
+
12
+ def call(after:, limit: 100)
13
+ # Resolve a configured class name on every poll, respecting Rails reloads.
14
+ handler = @handler.is_a?(String) ? @handler.constantize : @handler
15
+ raise ConfigurationError, 'configure config.hailer.feedback_handler with a callable handler' unless handler.respond_to?(:call)
16
+
17
+ page = @client.poll(after: after, limit: limit)
18
+ page.events.each { |event| handler.call(event) }
19
+ page.next_cursor
20
+ end
21
+ end
22
+ end
@@ -0,0 +1,19 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'action_mailer/railtie'
4
+
5
+ module Hailer
6
+ class Railtie < ::Rails::Railtie
7
+ # A default, not an after-boot override: an app's test environment can
8
+ # still choose :test without contacting the service.
9
+ config.action_mailer.delivery_method = :hailer
10
+ config.hailer = ActiveSupport::OrderedOptions.new
11
+ config.hailer.feedback_handler = nil
12
+
13
+ initializer 'hailer.action_mailer', before: 'action_mailer.set_configs' do
14
+ ActiveSupport.on_load(:action_mailer) do
15
+ add_delivery_method :hailer, Hailer::DeliveryMethod
16
+ end
17
+ end
18
+ end
19
+ end
@@ -0,0 +1,82 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'json'
4
+ require 'net/http'
5
+ require 'uri'
6
+
7
+ module Hailer
8
+ # No persistent connection across forks, redirects, HTTP_PROXY, or retries.
9
+ class Transport
10
+ DEFAULTS = { open_timeout: 5, read_timeout: 30, write_timeout: 30 }.freeze
11
+ MAX_RESPONSE_BYTES = 65_536
12
+ HTTP_ERRORS = {
13
+ 400 => ValidationError, 422 => ValidationError,
14
+ 502 => RejectedError, 503 => UnavailableError
15
+ }.freeze
16
+
17
+ def initialize(endpoint: ENV['HAILER_URL'], **timeouts)
18
+ @uri = URI.parse(endpoint.to_s)
19
+ unless @uri.is_a?(URI::HTTP) && @uri.host && !@uri.host.empty? &&
20
+ !@uri.userinfo && !@uri.query && !@uri.fragment &&
21
+ @uri.path == '/internal/mail/send'
22
+ raise ConfigurationError, 'set HAILER_URL or endpoint to an HTTP(S) URL ending in /internal/mail/send'
23
+ end
24
+ @timeouts = DEFAULTS.merge(timeouts)
25
+ raise ConfigurationError, 'unknown Hailer timeout setting' unless (@timeouts.keys - DEFAULTS.keys).empty?
26
+
27
+ unless @timeouts.values.all? { |value| value.is_a?(Numeric) && value.finite? && value.positive? }
28
+ raise ConfigurationError, 'Hailer timeouts must be positive, finite numbers'
29
+ end
30
+ rescue URI::InvalidURIError
31
+ raise ConfigurationError, 'invalid Hailer endpoint URL'
32
+ end
33
+
34
+ def post(path, payload, max_response_bytes: MAX_RESPONSE_BYTES)
35
+ request = Net::HTTP::Post.new(path)
36
+ request.content_type = 'application/json'
37
+ request['Accept'] = 'application/json'
38
+ request.body = JSON.generate(payload)
39
+ http = Net::HTTP.new(@uri.hostname, @uri.port, nil)
40
+ http.use_ssl = @uri.scheme == 'https'
41
+ http.max_retries = 0
42
+ @timeouts.each { |name, value| http.public_send(:"#{name}=", value) }
43
+ status, body = exchange(http, request, max_response_bytes)
44
+ decode_response(status, body)
45
+ rescue Timeout::Error, IOError, EOFError, SystemCallError, SocketError,
46
+ OpenSSL::SSL::SSLError, Net::HTTPBadResponse, Net::HTTPHeaderSyntaxError => error
47
+ raise TransportError, "Hailer transport failed (#{error.class}); delivery outcome may be unknown"
48
+ end
49
+
50
+ private
51
+
52
+ def exchange(http, request, limit)
53
+ status = nil
54
+ body = +''
55
+ http.start do |connection|
56
+ connection.request(request) do |response|
57
+ status = response.code.to_i
58
+ response.read_body do |chunk|
59
+ raise ProtocolError, 'Hailer response exceeds the response size limit' if body.bytesize + chunk.bytesize > limit
60
+
61
+ body << chunk
62
+ end
63
+ end
64
+ end
65
+ [status, body]
66
+ end
67
+
68
+ def decode_response(status, body)
69
+ document = JSON.parse(body)
70
+ raise ProtocolError, 'Hailer response must be a JSON object' unless document.is_a?(Hash)
71
+
72
+ unless status == 200
73
+ message = document['error']
74
+ message = "Hailer returned HTTP #{status}" unless message.is_a?(String) && !message.empty?
75
+ raise HTTP_ERRORS.fetch(status, DeliveryError).new(message, status: status)
76
+ end
77
+ document
78
+ rescue JSON::ParserError
79
+ raise ProtocolError, "Hailer returned invalid JSON (HTTP #{status})"
80
+ end
81
+ end
82
+ end
@@ -0,0 +1,5 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Hailer
4
+ VERSION = '0.1.0'
5
+ end
data/lib/hailer.rb ADDED
@@ -0,0 +1,10 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative 'hailer/version'
4
+ require_relative 'hailer/errors'
5
+ require_relative 'hailer/transport'
6
+ require_relative 'hailer/client'
7
+ require_relative 'hailer/delivery_method'
8
+ require_relative 'hailer/feedback_client'
9
+ require_relative 'hailer/feedback_consumer'
10
+ require_relative 'hailer/railtie'
metadata ADDED
@@ -0,0 +1,190 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: hailer
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - Abdelkader Boudih
8
+ bindir: exe
9
+ cert_chain: []
10
+ date: 1980-01-02 00:00:00.000000000 Z
11
+ dependencies:
12
+ - !ruby/object:Gem::Dependency
13
+ name: actionmailer
14
+ requirement: !ruby/object:Gem::Requirement
15
+ requirements:
16
+ - - ">="
17
+ - !ruby/object:Gem::Version
18
+ version: '8.1'
19
+ - - "<"
20
+ - !ruby/object:Gem::Version
21
+ version: '9'
22
+ type: :runtime
23
+ prerelease: false
24
+ version_requirements: !ruby/object:Gem::Requirement
25
+ requirements:
26
+ - - ">="
27
+ - !ruby/object:Gem::Version
28
+ version: '8.1'
29
+ - - "<"
30
+ - !ruby/object:Gem::Version
31
+ version: '9'
32
+ - !ruby/object:Gem::Dependency
33
+ name: json
34
+ requirement: !ruby/object:Gem::Requirement
35
+ requirements:
36
+ - - ">="
37
+ - !ruby/object:Gem::Version
38
+ version: '2.18'
39
+ - - "<"
40
+ - !ruby/object:Gem::Version
41
+ version: '4'
42
+ type: :runtime
43
+ prerelease: false
44
+ version_requirements: !ruby/object:Gem::Requirement
45
+ requirements:
46
+ - - ">="
47
+ - !ruby/object:Gem::Version
48
+ version: '2.18'
49
+ - - "<"
50
+ - !ruby/object:Gem::Version
51
+ version: '4'
52
+ - !ruby/object:Gem::Dependency
53
+ name: mail
54
+ requirement: !ruby/object:Gem::Requirement
55
+ requirements:
56
+ - - ">="
57
+ - !ruby/object:Gem::Version
58
+ version: '2.9'
59
+ - - "<"
60
+ - !ruby/object:Gem::Version
61
+ version: '3'
62
+ type: :runtime
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - ">="
67
+ - !ruby/object:Gem::Version
68
+ version: '2.9'
69
+ - - "<"
70
+ - !ruby/object:Gem::Version
71
+ version: '3'
72
+ - !ruby/object:Gem::Dependency
73
+ name: net-http
74
+ requirement: !ruby/object:Gem::Requirement
75
+ requirements:
76
+ - - ">="
77
+ - !ruby/object:Gem::Version
78
+ version: '0.6'
79
+ - - "<"
80
+ - !ruby/object:Gem::Version
81
+ version: '1'
82
+ type: :runtime
83
+ prerelease: false
84
+ version_requirements: !ruby/object:Gem::Requirement
85
+ requirements:
86
+ - - ">="
87
+ - !ruby/object:Gem::Version
88
+ version: '0.6'
89
+ - - "<"
90
+ - !ruby/object:Gem::Version
91
+ version: '1'
92
+ - !ruby/object:Gem::Dependency
93
+ name: railties
94
+ requirement: !ruby/object:Gem::Requirement
95
+ requirements:
96
+ - - ">="
97
+ - !ruby/object:Gem::Version
98
+ version: '8.1'
99
+ - - "<"
100
+ - !ruby/object:Gem::Version
101
+ version: '9'
102
+ type: :runtime
103
+ prerelease: false
104
+ version_requirements: !ruby/object:Gem::Requirement
105
+ requirements:
106
+ - - ">="
107
+ - !ruby/object:Gem::Version
108
+ version: '8.1'
109
+ - - "<"
110
+ - !ruby/object:Gem::Version
111
+ version: '9'
112
+ - !ruby/object:Gem::Dependency
113
+ name: minitest
114
+ requirement: !ruby/object:Gem::Requirement
115
+ requirements:
116
+ - - "~>"
117
+ - !ruby/object:Gem::Version
118
+ version: '6.0'
119
+ type: :development
120
+ prerelease: false
121
+ version_requirements: !ruby/object:Gem::Requirement
122
+ requirements:
123
+ - - "~>"
124
+ - !ruby/object:Gem::Version
125
+ version: '6.0'
126
+ - !ruby/object:Gem::Dependency
127
+ name: rake
128
+ requirement: !ruby/object:Gem::Requirement
129
+ requirements:
130
+ - - "~>"
131
+ - !ruby/object:Gem::Version
132
+ version: '13.0'
133
+ type: :development
134
+ prerelease: false
135
+ version_requirements: !ruby/object:Gem::Requirement
136
+ requirements:
137
+ - - "~>"
138
+ - !ruby/object:Gem::Version
139
+ version: '13.0'
140
+ description: A thin ActionMailer adapter for the standalone Hailer Rust service. Rails
141
+ renders the message; Hailer owns SES delivery.
142
+ email:
143
+ - oss@seuros.com
144
+ executables:
145
+ - hailer
146
+ extensions: []
147
+ extra_rdoc_files: []
148
+ files:
149
+ - CHANGELOG.md
150
+ - LICENSE.txt
151
+ - README.md
152
+ - docs/delivery-contract.md
153
+ - docs/feedback.md
154
+ - exe/hailer
155
+ - hailer.example.toml
156
+ - lib/hailer.rb
157
+ - lib/hailer/binary.rb
158
+ - lib/hailer/client.rb
159
+ - lib/hailer/delivery_method.rb
160
+ - lib/hailer/errors.rb
161
+ - lib/hailer/feedback_client.rb
162
+ - lib/hailer/feedback_consumer.rb
163
+ - lib/hailer/railtie.rb
164
+ - lib/hailer/transport.rb
165
+ - lib/hailer/version.rb
166
+ homepage: https://github.com/seuros/hailer
167
+ licenses:
168
+ - BSD-3-Clause
169
+ metadata:
170
+ source_code_uri: https://github.com/seuros/hailer
171
+ changelog_uri: https://github.com/seuros/hailer/blob/master/CHANGELOG.md
172
+ rubygems_mfa_required: 'true'
173
+ rdoc_options: []
174
+ require_paths:
175
+ - lib
176
+ required_ruby_version: !ruby/object:Gem::Requirement
177
+ requirements:
178
+ - - ">="
179
+ - !ruby/object:Gem::Version
180
+ version: '4.0'
181
+ required_rubygems_version: !ruby/object:Gem::Requirement
182
+ requirements:
183
+ - - ">="
184
+ - !ruby/object:Gem::Version
185
+ version: '0'
186
+ requirements: []
187
+ rubygems_version: 4.0.10
188
+ specification_version: 4
189
+ summary: ActionMailer delivery through the Hailer mail ship
190
+ test_files: []