hailer 0.4.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: '0228ebf052de9ab7235f507c6c86c4138c15a1b2624916d3901779aa6c3b0d9f'
4
- data.tar.gz: 6369fd82507756b988a461780a46029a198522ea8f335d21a267781153c0a34b
3
+ metadata.gz: c8cd926b627c51b8e8c96b95ad79bb68b8fbba9c4ca26ab466f03f41283cc35b
4
+ data.tar.gz: 7a4344f35eccd875fe2110aa10a35b0767d42c6acf025260c6813c732e644cb8
5
5
  SHA512:
6
- metadata.gz: 4e76c0c6953b06986d936260fe177c91d22512073f8b3f87cb39b550e65b76920eb9c0e596e8e60c3a184b15b7da26ca3ac58dd3cdfd67f7abf0ae9ce34fb8d9
7
- data.tar.gz: 6c93a10653fa45b4815bb20e34e18a6403fddb2d506cf6479065ea52cef7882b52e4926ad23252cf5c164f0a48338269f61d285dc23aa96226a96ef16e74c844
6
+ metadata.gz: 5b9aeebb39e331fb930cfcce8a7063d28a33fedf2314881bab62e469b4674286e807bffdd14b532d1db47199fcf1eb55f2b3cd0612cb1fcbbea2028430faf055
7
+ data.tar.gz: 7622c206fdfccefbee24a7c05952500c8e83a4fca3d0a84bbe05da2f3ebdf9eec4db1a37d6398f04639ed240c023ca2442f88295d83bd11f6ea41dd537a7860d
data/README.md CHANGED
@@ -2,33 +2,45 @@
2
2
 
3
3
  Rails writes the email. Hailer deals with sending it.
4
4
 
5
- This repository prepares two things at version **0.2.0** (not published yet):
5
+ This repository is two things:
6
6
 
7
7
  - **`hailer` gem** — an ActionMailer adapter, feedback client, and executable
8
8
  launcher. Platform gems bundle the Rust binary. No Ruby AWS SDK or native extension.
9
9
  - **`hailer` crate** — a standalone Rust mail ship, supervised by Mothership.
10
10
  SESv2 is the first provider.
11
11
 
12
- ```text
13
- ActionMailer → hailer gem → private Mothership route → hailer binary → SES
12
+ ```mermaid
13
+ flowchart LR
14
+ A[ActionMailer] --> B[hailer gem]
15
+ B --> C[private Mothership route]
16
+ C --> D[hailer binary]
17
+ D --> E[SES]
14
18
  ```
15
19
 
16
- The working tree also implements an **opt-in durable queued adapter**. See
17
- [durable delivery](docs/durable-delivery.md) for ownership, recovery, routing
18
- and operator controls. It is not included in the preserved 0.2 release
19
- candidates. Current-source macOS, ARM64 Linux and x86_64 Linux verification
20
- gates pass locally; this work has not been published or deployed.
20
+ An **opt-in durable queued adapter** sits alongside the default synchronous
21
+ send path (`[queued]` in `hailer.example.toml`).
21
22
 
22
- ## Rails
23
-
24
- Requires Ruby 4.0+ and Rails 8.1 or newer within Rails 8.x.
25
- Rails main (8.2 alpha) is also exercised in CommonShare. Older releases are not supported.
23
+ ## Install
26
24
 
27
25
  ```ruby
28
- # Gemfile (use path: "/path/to/hailer" while developing)
26
+ gem "guardship"
27
+ gem "shuttlebay"
29
28
  gem "hailer"
30
29
  ```
31
30
 
31
+ Mothership picks up the bay and starts sending mail. Keep using your
32
+ mailers, interceptors, observers, `deliver_now`, and `deliver_later` -
33
+ nothing else about how Rails sends mail changes.
34
+
35
+ Everything past this point is reference material: protocol internals,
36
+ TOML knobs, and operator commands for people running or debugging the
37
+ service, not required reading to use it.
38
+
39
+ ## Rails
40
+
41
+ Requires Ruby 4.0+ and Rails 8.1 or newer within Rails 8.x.
42
+ Rails main (8.2 alpha) is also exercised. Older releases are not supported.
43
+
32
44
  The Railtie registers and selects Hailer automatically. Set the private service
33
45
  URL in the Rails process's environment:
34
46
 
@@ -53,11 +65,9 @@ when omitted from `hailer_settings`. There is no default host or fallback to SES
53
65
  The app can explicitly override the default, for example
54
66
  `config.action_mailer.delivery_method = :test` in its test environment.
55
67
 
56
- Keep using your mailers, interceptors, observers, `deliver_now`, and
57
- `deliver_later`. Rails still owns job scheduling. The adapter sends rendered MIME
58
- and separate envelope recipients, preserving explicit `smtp_envelope_from` and
59
- `smtp_envelope_to` overrides. After SES accepts the message, it sets
60
- `message.header[:ses_message_id]`.
68
+ The adapter sends rendered MIME and separate envelope recipients, preserving
69
+ explicit `smtp_envelope_from` and `smtp_envelope_to` overrides. After SES
70
+ accepts the message, it sets `message.header[:ses_message_id]`.
61
71
 
62
72
  ## Rust service
63
73
 
@@ -98,7 +108,7 @@ SES account/identities must be configured for sending.
98
108
 
99
109
  ### PostgreSQL
100
110
 
101
- PostgreSQL is the **only live backend** in 0.2. Hailer owns a `hailer` schema,
111
+ PostgreSQL is the **only live backend**. Hailer owns a `hailer` schema,
102
112
  not Rails tables. Use a separate database/role where practical:
103
113
 
104
114
  ```sh
@@ -123,8 +133,12 @@ shutdown timeout. PostgreSQL 18 is exercised locally and in CI.
123
133
 
124
134
  ## Bounces, complaints, and suppressions
125
135
 
126
- ```text
127
- SES → SNS → dedicated SQS queue → Hailer → durable events + suppressions
136
+ ```mermaid
137
+ flowchart LR
138
+ A[SES] --> B[SNS]
139
+ B --> C[dedicated SQS queue]
140
+ C --> D[Hailer]
141
+ D --> E[durable events + suppressions]
128
142
  ```
129
143
 
130
144
  Hailer atomically commits each event, deduplication key, and suppression change
@@ -136,7 +150,7 @@ are left for SQS's dead-letter policy.
136
150
  - Transient/undetermined bounces are recorded without permanent suppression.
137
151
  - An explicit `not-spam` complaint does not create a new suppression.
138
152
  - Suppression checks cover the actual To/Cc/Bcc envelope, including overrides.
139
- Matching preserves CommonShare's case/plus-alias policy.
153
+ Matching is case-insensitive and ignores `+alias` tags.
140
154
  - Mixed messages keep allowed recipients and their original addresses; MIME
141
155
  is unchanged. If every recipient is suppressed, Hailer returns HTTP 422
142
156
  without calling SES. A suppression lookup failure fails closed with HTTP 503.
@@ -226,11 +240,16 @@ the section disables scheduled sync, not existing suppression enforcement.
226
240
  ### Control plane
227
241
 
228
242
  The behavior circuit evaluates **pure admission decisions**: input validity,
229
- service/storage readiness, suppression, and local rate admission. PostgreSQL,
243
+ service/storage readiness, suppression, and rate admission. PostgreSQL,
230
244
  AWS calls, clocks, and token reservations remain owned by the async host.
231
245
  The SES circuit breaker counts transient/ambiguous failures, not bad addresses
232
246
  or permanent request rejections. An optional `[rate_limit]` budgets envelope
233
- recipients **per process**, not per account; it does not discover SES quotas.
247
+ recipients through a single shared Postgres-backed token bucket - every
248
+ replica draws down the same account-wide budget, not an independent
249
+ per-process one. `discover_from_aws = true` replaces the configured rate
250
+ with the SES account's real `SendQuota.MaxSendRate`, re-checked
251
+ periodically; a rate-state read/write failure fails closed (denies
252
+ admission) rather than risk exceeding the real AWS quota blind.
234
253
 
235
254
  Private `GET /internal/mail/health` returns sanitized component states and
236
255
  counters. `GET /internal/mail/ready` returns 503 during initial sync or storage
@@ -261,8 +280,13 @@ turn off.
261
280
 
262
281
  ## Domain blocklist
263
282
 
264
- ```text
265
- SendRequest -> impossible-address check -> domain blocklist (free, local) -> suppression (free, local) -> verification (paid, external) -> SES
283
+ ```mermaid
284
+ flowchart LR
285
+ A[SendRequest] --> B[impossible-address check]
286
+ B --> C["domain blocklist<br/>(free, local)"]
287
+ C --> D["suppression<br/>(free, local)"]
288
+ D --> E["verification<br/>(paid, external)"]
289
+ E --> F[SES]
266
290
  ```
267
291
 
268
292
  Always active - an empty `hailer.domain_blocks` table is a no-op, there is
@@ -283,8 +307,11 @@ cannot be removed this way, by design.
283
307
 
284
308
  ## Address verification
285
309
 
286
- ```text
287
- SendRequest -> suppression check (cheap, local) -> address verification (paid, external) -> SES
310
+ ```mermaid
311
+ flowchart LR
312
+ A[SendRequest] --> B["suppression check<br/>(cheap, local)"]
313
+ B --> C["address verification<br/>(paid, external)"]
314
+ C --> D[SES]
288
315
  ```
289
316
 
290
317
  Opt-in pre-send deliverability check, off by default. Enable with
@@ -384,49 +411,14 @@ The SES SDK makes **one send attempt**. AWS throttling/transient failures return
384
411
  IDs carry the explicit unknown-outcome code. None of these responses authorizes
385
412
  an automatic retry of an ambiguous send.
386
413
 
387
- ## Status and migration
388
-
389
- The delivery contract and Hailer-side feedback processing are implemented. CommonShare's
390
- existing sender is the compatibility reference, not a runtime dependency.
391
-
392
- ### Upgrade from 0.1 redb
393
-
394
- 1. Stop all old Hailer processes cleanly. Keep the source file and a backup.
395
- 2. Provision an **empty** Hailer PostgreSQL database and set its URL.
396
- 3. Run `hailer migrate-redb --file /private/path/state.redb`.
397
- 4. Check reported counts, event cursors, and suppression enforcement before
398
- starting 0.2. Keep the source backup; do not restart old writers after cutover.
399
-
400
- Migration opens redb read-only, checks event/dedup mappings and suppression keys,
401
- and copies bounded batches in **one destination transaction**. It preserves
402
- event IDs, sequence values, suppression metadata, and next-cursor continuity.
403
- It rejects missing/corrupt sources and nonempty targets without wiping anything.
404
- An unclean redb file may require recovery with the old version against a backup
405
- before migration. There is no implicit fallback and no dual-write period.
406
-
407
- Before switching an existing app:
408
-
409
- 1. Provision PostgreSQL, run Hailer on a private Mothership bind, check readiness,
410
- and verify real SES delivery using a deliberate test.
411
- 2. Add the gem, set `HAILER_URL`, and remove the app's old delivery-method
412
- override so Hailer's default takes effect.
413
- 3. Subscribe a dedicated SQS queue before exporting and importing existing
414
- suppressions. Let Hailer catch up and verify the event feed.
415
- 4. Keep existing receivers/interceptors until routing, suppression import,
416
- and any provider-neutral app history consumer have been verified.
417
- 5. Only then remove the old sender and AWS-specific feedback handling.
418
-
419
- CommonShare's local 0.2 integration is tested with its real application,
420
- test database, Mothership, the optimized binary from the macOS gem, its installed 0.1 Ruby
421
- client, and a loopback SES fixture. Its existing AWS
422
- receivers remain until the real subscription and suppression cutover is verified.
423
- No deployed AWS routing has been changed.
424
- The current-source queued adapter and dispatcher also pass a CommonShare smoke
425
- using an isolated Bundler Gemfile, real Mothership and loopback SES. No app
426
- Gemfile or production configuration was changed. Its legacy history observer
427
- still needs queued-ownership semantics before opt-in deployment.
428
- Validation/behavior plugins and distributed account-wide quotas remain later
429
- work. No plugin API or provider exactly-once delivery is promised.
414
+ ## Status
415
+
416
+ The delivery contract, Hailer-side feedback processing, domain blocklist,
417
+ impossible-address rejection, address verification, the queued/quarantine
418
+ pipeline, and SES account diagnostics are all implemented and shipped.
419
+ `[queued]`, `[domain_intel]`, `[suppression_sync]`, `[feedback]`, and
420
+ `[email_verification]` ship disabled by default - absent config, not
421
+ missing code.
430
422
 
431
423
  ## Development
432
424
 
@@ -435,8 +427,6 @@ bundle install
435
427
  bundle exec rake
436
428
  HAILER_TEST_DATABASE_URL=postgres://test_user:password@127.0.0.1/hailer_test cargo test --locked
437
429
  cargo fmt --all -- --check
438
- gem build hailer.gemspec --output /tmp/hailer-0.2.0.gem
439
- bundle exec rake native:package
440
430
  ```
441
431
 
442
432
  The Ruby suite exercises real local HTTP exchanges and a booted Rails app.
@@ -448,76 +438,8 @@ each test creates and drops its own database. Never point this variable at a
448
438
  customer or Rails production database. Neither suite sends real mail or requires
449
439
  AWS credentials.
450
440
 
451
- For the opt-in CommonShare compatibility smoke, create a disposable Hailer
452
- database, then run from that application's checkout:
453
-
454
- ```sh
455
- RAILS_ENV=test HAILER_SMOKE_BINARY=/path/to/hailer/target/debug/hailer \
456
- HAILER_DATABASE_URL=postgres://test_user:password@127.0.0.1/hailer_smoke \
457
- bundle exec rails runner /path/to/hailer/test/commonshare_smoke.rb
458
- ```
459
-
460
- It refuses a non-test app database, points the AWS SDK at a loopback fixture,
461
- and rolls back application-history changes. It does not change the app's
462
- dependency lockfile or send external mail.
463
-
464
- `native:package` builds a locked release and writes
465
- `pkg/hailer-0.2.0-<platform>.gem`. Each platform gem contains exactly its host
466
- binary and a version/platform/SHA-256 manifest. The launcher verifies that
467
- manifest before `exec`, preserving arguments, exit status, and signal handling.
468
441
  `HAILER_BINARY` is an explicit, version-checked source-build override; there is
469
442
  no download or PATH fallback.
470
443
 
471
- Build on each supported release host before publishing its artifact. Nothing
472
- in the packaging task publishes a gem.
473
-
474
- ## Linux releases and RubyGems
475
-
476
- The **Linux binary gems** GitHub Actions workflow builds `x86_64-linux-gnu`
477
- and `aarch64-linux-gnu` natively against Debian 13 Trixie (glibc 2.41).
478
- New Trixie-built artifacts target glibc 2.41 or newer, not Bookworm or
479
- Alpine/musl. The preserved older 0.2 candidates retain their original baseline.
480
- Local OrbStack verification uses `test/linux/Dockerfile` for each architecture
481
- and a separate `ruby:4.0-slim-trixie` runtime.
482
- Use a copied checkout in separate per-architecture Docker work volumes; do not
483
- reuse the macOS `target/` or Ruby bundle. Run the locked Ruby tests, release Rust
484
- tests and `native:package` with an isolated PostgreSQL fixture, then install each
485
- gem in its slim runtime and exercise the launcher, migrations and import CLI.
486
- When transferring a macOS snapshot, use `COPYFILE_DISABLE=1 tar --no-xattrs`
487
- so AppleDouble `._*` metadata cannot appear as spurious SQL migrations.
488
-
489
- ```sh
490
- gh workflow run linux-gems.yml --repo seuros/hailer -f tag=v0.2.0
491
- ```
492
-
493
- The workflow checks out the requested release tag, runs the Ruby and Rust
494
- suites against an isolated PostgreSQL 18 service, and installs each binary gem
495
- in a separate slim Debian container. Pushes/PRs also verify the branch without
496
- creating a release or publishing anything; only explicit workflow dispatch
497
- can attach assets to an existing release.
498
- It verifies the installed platform, bundled-binary manifest, executable version,
499
- and PostgreSQL suppression import before attaching either Linux gem to the
500
- existing release. Existing gem assets are never overwritten; `SHA256SUMS` is
501
- updated to cover all release gems.
502
-
503
- Publishing a GitHub release does **not** publish to RubyGems. Like Shuttlebay,
504
- release assets can be fetched and published with Rake:
505
-
506
- ```sh
507
- bundle exec rake gems:fetch
508
- bundle exec rake gems:push_all
509
- # Or select an explicit release:
510
- bundle exec rake 'gems:push_all[v0.2.0]'
511
- ```
512
-
513
- `gems:push_all` fetches fresh assets, checks every archive's SHA-256,
514
- name/version/platform, and publishes only that verified set. It skips
515
- already-published platforms only when RubyGems reports the identical archive
516
- hash, so an interrupted publication can be resumed without replacing artifacts.
517
- `gems:push` is an alias. No release task rebuilds or silently republishes gems.
518
-
519
- RubyGems may require MFA or a suitably scoped API key. Keep credentials out of
520
- the repository. Publishing makes the packaged Ruby code and binaries public,
521
- even when the GitHub repository is private. Once published, Bundler selects
522
- the matching native gem; do not set `force_ruby_platform` in deployment if
523
- you want the bundled executable.
444
+ Bundler selects the matching native platform gem automatically; do not set
445
+ `force_ruby_platform` in deployment if you want the bundled executable.
data/hailer.example.toml CHANGED
@@ -12,13 +12,18 @@ shutdown_timeout_seconds = 30
12
12
  [suppression_sync]
13
13
  interval_seconds = 3600
14
14
 
15
- # Optional PER-PROCESS budget, not an account-wide SES quota.
15
+ # Optional. Shared across every replica (one Postgres-backed token bucket,
16
+ # not a per-process budget) - a second replica cannot double the admitted rate.
16
17
  # [rate_limit]
17
18
  # recipients_per_second = 10.0
18
19
  # burst = 50
20
+ # Replace recipients_per_second with the SES account's real
21
+ # SendQuota.MaxSendRate, re-checked periodically. recipients_per_second/burst
22
+ # above become just the seed used until the first successful discovery.
23
+ # discover_from_aws = true
24
+ # discovery_interval_seconds = 900
19
25
 
20
- # In development: durable intake requires explicit logical payload budgets.
21
- # Requires the durable-dispatch build, not the existing 0.2 release candidates.
26
+ # Durable intake requires explicit logical payload budgets.
22
27
  # [queued]
23
28
  # max_bytes = 268435456
24
29
  # max_messages = 10000
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Hailer
4
- VERSION = '0.4.0'
4
+ VERSION = '0.6.0'
5
5
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: hailer
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.4.0
4
+ version: 0.6.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Abdelkader Boudih
@@ -123,6 +123,20 @@ dependencies:
123
123
  - - "~>"
124
124
  - !ruby/object:Gem::Version
125
125
  version: '6.0'
126
+ - !ruby/object:Gem::Dependency
127
+ name: padded_cell
128
+ requirement: !ruby/object:Gem::Requirement
129
+ requirements:
130
+ - - ">="
131
+ - !ruby/object:Gem::Version
132
+ version: '0'
133
+ type: :development
134
+ prerelease: false
135
+ version_requirements: !ruby/object:Gem::Requirement
136
+ requirements:
137
+ - - ">="
138
+ - !ruby/object:Gem::Version
139
+ version: '0'
126
140
  - !ruby/object:Gem::Dependency
127
141
  name: rake
128
142
  requirement: !ruby/object:Gem::Requirement
@@ -149,7 +163,6 @@ files:
149
163
  - CHANGELOG.md
150
164
  - LICENSE.txt
151
165
  - README.md
152
- - docs/durable-delivery.md
153
166
  - exe/hailer
154
167
  - hailer.example.toml
155
168
  - lib/hailer.rb
@@ -1,217 +0,0 @@
1
- # Durable delivery (in development)
2
-
3
- The queued Ruby interface, PostgreSQL spool and supervised dispatcher are
4
- implemented. CommonShare source-client and installed macOS binary-gem smoke
5
- tests pass. ARM64 and x86_64 Linux release tests, binary packaging and clean slim
6
- Trixie runtime checks also pass in OrbStack (x86_64 under emulation).
7
- Do not enable queued delivery against the existing 0.2 release
8
- candidates or treat this working tree as a published durable-delivery release.
9
-
10
- ## Opt-in Rails configuration
11
-
12
- Keep `HAILER_URL` pointing to the existing private `/internal/mail/send` endpoint.
13
- The queued client uses the same validated origin with `/internal/mail/messages`.
14
-
15
- ```ruby
16
- # config/environments/production.rb
17
- config.action_mailer.delivery_method = :hailer_queued
18
- ```
19
-
20
- The initializer registers the adapter and selects `Hailer::MailDeliveryJob` for
21
- this opt-in configuration. The default remains the synchronous `:hailer`
22
- adapter, whose HTTP requests never retry.
23
-
24
- For a per-mailer opt-in, select both:
25
-
26
- ```ruby
27
- self.delivery_method = :hailer_queued
28
- self.delivery_job = Hailer::MailDeliveryJob
29
- ```
30
-
31
- An explicitly configured application delivery job is not replaced. It must
32
- inherit `Hailer::MailDeliveryJob` or include `Hailer::DeliveryJobContext` to
33
- obtain stable submission keys across job retries. This uses public ActiveJob
34
- callbacks, not Rails monkey patches.
35
-
36
- ## Identities and retries
37
-
38
- - Standard queued jobs use job ID plus delivery ordinal. Different jobs are
39
- different intents unless the application supplies a business key.
40
- - Direct `deliver_now` calls generate a fresh key per invocation.
41
- - Supply a business key through `delivery_method_options` when separate calls
42
- represent the same intended message:
43
-
44
- ```ruby
45
- mail(
46
- to: recipient,
47
- subject: 'Invoice',
48
- delivery_method_options: { submission_key: "invoice:#{invoice.id}:issued" }
49
- )
50
- ```
51
-
52
- - Supply an optional ISO 8601 `expires_at` through those same options. The
53
- service contract defaults to 24 hours and allows shorter deadlines.
54
- - A queued intake call serializes its command once. Up to three HTTP attempts
55
- reuse those exact bytes, key and deadline after transport loss or retryable
56
- admission errors (429/503). Set `intake_attempts: 1` to disable these retries.
57
- - Validation, submission conflicts, malformed responses and provider-unknown
58
- errors do not retry.
59
- - Rails re-renders mail when a job is performed again. If the bytes or envelope
60
- change under an accepted key, the service must return a conflict; the adapter
61
- raises `Hailer::SubmissionConflictError`. Job keys alone do not provide
62
- transparent exactly-once business delivery.
63
-
64
- Submission keys are command metadata, not MIME headers.
65
-
66
- ## Private status and lifecycle feed
67
-
68
- ```ruby
69
- client = Hailer::LifecycleClient.new # same HAILER_URL origin
70
- status = client.status(hailer_message_id)
71
- page = client.poll(after: saved_cursor, limit: 100)
72
- ```
73
-
74
- Status distinguishes the original RFC Message-ID, Hailer ID and provider ID,
75
- and never returns MIME. `status.attempts` and `status.recipient_outcomes` contain
76
- the latest 100 records each, in chronological order. Their corresponding
77
- `*_truncated` flags report older records; use the lifecycle feed for complete
78
- observations. An accepted message can still have an uncertain historical
79
- attempt after an operator-authorized retry. Lifecycle events have their own cursor space; do not
80
- reuse an existing `FeedbackClient` cursor. Persist `page.next_cursor` only after
81
- processing the page successfully. Re-reading a cursor does not delete events.
82
-
83
- The service's `[queued]` configuration requires explicit `max_bytes` and
84
- `max_messages`. Bytes measure canonical logical command payload, not PostgreSQL
85
- table/index/WAL/backups or a physical disk quota. All replicas of a deployment
86
- must use the same budgets. Admission is transactional; a replay does not consume
87
- another slot, and accepted messages are never evicted to admit new ones.
88
-
89
- ## Acceptance is not delivery
90
-
91
- HTTP 202 means the service durably owns the message. The adapter exposes its ID
92
- through `message.header[:hailer_message_id]` before Rails observers run. It does
93
- not fabricate `ses_message_id`; provider acceptance and recipient delivery
94
- are later observations.
95
-
96
- Application observers must distinguish durable ownership from provider
97
- acceptance: persist the Hailer ID and an owned/queued state, then consume
98
- lifecycle facts separately. CommonShare's existing observer still writes
99
- `status: "sent"` and only the SES ID for every delivered callback. The isolated
100
- queued smoke verifies transport compatibility, not that observer's readiness
101
- for production opt-in. Keep its synchronous configuration until this application
102
- cutover work is done.
103
-
104
- Unknown provider outcomes are held without an automatic resend. Seven-day
105
- unknown-body retention (measured from durable intake) does not extend the
106
- 24-hour send-start deadline. Body purge retains receipts and uncertainty.
107
- Only positive evidence of no acceptance permits automatic retry, with jittered
108
- backoff capped at 600 seconds and at most 255 automatic attempts per intent.
109
- Expiry stops new attempts; it never proves a previous attempt was unsent.
110
-
111
- Live bodies are purged after confirmed acceptance, terminal rejection or
112
- suppression, cancellation, or safe queued expiry. Purge releases logical
113
- capacity, not PostgreSQL allocated disk, backups or copies already in-flight.
114
-
115
- ## Feedback and private routing
116
-
117
- Set root `ses_configuration_set = "hailer-feedback"` in Hailer's TOML, or use
118
- an equivalent default configuration set on the SES sending identity. Configure
119
- SES event publishing through the dedicated SNS/SQS feedback route, preserving
120
- `mail.tags.hailer_message_id` and `mail.tags.hailer_attempt_id`. These unique tags
121
- allow feedback to resolve an unknown SDK response. A provider ID already
122
- captured from an accepted response also correlates later untagged feedback.
123
- Without tags and without a captured provider ID, feedback cannot resolve
124
- uncertainty. Hailer never guesses from recipient, timestamps or RFC Message-ID.
125
- Feedback received before an untagged provider ID is captured is not retroactively
126
- correlated; retain the tags for reliable recovery.
127
-
128
- Mothership must route the new private paths as well as the old synchronous
129
- ones. A suitable bind pattern is:
130
-
131
- ```toml
132
- routes = [{ bind = "mail", pattern = "^/internal/mail/(send|events|health|ready|intake-ready|dispatch-ready|messages|messages/events|messages/[0-9a-f-]{36})$" }]
133
- ```
134
-
135
- Keep this bind private. Intake readiness depends on durable writable capacity,
136
- not SES health. `/internal/mail/dispatch-ready` separately reports worker,
137
- bootstrap and control eligibility. Neither readiness response guarantees that
138
- a later send will succeed.
139
-
140
- ## Operator control
141
-
142
- Run against the intended database using the same `HAILER_DATABASE_URL`:
143
-
144
- ```sh
145
- hailer message-status --id MESSAGE_UUID
146
- hailer cancel-message --id MESSAGE_UUID --reason "intent withdrawn"
147
- hailer resolve-message --id MESSAGE_UUID --outcome accepted \
148
- --provider-message-id VERIFIED_SES_ID --reason "verified provider evidence"
149
- hailer resolve-message --id MESSAGE_UUID --outcome not-accepted \
150
- --reason "verified no provider acceptance"
151
- hailer resolve-message --id MESSAGE_UUID --outcome retry \
152
- --acknowledge-duplicate-risk --reason "investigated; accept duplicate risk"
153
- ```
154
-
155
- Cancellation is only possible before dispatch and cannot unsend mail.
156
- Resolution defaults to the current held-unknown attempt. Use `--attempt-id
157
- ATTEMPT_UUID` to resolve an older unknown journal entry without overwriting a
158
- later attempt. Retry is allowed only for the current held intent, with an
159
- available body, an unexpired deadline, no acceptance latch and explicit
160
- duplicate-risk acknowledgement. A new attempt never erases the old uncertainty.
161
- All decisions require an audit reason. After expiry or body purge, sending
162
- requires a new intent/key.
163
-
164
- ## Quarantine
165
-
166
- Opt-in, queued pipeline only - a synchronous `:hailer` send has no way to
167
- hold a message for later review, so a non-ASCII domain or a newly
168
- registered one makes the synchronous adapter reject outright. The queued
169
- adapter holds instead:
170
-
171
- ```toml
172
- [domain_intel]
173
- provider = "rdap"
174
- new_domain_threshold_days = 30
175
- ```
176
-
177
- A recipient domain that is not plain ASCII (a malformed/unencoded IDN, or a
178
- homograph-spoofing attempt - legitimate IDNs are transmitted as ASCII
179
- punycode) is always checked, regardless of this block. `[domain_intel]`
180
- additionally checks the domain's RDAP registration date; newer than the
181
- threshold holds the message as `quarantined` instead of admitting it as
182
- `queued`. The whole message is quarantined, not just the flagged recipient -
183
- this is an operator review gate on intent, not a per-recipient filter like
184
- suppression or the domain blocklist (`hailer ban-domain`). RDAP's bootstrap
185
- registry is public; no credential is needed.
186
-
187
- ```sh
188
- hailer release-quarantine --id MESSAGE_UUID --reason "reviewed, legitimate"
189
- hailer cancel-message --id MESSAGE_UUID --reason "rejected on review"
190
- ```
191
-
192
- Releasing resets `next_attempt_at` to now, so the message re-enters the
193
- normal dispatch queue immediately. `cancel-message` also accepts a
194
- quarantined message. An unactioned quarantine still expires on the same
195
- deadline as a normal `queued` message.
196
-
197
- ## Local application verification
198
-
199
- `test/commonshare_smoke.rb` runs inside CommonShare's test environment with a
200
- real Mothership bay and loopback SES, never real AWS credentials. Use a fresh
201
- loopback PostgreSQL database named `hailer_smoke_*` for each queued run.
202
- The script intentionally reuses one business key to check replay after purge.
203
-
204
- ```sh
205
- RAILS_ENV=test DISABLE_BOOTSNAP=1 \
206
- HAILER_SMOKE_QUEUED=1 \
207
- HAILER_SMOKE_BINARY=/absolute/path/to/current/hailer \
208
- HAILER_DATABASE_URL=postgres://...@127.0.0.1:5432/hailer_smoke_queued \
209
- bundle exec rails runner /absolute/path/to/hailer/test/commonshare_smoke.rb
210
- ```
211
-
212
- An isolated temporary `BUNDLE_GEMFILE` can select the working-tree gem or locally
213
- installed verification gem without editing CommonShare's Gemfile or lock.
214
- Set `HAILER_SMOKE_EXPECTED_PLATFORM=arm64-darwin` to assert native-gem selection.
215
- Omit `HAILER_SMOKE_QUEUED` for the unchanged synchronous contract. The test holds
216
- the provider response until Rails has its receipt, then checks the journal,
217
- correlation tags, envelope/attachment, body purge, replay, conflict and feeds.