bouncy 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +24 -0
- data/README.md +185 -93
- data/guides/README.md +20 -0
- data/guides/admin.md +2 -2
- data/guides/amazon-ses.md +70 -9
- data/guides/api.md +129 -0
- data/guides/bounce-handling.md +38 -1
- data/guides/compatibility.md +16 -2
- data/guides/configuration.md +83 -0
- data/guides/delivery.md +46 -8
- data/guides/migrating.md +3 -3
- data/guides/operations.md +104 -0
- data/guides/privacy.md +1 -1
- data/guides/troubleshooting.md +1 -1
- data/lib/bouncy/address_lock.rb +321 -0
- data/lib/bouncy/body_limit.rb +55 -0
- data/lib/bouncy/configuration.rb +38 -2
- data/lib/bouncy/event.rb +8 -3
- data/lib/bouncy/ingestor.rb +5 -3
- data/lib/bouncy/interceptor.rb +85 -16
- data/lib/bouncy/notification.rb +70 -0
- data/lib/bouncy/providers/ses.rb +57 -2
- data/lib/bouncy/providers/ses_parser.rb +20 -0
- data/lib/bouncy/providers/sns_verifier.rb +74 -6
- data/lib/bouncy/prune_job.rb +3 -0
- data/lib/bouncy/scope_lock.rb +3 -2
- data/lib/bouncy/store.rb +4 -0
- data/lib/bouncy/sync_job.rb +4 -0
- data/lib/bouncy/version.rb +1 -1
- data/lib/bouncy/webhook.rb +60 -9
- data/lib/bouncy.rb +34 -2
- data/lib/generators/bouncy/templates/initializer.rb.tt +14 -0
- metadata +13 -3
data/guides/amazon-ses.md
CHANGED
|
@@ -4,7 +4,7 @@ Use this when a Rails mailer succeeds but SES refuses delivery because an addres
|
|
|
4
4
|
|
|
5
5
|
## Supported sending policy
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
Bouncy supports one AWS account, one region and account-level suppression of both bounces and complaints. Configure `ses:<account>:<region>:account`. The adapter verifies the AWS caller account and SES client region, reads the enabled account reasons, and checks suppression overrides on supplied configuration sets, including `config.ses.configuration_set` and defaults found on configured identities.
|
|
8
8
|
|
|
9
9
|
List all sending identities and configuration sets, then set `all_sending_paths_listed = true` to confirm the lists are complete. Dynamic per-message overrides, SES tenants and other accounts are not inferred from a default mailer. Unknown or conflicting policy leaves sync in observation mode: the list is mirrored and provider-derived interception is suspended. Historical restrictions and independent manual holds remain. Read `policy_reason` in doctor, the sync summary or `Bouncy.status(address)` for the explanation. Bouncy cannot discover every future sending decision.
|
|
10
10
|
|
|
@@ -12,8 +12,8 @@ List all sending identities and configuration sets, then set `all_sending_paths_
|
|
|
12
12
|
|
|
13
13
|
1. Configure the exact topic allowlist and mount `/bouncy`. Deploy `POST /bouncy/webhooks/ses` before subscribing.
|
|
14
14
|
2. Review the topic's publisher policy. Allow the SES service principal only with the intended `AWS:SourceAccount` and `AWS:SourceArn` conditions. Preserve existing statements and destinations. A signature alone is insufficient if unrelated publishers can write to the authorized topic.
|
|
15
|
-
3. Subscribe the public HTTPS URL. Use normal SNS envelopes, not raw message delivery. Bouncy verifies the signature and confirms with the authenticated topic and token.
|
|
16
|
-
4. Connect bounce and complaint feedback through existing identity notification settings or configuration-set destinations. Inspect existing forwarding/defaults before changing them. Delivery events are optional.
|
|
15
|
+
3. Subscribe the public HTTPS URL. Use normal SNS envelopes, not raw message delivery. Bouncy verifies the signature and confirms with the authenticated topic and token through `sns:ConfirmSubscription`. With `config.ses.subscription_confirmation = :subscribe_url` it visits the signed `SubscribeURL` instead: only an HTTPS URL on the topic's own regional SNS host, with no redirects, and only a 2xx answer counts. Any other answer or a transport failure answers 503, so SNS sends the confirmation again.
|
|
16
|
+
4. Connect bounce and complaint feedback through existing identity notification settings or configuration-set destinations. Inspect existing forwarding/defaults before changing them. Delivery events are optional. To route every message through one configuration set, set `config.ses.configuration_set`; the interceptor stamps `X-SES-CONFIGURATION-SET` on each message without one.
|
|
17
17
|
5. Confirm the subscription is active and test feedback. Configure suitable SNS retries and optionally a dead-letter queue. Sync cannot reconstruct lost delivery or soft-bounce history.
|
|
18
18
|
6. Run `bouncy:doctor` and `bouncy:sync`. Schedule `Bouncy::SyncJob` hourly and `Bouncy::PruneJob` daily. Check that workers actually execute them.
|
|
19
19
|
7. Test a mixed-recipient message in staging. Enable `:drop` after reviewing scope and imported restrictions.
|
|
@@ -24,12 +24,71 @@ List all sending identities and configuration sets, then set `all_sending_paths_
|
|
|
24
24
|
|
|
25
25
|
Add `aws-sdk-sesv2`, `aws-sdk-sns`, and `aws-sdk-sts` to the host bundle. These load lazily.
|
|
26
26
|
|
|
27
|
-
Runtime reads use `ses:ListSuppressedDestinations`, `ses:GetSuppressedDestination`, `ses:GetAccount`, `sts:GetCallerIdentity`, plus `ses:GetEmailIdentity` and `ses:GetConfigurationSet` for configured policies. Recovery adds `ses:DeleteSuppressedDestination`. Confirmation uses `sns:ConfirmSubscription`; doctor uses `sns:GetTopicAttributes`. Restrict resource-scoped actions to intended resources where AWS supports it. This release never calls `PutSuppressedDestination` or provisioning APIs.
|
|
27
|
+
Runtime reads use `ses:ListSuppressedDestinations`, `ses:GetSuppressedDestination`, `ses:GetAccount`, `sts:GetCallerIdentity`, plus `ses:GetEmailIdentity` and `ses:GetConfigurationSet` for configured policies. Recovery adds `ses:DeleteSuppressedDestination`. Confirmation uses `sns:ConfirmSubscription`, unless `subscription_confirmation` is `:subscribe_url`; doctor uses `sns:GetTopicAttributes`. Restrict resource-scoped actions to intended resources where AWS supports it. This release never calls `PutSuppressedDestination` or provisioning APIs.
|
|
28
28
|
|
|
29
29
|
Set `config.ses.credentials` to an AWS credential provider to share the same credentials across SES, SNS and STS. For `aws-actionmailer-ses`, pass the credential object and region from the mailer's `ses_settings`; the AWS default chain does not read Rails encrypted credentials. [AWS credential-provider documentation](https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/credential-providers.html).
|
|
30
30
|
|
|
31
31
|
Inject SDK clients through `config.ses.client`, `sns_client`, and `sts_client` if needed. Keep account and region consistent. Default clients use bounded connection/read timeouts and retries. Never print credentials in setup output.
|
|
32
32
|
|
|
33
|
+
### Rails encrypted credentials
|
|
34
|
+
|
|
35
|
+
When the mailer already builds an AWS credential provider, share that object with Bouncy. Otherwise, for a host storing an `aws` section in encrypted credentials:
|
|
36
|
+
|
|
37
|
+
```ruby
|
|
38
|
+
# config/initializers/bouncy.rb, alongside the scope/topic configuration
|
|
39
|
+
require "aws-sdk-sesv2"
|
|
40
|
+
|
|
41
|
+
settings = Rails.application.credentials.fetch(:aws)
|
|
42
|
+
Bouncy.configure do |config|
|
|
43
|
+
config.ses.region = settings.fetch(:region)
|
|
44
|
+
config.ses.credentials = Aws::Credentials.new(
|
|
45
|
+
settings.fetch(:access_key_id),
|
|
46
|
+
settings.fetch(:secret_access_key),
|
|
47
|
+
settings[:session_token]
|
|
48
|
+
)
|
|
49
|
+
end
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
These are example credential key names; use your app's existing schema. Prefer the SDK credential chain when your deployment supplies role credentials that refresh automatically. A static `Aws::Credentials` object will not refresh expiring session credentials. Never copy SMTP passwords into these settings.
|
|
53
|
+
|
|
54
|
+
### Permission checklist
|
|
55
|
+
|
|
56
|
+
The following are the API actions the shipped adapter calls. Grant them to the app's API identity, independently of the identity used to send through SMTP:
|
|
57
|
+
|
|
58
|
+
| Action | Used for |
|
|
59
|
+
|---|---|
|
|
60
|
+
| `ses:ListSuppressedDestinations` | Full provider enumeration for sync and release |
|
|
61
|
+
| `ses:GetSuppressedDestination` | Exact-identifier absence checks |
|
|
62
|
+
| `ses:GetAccount` | Account suppression reasons |
|
|
63
|
+
| `ses:GetEmailIdentity` | Default configuration sets on each configured identity |
|
|
64
|
+
| `ses:GetConfigurationSet` | Suppression overrides on explicit and discovered default sets |
|
|
65
|
+
| `ses:DeleteSuppressedDestination` | Explicit provider recovery only |
|
|
66
|
+
| `sns:ConfirmSubscription` | Confirm the authorized receiver subscription (not needed with `:subscribe_url`) |
|
|
67
|
+
| `sns:GetTopicAttributes` | Doctor's topic readability/publisher-policy review information |
|
|
68
|
+
| `sts:GetCallerIdentity` | Verify the credentials' account |
|
|
69
|
+
|
|
70
|
+
Scope grants to the intended region and resources where supported. Account-level SES actions need account-level access; a send-only grant on a domain identity is insufficient. STS caller identity is an identity lookup, not proof of any SES/SNS grant. A read-only role can run observation/sync with the required reads, but cannot perform provider recovery. Doctor does not prove deletion permission.
|
|
71
|
+
|
|
72
|
+
### Topic publisher policy
|
|
73
|
+
|
|
74
|
+
The runtime IAM identity and the SNS topic publisher policy are separate. For identity-level SES feedback, review a statement like this on your intended topic, replacing every example value:
|
|
75
|
+
|
|
76
|
+
```json
|
|
77
|
+
{
|
|
78
|
+
"Sid": "AllowIntendedSesIdentity",
|
|
79
|
+
"Effect": "Allow",
|
|
80
|
+
"Principal": { "Service": "ses.amazonaws.com" },
|
|
81
|
+
"Action": "sns:Publish",
|
|
82
|
+
"Resource": "arn:aws:sns:us-east-1:123456789012:feedback",
|
|
83
|
+
"Condition": {
|
|
84
|
+
"StringEquals": { "AWS:SourceAccount": "123456789012" },
|
|
85
|
+
"ArnEquals": { "AWS:SourceArn": "arn:aws:ses:us-east-1:123456789012:identity/example.com" }
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
This is one statement to merge into a reviewed policy, not a replacement for the whole policy. Check other statements for unrelated publisher access. Configuration-set event destinations need the source conditions appropriate to that destination. Follow [AWS's SNS notification setup](https://docs.aws.amazon.com/ses/latest/dg/configure-sns-notifications.html) for identity notifications and preserve existing destinations. Review the [account suppression settings](https://docs.aws.amazon.com/ses/latest/dg/sending-email-suppression-list.html) before asserting that both bounce and complaint suppression apply to every sending path.
|
|
91
|
+
|
|
33
92
|
## Testing your mounted receiver
|
|
34
93
|
|
|
35
94
|
A host test that posts to the mounted route would otherwise download Amazon's signing certificate over the network. `config.ses.sns_message_verifier` injects the signature verifier. The following test subclass changes only certificate retrieval while retaining RSA verification:
|
|
@@ -44,19 +103,21 @@ end
|
|
|
44
103
|
Bouncy.configuration.ses.sns_message_verifier = LocalCertificate.new
|
|
45
104
|
```
|
|
46
105
|
|
|
47
|
-
Topic authorization
|
|
106
|
+
Topic authorization and certificate-URL checks run outside the injected verifier. This example also retains the SDK's RSA signature check. The injected object itself is trusted: replacing it with a verifier that always succeeds would skip cryptographic authentication, so keep the default in production. Sign fixtures with a locally generated key and assert that a valid signature on an unlisted `TopicArn` is rejected.
|
|
48
107
|
|
|
49
108
|
## Failure and ordering
|
|
50
109
|
|
|
51
|
-
The webhook commits each recipient's event and address transition before acknowledging. Retries deduplicate independently. Authentication failure returns 401, malformed input 400, oversized bodies 413 and recognized transient provider/database failures 503.
|
|
110
|
+
The webhook commits each recipient's event and address transition before acknowledging, then calls `config.on_notification`. Retries deduplicate independently. A raise in that hook answers 503. Authentication failure returns 401, malformed input 400, oversized bodies 413 and recognized transient provider/database failures 503.
|
|
111
|
+
|
|
112
|
+
Bodies are bounded to 2 MiB before parsing. Rails reads a whole chunked body, and parses a JSON body for its log, before the route runs, so insert `Bouncy::BodyLimit` (300 KB by default) first in the middleware stack and set a corresponding proxy limit. See [receiver body limit](configuration.md#receiver-body-limit). Certificate retrieval permits strict regional SNS HTTPS URLs, verifies TLS, rejects redirects and caps responses. Transport outages remain retryable. A fetched certificate is kept in memory for the process, for at most 16 URLs and for one hour each, and only once it parses as an X.509 certificate with an RSA key; any other body is refused and never kept, so an app that receives one notification per mail does not download the certificate per mail. When a signature fails with a cached certificate fetched more than a minute ago, the certificate is fetched again once, which covers a rotation at the same URL; the allowlist and the download limits apply to every fetch.
|
|
52
113
|
|
|
53
|
-
|
|
114
|
+
Each refused message logs one warning, `[bouncy] refused an SNS message (<status>): <reason>`, followed by the topic ARN it names and, for a subscription confirmation, the `SubscribeURL` host. The message is not authenticated, so the topic is written only when it has strict SNS ARN syntax and the host only when it passes the SNS host check; otherwise the line says `<invalid>`. No raw field of the body, address or token is ever logged, and a database error logs only its class.
|
|
54
115
|
|
|
55
|
-
Sync enumerates all pages without a time filter. Failure or unsupported policy is not a successful empty list. Clearing missing restrictions requires complete enumeration, exact identifier checks and unchanged local state. Provider pagination is not an atomic snapshot; remote writers can change state after any observation. Inspect freshness and failures.
|
|
116
|
+
Sync enumerates all pages without a time filter. Failure or unsupported policy is not a successful empty list. Clearing missing restrictions requires complete enumeration, exact identifier checks and unchanged local state. Webhook-only blocking evidence is not cleared on a single absent list entry: it must be over an hour old and absent across two qualifying checks. Independent manual and soft holds are preserved. Provider pagination is not an atomic snapshot; remote writers can change state after any observation. Inspect freshness and failures.
|
|
56
117
|
|
|
57
118
|
SES management addresses are case-sensitive. Bouncy retains exact spelling independently of lowercase local lookup keys and releases all observed variants. A lowercase NotFound does not establish that another case variant is absent.
|
|
58
119
|
|
|
59
|
-
Ordinary complaint notifications can name candidate recipients when the mailbox provider redacts the complainer. Bouncy blocks locally only when exactly one recipient is named; otherwise it records candidates and lets the next complete sync establish the restriction from the provider's own list. Account-list refusal events are hints rather than new mailbox failures. Complaint subtypes `OnAccountSuppressionList` and `OnTenantSuppressionList` describe existing suppression, not a new complaint: the former is a provider-list hint, the latter an unsupported-scope diagnostic. Unknown complaint subtypes remain diagnostics. These distinctions follow the [SES notification contract](https://docs.aws.amazon.com/ses/latest/dg/notification-contents.html). Allow up to the next scheduled sync for account-list hints in the initial implementation.
|
|
120
|
+
Ordinary complaint notifications can name candidate recipients when the mailbox provider redacts the complainer. With `webhook_blocking = :cautious`, Bouncy blocks locally only when exactly one recipient is named; otherwise it records candidates and lets the next complete sync establish the restriction from the provider's own list. With `:strict`, the default without a sync, every named recipient and every Permanent bounce blocks; see [bounce handling](bounce-handling.md#ses-event-classification). Account-list refusal events are hints rather than new mailbox failures. Complaint subtypes `OnAccountSuppressionList` and `OnTenantSuppressionList` describe existing suppression, not a new complaint: the former is a provider-list hint, the latter an unsupported-scope diagnostic. Unknown complaint subtypes remain diagnostics. These distinctions follow the [SES notification contract](https://docs.aws.amazon.com/ses/latest/dg/notification-contents.html). Allow up to the next scheduled sync for account-list hints in the initial implementation.
|
|
60
121
|
|
|
61
122
|
Reconciliation is idempotent: an identical address observation refreshes its check time without address events, hooks or version bumps. Changed provider timestamps are saved and advance the row version so concurrent recovery cannot clear newer evidence; they create no restriction event when the address and reason are unchanged. Every run still records a sync summary. Partial lists can refresh listed variants but cannot erase missing ones. Rows with neither provider evidence nor a webhook-derived block are never looked up at the provider. Only one sync runs per scope at a time; a second one fails immediately with `Bouncy::ProviderError` instead of queueing behind a run that may be waiting on AWS.
|
|
62
123
|
|
data/guides/api.md
ADDED
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
# API reference
|
|
2
|
+
|
|
3
|
+
All operations use the configured sending scope. Addresses are compared after trimming whitespace and lowercasing; exact provider identifiers are retained separately for SES management calls. This is a lookup policy, not mailbox verification. Invalid input to a single-address lookup raises `Bouncy::InvalidAddress` when configured.
|
|
4
|
+
|
|
5
|
+
## Address status
|
|
6
|
+
|
|
7
|
+
```ruby
|
|
8
|
+
status = Bouncy.status(" Ada@Example.com ")
|
|
9
|
+
Bouncy.blocked?("ada@example.com") # Same as status.blocked?
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
| Reader | Meaning |
|
|
13
|
+
|---|---|
|
|
14
|
+
| `blocked?` | Any effective local restriction exists. Does not incorporate the interceptor's mode or global sync freshness. |
|
|
15
|
+
| `reasons` | Array containing any of `:complaint`, `:hard_bounce`, `:provider_list`, `:manual`, `:soft_bounces`. |
|
|
16
|
+
| `reason` | First reason in the priority order above, or `nil`. |
|
|
17
|
+
| `knowledge` | `:observed` when a row exists; `:no_known_block` when none exists; `:unavailable` on recognized database outages; `:unconfigured` when no scope is set. An observed row can be inactive after release. |
|
|
18
|
+
| `email` | Normalized address from the row, or `nil` if there is no row. |
|
|
19
|
+
| `scope`, `provider` | Configuration captured when the status was constructed. |
|
|
20
|
+
| `record` | Underlying `Bouncy::Suppression`, or `nil`, for host-owned admin links. |
|
|
21
|
+
| `provider_listed?` | Exact provider entries exist in the local mirror, even if policy verification has not made them enforceable. |
|
|
22
|
+
| `provider_updated_at` | Latest update time among those provider entries, or `nil`. |
|
|
23
|
+
| `observed_at` | Provider check time if present, otherwise the last address event time, or `nil`. |
|
|
24
|
+
| `stale?` | True for unavailable/unconfigured knowledge, missing observation time or an observation older than `stale_after`. This is address freshness; use `Bouncy.sync_fresh?` for mirror health. |
|
|
25
|
+
| `policy_unverified?` | A locally listed address whose latest scoped sync did not report verified sending policy. |
|
|
26
|
+
| `policy_reason` | Explanation for that unverified policy; `nil` when verified or not listed locally. |
|
|
27
|
+
| `last_event` | Latest address event by occurrence time and ID, or `nil`; fetched lazily. |
|
|
28
|
+
| `release_supported?` | Whether the available status uses the supported SES provider. Does not test IAM rights or authorize an operator. |
|
|
29
|
+
|
|
30
|
+
A false boolean is not a deliverability guarantee. Recognized database connection outages return unavailable status; missing tables, invalid SQL and programming errors still raise. Obtain a new status after a state change or sync; existing objects are not live subscriptions.
|
|
31
|
+
|
|
32
|
+
## Batch lookup
|
|
33
|
+
|
|
34
|
+
```ruby
|
|
35
|
+
statuses = Bouncy.statuses(users.map(&:email))
|
|
36
|
+
statuses["ADA@example.com"].reasons
|
|
37
|
+
statuses.each { |email, status| puts [email, status.knowledge].join(": ") }
|
|
38
|
+
statuses.size
|
|
39
|
+
statuses.to_h
|
|
40
|
+
statuses.blocked # Hash containing only blocked entries
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
`Bouncy::StatusSet` includes `Enumerable`. Inputs are normalized and deduplicated; invalid entries are skipped. One query loads restriction rows. Accessing `last_event` or policy diagnostics can issue additional queries.
|
|
44
|
+
|
|
45
|
+
A lookup for an address not included in the batch does not query the database and returns no-known-block status. Include every address you need to inspect. When unconfigured or unavailable, each requested valid address remains in the enumeration with that knowledge, and lookups return it too.
|
|
46
|
+
|
|
47
|
+
## Model integration
|
|
48
|
+
|
|
49
|
+
```ruby
|
|
50
|
+
class User < ApplicationRecord
|
|
51
|
+
normalizes :email, with: ->(value) { value.strip.downcase }
|
|
52
|
+
bouncy :email
|
|
53
|
+
normalizes :billing_email, with: ->(value) { value.strip.downcase }
|
|
54
|
+
bouncy :billing_email
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
user.email_status
|
|
58
|
+
user.email_blocked?
|
|
59
|
+
user.email_bounced? # Hard bounce or active soft-bounce hold
|
|
60
|
+
user.email_complained?
|
|
61
|
+
|
|
62
|
+
User.email_blocked
|
|
63
|
+
User.email_bounced
|
|
64
|
+
User.email_unblocked
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
Each configured attribute gets the same methods with its own prefix. There is no `email_complained` class scope. Blank values are false for the boolean predicates and excluded from the unblocked scope; a direct `email_status` call still follows single-address input validation.
|
|
68
|
+
|
|
69
|
+
SQL scopes expect stored values to use the same trimmed lowercase comparison. Normalize every configured attribute and backfill existing data in the host. If you preserve mixed-case display values, use a persisted comparison column:
|
|
70
|
+
|
|
71
|
+
```ruby
|
|
72
|
+
bouncy :email, normalized_attribute: :canonical_email
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
The override applies to status and predicates as well as scopes. Your app must populate that column correctly. A normalized override accepts one attribute per macro call. No model integration is required for recipients without a host record.
|
|
76
|
+
|
|
77
|
+
## Queries and mutations
|
|
78
|
+
|
|
79
|
+
| Method | Behavior |
|
|
80
|
+
|---|---|
|
|
81
|
+
| `Bouncy.blocked` | Scoped ActiveRecord relation of currently restricted suppression rows. |
|
|
82
|
+
| `Bouncy.events` | Scoped event relation; chain `.for(address).recent.limit(50)`. |
|
|
83
|
+
| `Bouncy.block!(address, note:, actor: nil)` | Add a local manual hold; a nonblank note is required. Returns the suppression row. Does not call SES. |
|
|
84
|
+
| `Bouncy.release!(address, note: nil, actor: nil, at: :provider)` | Provider-aware release by default, requiring a note and verified provider snapshot. Returns the row; may raise on partial remote success. |
|
|
85
|
+
| `Bouncy.release!(address, at: :local)` | Clear manual and soft holds and their soft-bounce history; fence delayed pre-release soft feedback and preserve provider/webhook blocks. No provider request. A note is optional. |
|
|
86
|
+
| `Bouncy.forget!(address)` | Delete local state and events, including release metadata; no provider mutation. A later sync can reimport the address. |
|
|
87
|
+
| `Bouncy.sync!` | Run reconciliation now and return its summary event. Writes local state and reads AWS. |
|
|
88
|
+
| `Bouncy.bootstrap!` | Return a fresh complete sync, running one if needed; raise if freshness and policy cannot be established. Explicit cutover check. |
|
|
89
|
+
| `Bouncy.last_sync` | Latest sync summary, including recorded failures, or `nil`. |
|
|
90
|
+
| `Bouncy.last_successful_sync` | Latest summary marked complete, possibly old, or `nil`. |
|
|
91
|
+
| `Bouncy.sync_fresh?` | True only when the latest summary is complete and within `stale_after`. A later recorded failure makes it false. |
|
|
92
|
+
| `Bouncy.configured?` | Whether a nonblank scope is assigned; no AWS verification. |
|
|
93
|
+
| `Bouncy.scope` | Configured scope after local configuration validation. |
|
|
94
|
+
| `Bouncy.unblocked { ... }` | Temporarily bypass interception for synchronous delivery in this execution context. Does not travel with queued jobs. |
|
|
95
|
+
| `Bouncy.hold(address) { ... }` | Hold the address while the block runs, and inside a transaction until it ends, so no Bouncy write to it commits between the app's own check and its send. Returns the block's value. See [a sender that checks and sends](delivery.md#a-sender-that-checks-and-sends-in-one-step). |
|
|
96
|
+
|
|
97
|
+
## SES notifications
|
|
98
|
+
|
|
99
|
+
`config.on_notification` receives a `Bouncy::Notification` for every SES notification on an authorized topic, Send and Delivery included:
|
|
100
|
+
|
|
101
|
+
| Reader | Meaning |
|
|
102
|
+
|---|---|
|
|
103
|
+
| `type` | The SES event type as SES names it: `"Send"`, `"Delivery"`, `"Bounce"`, `"Complaint"`, `"DeliveryDelay"`, ... |
|
|
104
|
+
| `message_id` | The SES message id of the mail, or `nil`. |
|
|
105
|
+
| `recipients` | The addresses the event is about, exactly as SES spells them: bounced, complained, delivered or delayed recipients; the mail's destination for other types. |
|
|
106
|
+
| `occurred_at` | The event's own timestamp, else the mail's, else the time SNS published it. |
|
|
107
|
+
| `tags` | The mail's SES message tags: each name maps to an array of values. `{}` when there are none. |
|
|
108
|
+
| `raw` | The SES event as a Hash, for any other field. |
|
|
109
|
+
|
|
110
|
+
The hook runs after Bouncy committed what it records from the notification, outside any transaction or Bouncy lock, and before the receiver answers. A raise answers 503 and SNS delivers the notification again: Bouncy's own records deduplicate, so make the hook safe to repeat. Subscription confirmations and refused messages never reach it.
|
|
111
|
+
|
|
112
|
+
Use service methods for changes instead of editing or deleting suppression rows directly. Their independent evidence and release metadata are part of the consistency contract. Relations from `Bouncy.blocked` and `Bouncy.events` are empty when unconfigured, but explicit operations raise. See [recovery](recovery.md) and [operations](operations.md).
|
|
113
|
+
|
|
114
|
+
## Errors
|
|
115
|
+
|
|
116
|
+
All gem errors inherit from `Bouncy::Error`:
|
|
117
|
+
|
|
118
|
+
| Error | Meaning / response |
|
|
119
|
+
|---|---|
|
|
120
|
+
| `InvalidAddress` | Invalid lookup/mutation/hold input; correct the address. |
|
|
121
|
+
| `ConfigurationError` | Missing or invalid setup; fix configuration or dependencies. Also `Bouncy.hold` inside a PostgreSQL REPEATABLE READ or SERIALIZABLE transaction; hold around the transaction instead. |
|
|
122
|
+
| `ProviderError` | Provider/transport failure, or a sync lock already held. Inspect and retry through your job policy. |
|
|
123
|
+
| `UnsafeSnapshot < ProviderError` | Scope mismatch or inability to establish a safe bootstrap/release snapshot. |
|
|
124
|
+
| `ReleaseFailed < ProviderError` | Partial/failed provider release; inspect `error.outcomes` before retrying. |
|
|
125
|
+
| `ReleaseConflict` | Local evidence changed during release; reload and review the newer state. |
|
|
126
|
+
| `AuthenticationError` | SNS authorization/signature failure. |
|
|
127
|
+
| `MalformedMessage` | Invalid SNS input. |
|
|
128
|
+
|
|
129
|
+
Missing support notes, unsupported release destinations and `Bouncy.hold` without a block raise `ArgumentError`. An address lock that waits too long raises `ActiveRecord::LockWaitTimeout`. Rails/database errors not recognized as connection outages are not silently swallowed. A successful local database transaction does not make remote provider operations atomic. Never display a recovery success message from a rescue branch.
|
data/guides/bounce-handling.md
CHANGED
|
@@ -12,7 +12,44 @@ No User record is needed. Optional `bouncy :email` model integration adds predic
|
|
|
12
12
|
|
|
13
13
|
Start with [SES setup](amazon-ses.md). Both SES SMTP and Action Mailer SDK delivery can use the same management API mirror. API credentials are separate from SMTP credentials. Import historical restrictions before enabling delivery interception.
|
|
14
14
|
|
|
15
|
-
Webhooks handle new events promptly; recurring full sync covers prior history, shared-account changes, missed notifications and console releases. Neither mechanism proves every mailbox is reachable. Soft bounces are record-only unless you configure a threshold, an ordinary complaint blocks locally only when it names a single recipient, and
|
|
15
|
+
Webhooks handle new events promptly; recurring full sync covers prior history, shared-account changes, missed notifications and console releases. Neither mechanism proves every mailbox is reachable. Soft bounces are record-only unless you configure a threshold, and message-size failures never globally invalidate an address. While a sync runs, an ordinary complaint blocks locally only when it names a single recipient, and suppression-list refusal notices are recorded separately from new complaints. Without a sync, `config.webhook_blocking` is `:strict`: see the next section.
|
|
16
|
+
|
|
17
|
+
## SES event classification
|
|
18
|
+
|
|
19
|
+
The adapter supports identity `notificationType` and configuration-set `eventType` envelopes. It records the normalized kind separately from the provider subtype. With `config.webhook_blocking = :cautious`, the default while a sync runs:
|
|
20
|
+
|
|
21
|
+
| SES feedback | Local kind / effect |
|
|
22
|
+
|---|---|
|
|
23
|
+
| Permanent bounce, `General` or `NoEmail` | `hard_bounce`; records a webhook-derived block. |
|
|
24
|
+
| Transient or Undetermined bounce | `soft_bounce`; record-only unless eligible mailbox-full escalation is enabled. |
|
|
25
|
+
| `OnAccountSuppressionList` bounce or complaint | `provider_suppressed`; evidence of a prior refusal, not a new hard bounce/complaint block. Sync imports the list's reason. |
|
|
26
|
+
| `Suppressed`, `OnTenantSuppressionList`, `EmailValidationSuppressed` bounce | `unknown`; no inferred account-level block. |
|
|
27
|
+
| `UnsubscribedRecipient` bounce | `unsubscribe`; no global block or subscription management. |
|
|
28
|
+
| Ordinary complaint with exactly one named recipient | `complaint` with confirmed certainty; records a local block. |
|
|
29
|
+
| Ordinary complaint naming several recipients | `complaint` candidates; no block until the provider list confirms the address. |
|
|
30
|
+
| Complaint marked `not-spam` | `ignored`; does not automatically release existing evidence. |
|
|
31
|
+
| Other/unrecognized bounce or complaint subtype | `unknown`; retained for investigation. |
|
|
32
|
+
| Delivery | `delivery` per recipient only when `record_deliveries` is enabled; otherwise an addressless `ignored` observation. Does not release a block. |
|
|
33
|
+
| DeliveryDelay | `delay`; no mailbox-full escalation or global block. |
|
|
34
|
+
| Reject / Rendering Failure | Addressless `reject` / `rendering_failure` event; no global address restriction. |
|
|
35
|
+
| Other event types | `ignored`; no open/click tracking or subscription handling. |
|
|
36
|
+
|
|
37
|
+
With `config.webhook_blocking = :strict`, the default when `require_fresh_sync` is `false`, nothing reads the account list, so the webhook decides alone:
|
|
38
|
+
|
|
39
|
+
| SES feedback | Local kind / effect |
|
|
40
|
+
|---|---|
|
|
41
|
+
| Permanent bounce, any subtype (`General`, `NoEmail`, `Suppressed`, `OnAccountSuppressionList`, `OnTenantSuppressionList`, `UnsubscribedRecipient`, `EmailValidationSuppressed`, a missing or future one) | `hard_bounce`; records a webhook-derived block. The subtype stays in the event's `provider_reason`. |
|
|
42
|
+
| Transient or Undetermined bounce | `soft_bounce`; record-only, as above. An undetermined bounce can come from a mailbox that works tomorrow, and nothing would lift its block. |
|
|
43
|
+
| Complaint, with or without a subtype | `complaint`; every named recipient is blocked. The event keeps `certainty`: `confirmed` for one named recipient, `candidate` for several. |
|
|
44
|
+
| Complaint marked `not-spam` | `ignored`, as above. |
|
|
45
|
+
|
|
46
|
+
SES cannot say who complained. Its [documentation](https://docs.aws.amazon.com/ses/latest/dg/notification-contents.html#complained-recipients) states: "Most ISPs remove the email address of the recipient who submitted the complaint from their complaint notification. For this reason, this list contains information about recipients who might have sent the complaint, based on the recipients of the original message and the ISP from which we received the complaint." No field tells a named complainer from a list that SES worked out, so `:strict` blocks every listed recipient rather than risk mailing the one who reported you.
|
|
47
|
+
|
|
48
|
+
The receiving server's acceptance is not evidence of inbox placement. See [AWS's notification fields](https://docs.aws.amazon.com/ses/latest/dg/notification-contents.html) for provider payload definitions.
|
|
49
|
+
|
|
50
|
+
Invalid recipients cannot acquire a hold. Old, future-dated (more than five minutes ahead), out-of-order blocking feedback and feedback fenced by a release are retained with `details["ignored_for_policy"]` rather than applying a new restriction. Soft events can arrive out of order within their active window. Retries use a dedupe identity that includes scope, provider, feedback ID, kind and recipient.
|
|
51
|
+
|
|
52
|
+
Blocking evidence is separate from enforcement: even a confirmed hard bounce requires a fresh authoritative sync for provider-derived interception, unless `require_fresh_sync` is `false`. Manual and active soft holds are independent local policy. See [delivery](delivery.md).
|
|
16
53
|
|
|
17
54
|
## Repeated soft bounces
|
|
18
55
|
|
data/guides/compatibility.md
CHANGED
|
@@ -2,7 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
Bouncy's compatibility suite covers Ruby 3.3, 3.4 and 4.0 with Rails 7.2, 8.0 and 8.1. Each combination runs on SQLite, PostgreSQL and MySQL with bigint and UUID keys. Rails 7.2 uses Minitest 5 because Active Support constrains that dependency; Rails 8 uses Minitest 6.
|
|
4
4
|
|
|
5
|
-
The September
|
|
5
|
+
The v0.1.0 release run on September 9, 2026 passed all 54 combinations using Ruby 3.3.5/3.4.7/4.0.5, Rails 7.2.3.2/8.0.5.1/8.1.3.1, PostgreSQL 18.3 and MySQL 8.4.11. Every combination passed the 90% line and branch coverage gates (131 tests on SQLite, 135 on PostgreSQL/MySQL). A fresh Rails app installed from the release archive also passed generator/migration, model/batch APIs, normal mail interception, local recovery and mounted receiver checks. The [first public GitHub Actions run](https://github.com/rameerez/bouncy/actions/runs/34306365294) also passed all 54 matrix jobs plus lint/package build. These automated checks do not establish a live SES lifecycle. Live SES lifecycle and independent installation validation remain outstanding for this first release.
|
|
6
|
+
|
|
7
|
+
The v0.2.0 local run on October 25, 2026 passed all 54 combinations with the same Ruby, Rails, PostgreSQL and MySQL versions and SQLite 3.54.1, and the same 18 again with Rails 8.0.2, the oldest 8.0 release the transactional-fixture support was checked on: 201 tests on SQLite (one skipped there: a second pool cannot share an in-memory database), 220 on PostgreSQL and 219 on MySQL with Rails 8.1, one fewer with Rails 7.2 and 8.0, and at least 96.5% line and 91.5% branch coverage. It adds the address-lock tests with real connections on PostgreSQL and MySQL. No live SES run was part of it.
|
|
6
8
|
|
|
7
9
|
The runtime JSON gem is constrained below version 3 because the supported Rails releases still pass a positional options hash to `JSON.parse`. Optional AWS SDKs load lazily and are host dependencies. There is no database or AWS lookup during boot.
|
|
8
10
|
|
|
@@ -39,4 +41,16 @@ PostgreSQL uses a session advisory lock; MySQL uses a named connection lock. Bot
|
|
|
39
41
|
|
|
40
42
|
SQLite uses an OS file lock alongside the database file. All workers must access the same database file and lock path on a filesystem that supports `flock`. In-memory SQLite only exists inside its process and uses a process-specific lock file. Network filesystems and replicated SQLite arrangements need separate validation.
|
|
41
43
|
|
|
42
|
-
|
|
44
|
+
## Address locks
|
|
45
|
+
|
|
46
|
+
`Bouncy.hold` and every Bouncy write to an address share one lock per address, with no schema change. Writes take it exclusively, first in their transaction and until it ends; a hold takes it shared, for its block and, inside a transaction, until that transaction ends.
|
|
47
|
+
|
|
48
|
+
- **PostgreSQL** uses transaction-level advisory locks inside a transaction and a session lock for a hold outside one. The key is a signed 64-bit hash of the scope and the address. Waits follow `lock_timeout`. A failed statement aborts the transaction, and PostgreSQL then releases its locks at once. Inside a transaction a hold requires READ COMMITTED, the default, and raises `Bouncy::ConfigurationError` at REPEATABLE READ or SERIALIZABLE; hold the address around such a transaction instead.
|
|
49
|
+
- **MySQL** uses named `GET_LOCK` locks, which have no shared mode: senders to one address take turns. A wait lasts up to `innodb_lock_wait_timeout`, then raises `ActiveRecord::LockWaitTimeout`. Named locks are server-wide; the name includes the database. The lock statements skip the query cache. Reads inside a hold use `LOCK IN SHARE MODE`, so a REPEATABLE READ snapshot cannot hide a newer block.
|
|
50
|
+
- **SQLite** uses one `flock` file per database (`<database>.bouncy-addresses.lock`), shared or exclusive, held per connection. Threads that share one connection, as Rails tests do, share its lock. Hold the address around the transaction, or before its first read.
|
|
51
|
+
|
|
52
|
+
MySQL and SQLite locks belong to the connection, and three paths release them at the end of the outermost transaction. First, the `transaction.active_record` event: Rails sends it right after the COMMIT or ROLLBACK and before any `after_commit` or `after_rollback` callback, so a host callback that raises cannot keep a lock. Bouncy subscribes when it loads, but Rails gives a transaction the subscribers that exist when it starts, so a transaction already open then is not reported. Second, `after_commit` and `after_rollback` on the transaction itself cover that case; a savepoint that rolls back passes them up to its parent. Third, a connection that goes back to the pool releases whatever is left.
|
|
53
|
+
|
|
54
|
+
Bouncy's `after_*` hooks run after the commit of the outermost transaction, after its releases, and after the enclosing hold. A rollback drops them. Under the app's own `transaction(joinable: false)` they wait for the outer transaction. Rails transactional tests keep 0.1's timing. Bouncy marks the fixture transaction of each pool that `ActiveRecord::TestFixtures` enrolls, as listed in its own `@fixture_connection_pools`: at `setup_transactional_fixtures`, and for a pool established later from a `!connection.active_record` subscriber that runs after Rails' own. It clears the marks in `teardown_transactional_fixtures` (Rails 7.2, 8.0 and 8.1). While a marked transaction is the only one open, hooks run when the app's own transaction commits. A database excluded with `skip_transactional_tests_for_database` (Rails 8.1), or a connection pinned in any other way, keeps the outer-commit rule. Holds and the hooks that wait for them belong to the connection, so threads that share the fixture connection, as in system tests, share them too. The tests cover waiting, release at commit and rollback, the three release paths, host callbacks that raise, non-joinable transactions, real transactional fixtures, the query cache, a snapshot taken before the hold, PostgreSQL REPEATABLE READ, nested use in one connection and the race between a bounce and a send on PostgreSQL and MySQL with real connections.
|
|
55
|
+
|
|
56
|
+
Background scheduling remains host-owned. Action Mailer delivery, its bang methods and a correctly configured job worker are tested; custom senders have [explicit boundaries](delivery.md).
|
|
@@ -0,0 +1,83 @@
|
|
|
1
|
+
# Configuration reference
|
|
2
|
+
|
|
3
|
+
Configure Bouncy once in `config/initializers/bouncy.rb`, using the same settings in web and job processes. Configuration is process-wide; do not switch the sending scope per request or tenant. One installed instance supports one SES account and region. Host authorization remains separate.
|
|
4
|
+
|
|
5
|
+
```ruby
|
|
6
|
+
Bouncy.configure do |config|
|
|
7
|
+
config.scope = "ses:123456789012:us-east-1:account"
|
|
8
|
+
config.ses.region = "us-east-1"
|
|
9
|
+
config.ses.topic_arns = ["arn:aws:sns:us-east-1:123456789012:feedback"]
|
|
10
|
+
config.ses.identities = ["example.com"]
|
|
11
|
+
config.ses.configuration_sets = []
|
|
12
|
+
config.ses.all_sending_paths_listed = true
|
|
13
|
+
config.interception = :log
|
|
14
|
+
end
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
Replace the examples with your actual deployment. Set `all_sending_paths_listed` only after inventorying every sending identity and configuration set, including per-message selections. See [SES policy verification](amazon-ses.md#supported-sending-policy).
|
|
18
|
+
|
|
19
|
+
## Core settings
|
|
20
|
+
|
|
21
|
+
| Setting | Default | Meaning |
|
|
22
|
+
|---|---|---|
|
|
23
|
+
| `provider` | `:ses` | SES is the only built-in provider. |
|
|
24
|
+
| `scope` | `nil` | `ses:<account-id>:<region>:account`; at most 191 characters. Must match the AWS caller and SES client region. |
|
|
25
|
+
| `interception` | `:log` | `:log` previews dropping, `:drop` removes enforceable recipients, `:off` disables the interceptor. |
|
|
26
|
+
| `stale_after` | `2.hours` | Freshness window for complete syncs and address observations. Keep it positive and longer than your sync interval. |
|
|
27
|
+
| `require_fresh_sync` | `true` | `true` enforces provider and webhook evidence only while a complete sync is fresh. `false` enforces every recorded block with no sync, for an app that runs no `SyncJob`. |
|
|
28
|
+
| `webhook_blocking` | `:cautious` with `require_fresh_sync`, `:strict` without | Which notifications block. `:cautious` blocks General/NoEmail bounces and complaints naming one recipient; the sync covers the rest. `:strict` blocks every Permanent bounce and every recipient a complaint names. See [bounce handling](bounce-handling.md#ses-event-classification). |
|
|
29
|
+
| `record_deliveries` | `false` | Record normalized per-recipient SES Delivery events when enabled and routed to the topic. Server acceptance is not inbox placement. |
|
|
30
|
+
| `retention` | `90.days` | Delete events older than this by local creation time when the prune job runs. Does not delete state rows. |
|
|
31
|
+
| `maximum_event_age` | `90.days` | Feedback older than this cannot apply a new restriction. Retention must be at least this long; both must be positive. |
|
|
32
|
+
| `soft_bounce_threshold` | `nil` | Record-only by default. An integer from 1 to 50 enables escalation for repeated `MailboxFull` bounces. |
|
|
33
|
+
| `soft_bounce_window` | `30.days` | Positive rolling window of eligible occurrence times. |
|
|
34
|
+
| `soft_bounce_block_for` | `30.days` | Positive duration of the temporary local hold triggered by escalation. |
|
|
35
|
+
| `after_event` | No-op lambda | Called with every newly committed event. |
|
|
36
|
+
| `after_block` | No-op lambda | Called when an event records a transition to blocked. |
|
|
37
|
+
| `after_release` | No-op lambda | Called for explicit `release` events, including local-only releases. |
|
|
38
|
+
| `on_notification` | No-op lambda | Called with a `Bouncy::Notification` for every SES notification, after its state commits and before the receiver answers. A raise answers 503 so SNS retries. |
|
|
39
|
+
| `adapter` | `nil` | Advanced injection point, primarily for tests. The internal adapter contract is not a promise of support for other providers. |
|
|
40
|
+
|
|
41
|
+
The `after_*` callbacks take one `Bouncy::Event`. They are best effort; see [hook semantics](operations.md#hooks-and-instrumentation). Read current settings through `Bouncy.configuration`. `Bouncy.configure` assigns settings; `Bouncy.configuration.validate!` explicitly checks the local configuration without calling AWS. It raises on an unset scope or invalid supported options; it does not verify credentials or sending policy.
|
|
42
|
+
|
|
43
|
+
## SES settings
|
|
44
|
+
|
|
45
|
+
All are under `config.ses`:
|
|
46
|
+
|
|
47
|
+
| Setting | Default | Meaning |
|
|
48
|
+
|---|---|---|
|
|
49
|
+
| `region` | `nil` | AWS region matching the scope, SDK clients and SNS topics. |
|
|
50
|
+
| `credentials` | `nil` | AWS SDK credential provider shared by default SES, SNS and STS clients. `nil` uses the normal SDK chain. |
|
|
51
|
+
| `topic_arns` | `[]` | Exact allowed SNS topic ARNs. No wildcard or automatically trusted topic. |
|
|
52
|
+
| `identities` | `[]` | Every sending domain/address identity; default configuration sets are also checked. |
|
|
53
|
+
| `configuration_sets` | `[]` | Every explicitly selected configuration set. Empty is valid if none are used. |
|
|
54
|
+
| `all_sending_paths_listed` | `false` | Your assertion that the two lists are complete. Until true, policy verification cannot pass. |
|
|
55
|
+
| `configuration_set` | `nil` | Configuration set that the interceptor stamps as `X-SES-CONFIGURATION-SET` on every message without one, in every interception mode. It joins the policy check; you need not repeat it in `configuration_sets`. Blank stamps nothing. |
|
|
56
|
+
| `subscription_confirmation` | `:api` | `:api` confirms SNS subscriptions with `sns:ConfirmSubscription`. `:subscribe_url` visits the signed `SubscribeURL` instead and requires a 2xx answer. |
|
|
57
|
+
| `client` | `nil` | Optional injected `Aws::SESV2::Client`. |
|
|
58
|
+
| `sns_client` | `nil` | Optional injected `Aws::SNS::Client`. |
|
|
59
|
+
| `sts_client` | `nil` | Optional injected `Aws::STS::Client`. |
|
|
60
|
+
| `sns_message_verifier` | `nil` | Trusted signature-verifier replacement for offline tests. Keep the default in production. |
|
|
61
|
+
|
|
62
|
+
Default clients use a 3-second open timeout, 10-second read timeout and SDK retry limit of 2. These are per-request settings, not an overall sync deadline. Injected clients own their credentials/timeouts; `ses.credentials` does not override them. The bounded SNS certificate downloader and the `SubscribeURL` visit separately use a 3-second open timeout, 5-second read timeout, no redirects and no transport retries.
|
|
63
|
+
|
|
64
|
+
## Receiver body limit
|
|
65
|
+
|
|
66
|
+
`Bouncy::BodyLimit` is Rack middleware, not a setting. Insert it first, with the path your app routes to the receiver:
|
|
67
|
+
|
|
68
|
+
```ruby
|
|
69
|
+
# config/application.rb
|
|
70
|
+
config.middleware.insert_before 0, Bouncy::BodyLimit, path: "/bouncy/webhooks/ses", limit: 300.kilobytes
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
`path` defaults to `/bouncy/webhooks/ses` and `limit` to 300 KB. A request to that path reads at most `limit + 1` bytes, and a longer body answers 413. The path matches after repeated slashes are squeezed, with an optional trailing slash or format suffix (`.json`), the forms a Rails route accepts. It is compared with `PATH_INFO` and with `SCRIPT_NAME + PATH_INFO`, so it holds when the app is mounted under a prefix. Other paths, adjacent routes such as `/bouncy/webhooks/ses-admin` included, pass untouched. The receiver keeps its own 2 MiB limit.
|
|
74
|
+
|
|
75
|
+
SDK gems remain host dependencies even when injecting clients. See [SES setup](amazon-ses.md) for credentials and the signature-verifier test example.
|
|
76
|
+
|
|
77
|
+
## Development and test environments
|
|
78
|
+
|
|
79
|
+
Leave `scope` unset in environments that should not use Bouncy. Status reports `:unconfigured`, relations are empty, the interceptor leaves mail untouched and one warning is logged per process. Explicit mutations and sync raise `Bouncy::ConfigurationError`; the receiver returns 503 and logs the refusal. `Bouncy::SyncJob` and `Bouncy::PruneJob` skip with one info line.
|
|
80
|
+
|
|
81
|
+
`:off` only disables interception. It does not disable configured webhooks, synchronization or explicit mutations. A test that exercises those paths needs an isolated database and stubbed provider clients. Never use production credentials for an offline test.
|
|
82
|
+
|
|
83
|
+
There is no built-in admin framework, scheduler, AWS provisioning mode, remote manual block, marketing unsubscribe manager or resend setting. The host supplies those separate workflows where needed.
|
data/guides/delivery.md
CHANGED
|
@@ -1,21 +1,59 @@
|
|
|
1
1
|
# Action Mailer delivery boundaries
|
|
2
2
|
|
|
3
|
-
Supported paths are
|
|
3
|
+
Supported paths are `deliver_now`, `deliver_now!`, `deliver_later` and `deliver_later!`, with queued jobs executing in a configured worker, and messages delivered with Mail directly. Bouncy registers a Mail interceptor through the Rails engine.
|
|
4
4
|
|
|
5
|
-
The interceptor
|
|
5
|
+
The interceptor wraps the message's delivery method in `Bouncy::Interceptor::Gate`, through Mail's own `Message#delivery_method(klass, settings)`. Mail's `deliver` and `deliver!` both end by calling that delivery method, after every interceptor and every Action Mailer callback, so the gate is the one point that sees the recipients the transport would get. A `before_deliver` callback or an interceptor runs before some of them; the gate runs after all of them.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
The gate checks To/Cc/Bcc and the actual SMTP envelope, persists skip evidence, then removes blocked recipients in `:drop` mode. Each remaining entry keeps its text, display name included; only its bare address is looked up, so `"Ada <ADA@example.com>"` in an explicit envelope matches `ada@example.com`. When no recipient is left, the gate does not call the transport, so the bang methods, which ignore `perform_deliveries`, send nothing either. A host callback that replaces a blocked recipient with an allowed one sends to the allowed one: the decision follows what the transport gets. Recognized database outages preserve original headers, envelope and delivery flag.
|
|
8
|
+
|
|
9
|
+
`:log` observes while allowing delivery. `:off` disables interception. Manual holds apply independently; provider-derived dropping requires a recent complete sync, unless `require_fresh_sync` is `false`. An interrupted worker or unsupported policy must not enforce an old mirror forever. Log mode applies the same freshness rule, so each `skipped` event's `would_drop` and `stale_provider_evidence` flags show exactly what drop mode would have done. An unconfigured scope disables interception entirely and logs one warning.
|
|
8
10
|
|
|
9
11
|
`Bouncy.unblocked { ... }` is an execution-context exception with nested/exception-safe cleanup. It only wraps synchronous delivery; it does not serialize into jobs or apply to another process.
|
|
10
12
|
|
|
11
|
-
##
|
|
13
|
+
## The configuration set
|
|
14
|
+
|
|
15
|
+
With `config.ses.configuration_set`, the interceptor stamps `X-SES-CONFIGURATION-SET` on every message that does not name its own set. SES reads the header and removes it before delivery. The stamp does not depend on the interception mode, `Bouncy.unblocked` or the scope: turning interception off never stops the events that SES publishes for the set. The set joins the sending-policy check.
|
|
16
|
+
|
|
17
|
+
## A sender that checks and sends in one step
|
|
18
|
+
|
|
19
|
+
The interceptor decides about each message, but an app that records its own outcome ("sent", "skipped") checks the address itself first. Without a lock, a bounce can commit between that check and the send; the interceptor then drops a message the app records as sent. Hold the address around both:
|
|
20
|
+
|
|
21
|
+
```ruby
|
|
22
|
+
Order.transaction do
|
|
23
|
+
order.lock!
|
|
24
|
+
Bouncy.hold(order.email) do
|
|
25
|
+
if Bouncy.blocked?(order.email)
|
|
26
|
+
order.update!(receipt: "skipped")
|
|
27
|
+
else
|
|
28
|
+
ReceiptMailer.with(order:).receipt.deliver_now
|
|
29
|
+
order.update!(receipt: "sent")
|
|
30
|
+
end
|
|
31
|
+
end
|
|
32
|
+
end
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
`Bouncy.hold` takes a shared lock for the address. Every Bouncy write to that address (webhook, sync, `block!`, `release!`) takes it exclusively first, before any row lock, until its own transaction ends. A new block therefore commits before the check or after the send. Inside a transaction the hold lasts until that transaction ends, so the outcome commits before the block. Take your own row locks before the hold, as above.
|
|
36
|
+
|
|
37
|
+
| Database | Lock | Two senders to one address |
|
|
38
|
+
|---|---|---|
|
|
39
|
+
| PostgreSQL | Advisory lock on a 64-bit key of the scope and address; transaction-level inside a transaction | Both proceed |
|
|
40
|
+
| MySQL | Named lock (`GET_LOCK`), waiting up to `innodb_lock_wait_timeout` | They take turns |
|
|
41
|
+
| SQLite | `flock` on one file beside the database, shared or exclusive, per connection | Both proceed |
|
|
42
|
+
|
|
43
|
+
A wait that times out raises `ActiveRecord::LockWaitTimeout`; the receiver answers 503 and SNS retries. A write inside the same connection's own hold does not wait for itself. The hold needs no migration. Unconfigured, `Bouncy.hold` only runs the block.
|
|
44
|
+
|
|
45
|
+
The check must see the latest committed state, not an older snapshot. Inside a hold, `Bouncy.status`, `Bouncy.blocked?`, `Bouncy.statuses` and the gate read with `LOCK IN SHARE MODE` on MySQL, which reads the latest committed row at any isolation level. PostgreSQL reads the latest state with every statement at READ COMMITTED, its default.
|
|
46
|
+
|
|
47
|
+
At PostgreSQL REPEATABLE READ or SERIALIZABLE, the statement that takes the lock fixes the transaction's snapshot before it waits, so a block committed during the wait stays invisible. `Bouncy.hold` inside such a transaction therefore raises `Bouncy::ConfigurationError` at once. Hold the address outside the transaction instead, around it: the session-level lock lasts until the block ends, and the snapshot starts after the lock. On SQLite, whose snapshot starts at a transaction's first read, use the same pattern, or take the hold before the transaction reads anything.
|
|
48
|
+
|
|
49
|
+
Hooks never run inside a lock. A write releases its address when the outermost transaction ends, and the hooks wait for that transaction's commit: after a commit callback of a `transaction(joinable: false)` savepoint they still wait for the outer transaction, and a rollback drops them. Rails transactional tests keep 0.1's timing. Bouncy marks the fixture transaction of each pool that `ActiveRecord::TestFixtures` enrolls, as listed in its own `@fixture_connection_pools`: at `setup_transactional_fixtures`, and for a pool established later from a `!connection.active_record` subscriber that runs after Rails' own. It clears the marks in `teardown_transactional_fixtures` (Rails 7.2, 8.0 and 8.1). While a marked transaction is the only one open, hooks run when the app's own transaction commits. A database excluded with `skip_transactional_tests_for_database` (Rails 8.1), or a connection pinned in any other way, keeps the outer-commit rule. The hooks of a write made inside a hold run when the hold ends.
|
|
12
50
|
|
|
13
|
-
|
|
51
|
+
## Direct sends and other senders
|
|
14
52
|
|
|
15
|
-
Direct SDK sends, custom network clients and sister apps do not run
|
|
53
|
+
Direct SDK sends, custom network clients and sister apps do not run the callback or the interceptor. Their account restrictions can still appear in a later sync. Custom transports must honor `smtp_envelope_to`; reconstructing recipients needs host integration and tests.
|
|
16
54
|
|
|
17
|
-
|
|
55
|
+
An interceptor that replaces the message's delivery method after Bouncy's interceptor ran removes the gate. Register such interceptors before Bouncy loads, or test the transport path.
|
|
18
56
|
|
|
19
57
|
## Host tests
|
|
20
58
|
|
|
21
|
-
Use synthetic blocked and allowed recipients in one message. Capture the actual transport envelope, not just rendered headers. Exercise Bcc, an explicit envelope different from headers, an all-blocked message and queued delivery. Simulate a database outage after lookup to prove no partial mutation occurs.
|
|
59
|
+
Use synthetic blocked and allowed recipients in one message. Capture the actual transport envelope, not just rendered headers. Exercise Bcc, display names, an explicit envelope different from headers, an all-blocked message with each of the four delivery methods and queued delivery. Simulate a database outage after lookup to prove no partial mutation occurs.
|
data/guides/migrating.md
CHANGED
|
@@ -6,10 +6,10 @@ Migration and cutover belong in the implementing app. Bouncy's install migration
|
|
|
6
6
|
2. Install the additive schema. Write a host data migration using migration-local ActiveRecord classes or explicit SQL pinned to that schema. Never call live Bouncy/host models, callbacks, jobs or AWS from a historical migration.
|
|
7
7
|
3. Preserve manual holds independently. Map complaints as complaints and hard bounces as hard bounces. Account-list refusal is evidence of an existing restriction. Soft trackers are history; active legacy soft holds require an explicit preserved policy, not an accidental new threshold engine. Preserve original expiries for timed soft holds. Preserve indefinite holds as explicit manual holds with a migration note; do not invent a 30-day expiry. New escalation applies only to mailbox-specific evidence. Document that change if the old rule included content failures.
|
|
8
8
|
4. Use stable source IDs to make imported events idempotent. Preserve known provider feedback IDs using Bouncy's recipient-level dedupe identity so retried historical feedback does not count again. Preserve distinct occurrences, exact provider addresses when available, and relevant event times. Missing historical case variants are resolved by complete provider enumeration before recovery.
|
|
9
|
-
5. Rehearse the import against isolated synthetic fixtures
|
|
10
|
-
6. Quiesce old writers for cutover, or implement and test dual-write delivery with deterministic identity. Do not replace readers while another process continues writing exclusively to the old table.
|
|
9
|
+
5. Rehearse the import against isolated synthetic fixtures. Compare every predicate, support action and customer-facing status that depended on the old table. Rehearse bootstrap as part of this isolated cutover sequence.
|
|
10
|
+
6. Quiesce old writers for cutover, or implement and test dual-write delivery with deterministic identity. Perform the final import, switch the receiver/readers, then run `bouncy:bootstrap` before starting mail workers. It raises unless the mirror is fresh, complete and policy-verified. Ensure feedback received during the switch is retried or durably captured. Do not replace readers while another process continues writing exclusively to the old table.
|
|
11
11
|
7. Rehearse rollback, including restrictions and releases created after cutover. Keeping an old table is not a rollback plan if it missed later changes. Keep old schema until the agreed window ends. Do not delete imported rows in a schema rollback: they may now contain newer state. Fail conflicting imports and stop writers before copying in either direction.
|
|
12
12
|
|
|
13
13
|
Use the app's existing public webhook URL where possible. Test routing alongside unrelated webhooks, and preserve the existing interception mode deliberately. Host admin resources call Bouncy's recovery API; stock row deletion cannot stand in for provider release.
|
|
14
14
|
|
|
15
|
-
|
|
15
|
+
Bouncy has automated schema and lifecycle coverage. Rehearse your actual application import, cutover and rollback against its own mappings and writers; the gem cannot infer them from an arbitrary legacy table.
|