bouncy 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +32 -0
- data/LICENSE.txt +21 -0
- data/README.md +213 -0
- data/config/routes.rb +5 -0
- data/guides/admin.md +71 -0
- data/guides/amazon-ses.md +63 -0
- data/guides/bounce-handling.md +27 -0
- data/guides/compatibility.md +42 -0
- data/guides/delivery.md +21 -0
- data/guides/migrating.md +15 -0
- data/guides/privacy.md +13 -0
- data/guides/recovery.md +28 -0
- data/guides/troubleshooting.md +46 -0
- data/lib/bouncy/configuration.rb +80 -0
- data/lib/bouncy/engine.rb +15 -0
- data/lib/bouncy/event.rb +22 -0
- data/lib/bouncy/identity.rb +18 -0
- data/lib/bouncy/ingestor.rb +94 -0
- data/lib/bouncy/interceptor.rb +57 -0
- data/lib/bouncy/model.rb +30 -0
- data/lib/bouncy/providers/base.rb +24 -0
- data/lib/bouncy/providers/ses.rb +197 -0
- data/lib/bouncy/providers/ses_parser.rb +112 -0
- data/lib/bouncy/providers/sns_verifier.rb +97 -0
- data/lib/bouncy/prune_job.rb +13 -0
- data/lib/bouncy/reconciler.rb +150 -0
- data/lib/bouncy/record.rb +16 -0
- data/lib/bouncy/recovery.rb +56 -0
- data/lib/bouncy/scope_lock.rb +52 -0
- data/lib/bouncy/status.rb +59 -0
- data/lib/bouncy/status_set.rb +31 -0
- data/lib/bouncy/store.rb +30 -0
- data/lib/bouncy/suppression.rb +33 -0
- data/lib/bouncy/sync_job.rb +13 -0
- data/lib/bouncy/version.rb +5 -0
- data/lib/bouncy/webhook.rb +40 -0
- data/lib/bouncy.rb +185 -0
- data/lib/generators/bouncy/install_generator.rb +37 -0
- data/lib/generators/bouncy/templates/create_bouncy_tables.rb.erb +97 -0
- data/lib/generators/bouncy/templates/initializer.rb.tt +23 -0
- data/lib/tasks/bouncy.rake +36 -0
- metadata +126 -0
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Action Mailer says sent, but the email was not received
|
|
2
|
+
|
|
3
|
+
Start by separating what you know:
|
|
4
|
+
|
|
5
|
+
1. A mailer job completed: your app ran the send path.
|
|
6
|
+
2. SES accepted a send request: the provider accepted responsibility for processing it.
|
|
7
|
+
3. A delivery event arrived: the receiving server accepted the message.
|
|
8
|
+
4. The person saw the message: only the person or another appropriate signal can establish that.
|
|
9
|
+
|
|
10
|
+
Bouncy records known restrictions and selected provider events. Missing evidence does not mean successful delivery. Spam placement, recipient filters, content problems and incorrect recipient selection may require investigation elsewhere.
|
|
11
|
+
|
|
12
|
+
```ruby
|
|
13
|
+
status = Bouncy.status(address)
|
|
14
|
+
[status.knowledge, status.reasons, status.stale?, status.observed_at]
|
|
15
|
+
Bouncy.events.for(address).recent.limit(20)
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
## An address is on the SES suppression list
|
|
19
|
+
|
|
20
|
+
`OnAccountSuppressionList` means SES refused a send because a restriction already existed. It is not a new hard bounce. A complete `Bouncy.sync!` imports restrictions from before installation, sister apps on the same account, missed notifications and console changes.
|
|
21
|
+
|
|
22
|
+
Inspect the underlying reason and the account/region. After verifying that recovery is appropriate, an authorized support action can call `Bouncy.release!(address, note: ...)`. This operates on exact-case provider identifiers and confirms removal. A lowercase console/API lookup alone may miss another case variant.
|
|
23
|
+
|
|
24
|
+
Release never resends the original email. Regenerate time-sensitive links in an explicitly authorized host action if a resend is needed. See [recovery](recovery.md).
|
|
25
|
+
|
|
26
|
+
## Nothing is appearing locally
|
|
27
|
+
|
|
28
|
+
- Check the configured account, region, credentials and exact SNS topic allowlist.
|
|
29
|
+
- Run `bouncy:doctor`, then sync. An unknown/manual publisher-policy check is not a pass. If the latest sync did not verify policy, `Bouncy.status(address).policy_unverified?` is true for listed addresses and `policy_reason` explains why. Historical restrictions remain queryable while provider-derived interception is suspended.
|
|
30
|
+
- If `Bouncy.status(address).knowledge` is `:unconfigured`, `config.scope` is blank and Bouncy is inactive in that environment.
|
|
31
|
+
- Confirm the SNS HTTPS subscription is active and points at the actual mounted route.
|
|
32
|
+
- Confirm SES routes the needed event types to that topic, and raw SNS message delivery is disabled.
|
|
33
|
+
- Inspect bounded receiver status codes: 401 authorization/signature, 400 malformed input, 413 body limit, 503 retryable infrastructure failure.
|
|
34
|
+
- Run a complete sync and verify the scheduled job actually runs in production.
|
|
35
|
+
|
|
36
|
+
An hourly sync cannot reconstruct every lost delivery or soft-bounce event. SNS retries and operational monitoring still matter.
|
|
37
|
+
|
|
38
|
+
## The mirror is stale or unavailable
|
|
39
|
+
|
|
40
|
+
Compare `Bouncy.last_sync` and `Bouncy.last_successful_sync`. A failed/partial run cannot prove absence. A provider scope mismatch blocks authoritative reconciliation. Check credentials, API permissions, configuration-set overrides and worker execution.
|
|
41
|
+
|
|
42
|
+
Recognized database read outages produce an unavailable status and fail open for interception. Ordinary SQL/programming errors raise. In drop mode, provider-derived enforcement needs a recent complete sync; independent manual holds do not depend on provider freshness.
|
|
43
|
+
|
|
44
|
+
## A message was locally skipped
|
|
45
|
+
|
|
46
|
+
Inspect its recipient's `skipped` event and mode. `:log` observes the decision while delivering. `:drop` removes restricted recipients from both headers and envelope. Custom transports or later interceptors that rewrite recipients require host tests. Never infer a send from the absence of a skip record.
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Bouncy
|
|
4
|
+
class Configuration
|
|
5
|
+
class Ses
|
|
6
|
+
# sns_message_verifier replaces ONLY the Aws::SNS::MessageVerifier that downloads and
|
|
7
|
+
# caches Amazon's signing certificate, so a host can test its mounted receiver offline.
|
|
8
|
+
# Topic authorization and certificate-URL checks still run, which is the point: a test
|
|
9
|
+
# seam that skipped them would hide exactly the hole they exist to close.
|
|
10
|
+
attr_accessor :region, :credentials, :topic_arns, :client, :sns_client, :sts_client, :sns_message_verifier,
|
|
11
|
+
:identities, :configuration_sets, :all_sending_paths_listed
|
|
12
|
+
|
|
13
|
+
def initialize
|
|
14
|
+
@topic_arns = []
|
|
15
|
+
@identities = []
|
|
16
|
+
@configuration_sets = []
|
|
17
|
+
# Imported provider restrictions are enforced only after the policy check passes, and the
|
|
18
|
+
# check needs every sending identity and configuration set listed. This flag is the
|
|
19
|
+
# installer's statement that the lists above are complete.
|
|
20
|
+
@all_sending_paths_listed = false
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
# A soft bounce is a temporary refusal, so one of them says nothing. Repeated soft bounces
|
|
25
|
+
# for the same address inside a window are a different signal, and some hosts want them to
|
|
26
|
+
# stop the sending. Leave the threshold nil to keep soft bounces record-only.
|
|
27
|
+
SOFT_BOUNCE_OCCURRENCE_LIMIT = 50
|
|
28
|
+
|
|
29
|
+
attr_accessor :provider, :scope, :interception, :record_deliveries,
|
|
30
|
+
:retention, :maximum_event_age, :stale_after, :after_block, :after_release,
|
|
31
|
+
:after_event, :adapter, :soft_bounce_threshold, :soft_bounce_window,
|
|
32
|
+
:soft_bounce_block_for
|
|
33
|
+
attr_reader :ses
|
|
34
|
+
|
|
35
|
+
def initialize
|
|
36
|
+
@provider = :ses
|
|
37
|
+
@interception = :log
|
|
38
|
+
@record_deliveries = false
|
|
39
|
+
@retention = 90.days
|
|
40
|
+
@maximum_event_age = 90.days
|
|
41
|
+
@stale_after = 2.hours
|
|
42
|
+
@soft_bounce_threshold = nil
|
|
43
|
+
@soft_bounce_window = 30.days
|
|
44
|
+
@soft_bounce_block_for = 30.days
|
|
45
|
+
@ses = Ses.new
|
|
46
|
+
@after_block = @after_release = @after_event = ->(_event) {}
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
# Occurrence times retained per address so the window can roll. Bounded either way: a
|
|
50
|
+
# configured threshold needs no more entries than the threshold itself.
|
|
51
|
+
def soft_bounce_occurrences
|
|
52
|
+
return SOFT_BOUNCE_OCCURRENCE_LIMIT unless soft_bounce_escalation?
|
|
53
|
+
|
|
54
|
+
[soft_bounce_threshold, SOFT_BOUNCE_OCCURRENCE_LIMIT].min
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
def soft_bounce_escalation? = soft_bounce_threshold.to_i.positive?
|
|
58
|
+
|
|
59
|
+
def configured? = !scope.to_s.strip.empty?
|
|
60
|
+
|
|
61
|
+
def validate!
|
|
62
|
+
raise ConfigurationError, "Set config.scope to the provider account and region" unless configured?
|
|
63
|
+
raise ConfigurationError, "scope must be at most 191 characters" if scope.to_s.length > 191
|
|
64
|
+
raise ConfigurationError, "Only the SES provider is supported" unless provider == :ses || adapter
|
|
65
|
+
raise ConfigurationError, "interception must be :log, :drop or :off" unless %i[log drop off].include?(interception)
|
|
66
|
+
unless maximum_event_age.positive? && retention >= maximum_event_age
|
|
67
|
+
raise ConfigurationError, "retention must cover maximum_event_age, and both must be positive"
|
|
68
|
+
end
|
|
69
|
+
if soft_bounce_threshold && !(soft_bounce_threshold.is_a?(Integer) && (1..SOFT_BOUNCE_OCCURRENCE_LIMIT).cover?(soft_bounce_threshold))
|
|
70
|
+
raise ConfigurationError, "soft_bounce_threshold must be an integer from 1 to #{SOFT_BOUNCE_OCCURRENCE_LIMIT}, or nil"
|
|
71
|
+
end
|
|
72
|
+
unless [soft_bounce_window, soft_bounce_block_for].all? { |duration| duration.is_a?(Numeric) || duration.is_a?(ActiveSupport::Duration) } &&
|
|
73
|
+
soft_bounce_window.positive? && soft_bounce_block_for.positive?
|
|
74
|
+
raise ConfigurationError, "soft_bounce_window and soft_bounce_block_for must be positive durations"
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
self
|
|
78
|
+
end
|
|
79
|
+
end
|
|
80
|
+
end
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Bouncy
|
|
4
|
+
class Engine < Rails::Engine
|
|
5
|
+
isolate_namespace Bouncy
|
|
6
|
+
|
|
7
|
+
initializer "bouncy.model" do
|
|
8
|
+
ActiveSupport.on_load(:active_record) { extend Bouncy::Model }
|
|
9
|
+
end
|
|
10
|
+
|
|
11
|
+
initializer "bouncy.interceptor" do
|
|
12
|
+
ActiveSupport.on_load(:action_mailer) { register_interceptor Bouncy::Interceptor }
|
|
13
|
+
end
|
|
14
|
+
end
|
|
15
|
+
end
|
data/lib/bouncy/event.rb
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Bouncy
|
|
4
|
+
class Event < Record
|
|
5
|
+
self.table_name = "bouncy_events"
|
|
6
|
+
|
|
7
|
+
attribute :details, default: -> { {} }
|
|
8
|
+
|
|
9
|
+
scope :for, ->(email) { where(email: Identity.normalize(email)) }
|
|
10
|
+
scope :recent, -> { order(occurred_at: :desc, id: :desc) }
|
|
11
|
+
|
|
12
|
+
after_create_commit :notify_host
|
|
13
|
+
|
|
14
|
+
private
|
|
15
|
+
|
|
16
|
+
def notify_host
|
|
17
|
+
Bouncy.notify(:after_event, self)
|
|
18
|
+
Bouncy.notify(:after_block, self) if details["became_blocked"]
|
|
19
|
+
Bouncy.notify(:after_release, self) if kind == "release"
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
end
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Bouncy
|
|
4
|
+
module Identity
|
|
5
|
+
module_function
|
|
6
|
+
|
|
7
|
+
def normalize(email)
|
|
8
|
+
raise InvalidAddress, "Email addresses cannot contain control characters" if email.to_s.match?(/[\x00-\x1f\x7f]/)
|
|
9
|
+
|
|
10
|
+
value = email.to_s.strip
|
|
11
|
+
unless value.bytesize <= 254 && value.match?(/\A[^\s@<>\x00-\x1f\x7f]+@[^\s@<>\x00-\x1f\x7f]+\z/)
|
|
12
|
+
raise InvalidAddress, "Provide a nonblank email address without control characters"
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
value.downcase
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
end
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Bouncy
|
|
4
|
+
Observation = Data.define(:email, :kind, :provider_event_id, :message_id, :occurred_at, :details, :provider_reason, :status_code, :diagnostic)
|
|
5
|
+
|
|
6
|
+
class Ingestor
|
|
7
|
+
def call(observation)
|
|
8
|
+
email = observation.email && Identity.normalize(observation.email)
|
|
9
|
+
attributes = observation.to_h.merge(email: email)
|
|
10
|
+
identity = [Bouncy.scope, Bouncy.configuration.provider, observation.provider_event_id, observation.kind, email]
|
|
11
|
+
attributes[:dedupe_key] = Digest::SHA256.hexdigest(JSON.generate(identity))
|
|
12
|
+
return :duplicate if duplicate?(attributes[:dedupe_key])
|
|
13
|
+
|
|
14
|
+
if email
|
|
15
|
+
Store.change(email) do |row|
|
|
16
|
+
# Re-check under the row lock so a concurrent retry becomes a duplicate, not a unique violation.
|
|
17
|
+
next :duplicate if duplicate?(attributes[:dedupe_key])
|
|
18
|
+
|
|
19
|
+
persist(attributes, row)
|
|
20
|
+
end
|
|
21
|
+
else
|
|
22
|
+
Event.transaction { persist(attributes, nil) }
|
|
23
|
+
end
|
|
24
|
+
rescue ActiveRecord::RecordNotUnique
|
|
25
|
+
# The unique event and its state transition committed in the same transaction.
|
|
26
|
+
raise unless duplicate?(attributes[:dedupe_key])
|
|
27
|
+
|
|
28
|
+
:duplicate
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
private
|
|
32
|
+
|
|
33
|
+
def duplicate?(key)
|
|
34
|
+
Bouncy.events.exists?(dedupe_key: key)
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
def persist(attributes, row)
|
|
38
|
+
time = attributes.fetch(:occurred_at)
|
|
39
|
+
too_old = time < Bouncy.configuration.maximum_event_age.ago || time > 5.minutes.from_now
|
|
40
|
+
fenced = row&.released_before && time <= row.released_before
|
|
41
|
+
soft_fence = row && attributes[:kind] == "soft_bounce" && parse_time(row.details["soft_released_before"])
|
|
42
|
+
fenced ||= soft_fence && time <= soft_fence
|
|
43
|
+
out_of_order = attributes[:kind] != "soft_bounce" && row&.event_blocked_at && time < row.event_blocked_at
|
|
44
|
+
details = attributes.fetch(:details).merge("ignored_for_policy" => [too_old, fenced, out_of_order].any?)
|
|
45
|
+
if row && !details["ignored_for_policy"]
|
|
46
|
+
was_blocked = row.blocked?
|
|
47
|
+
# A complaint blocks locally only when the provider named exactly one recipient (SesParser
|
|
48
|
+
# marks it "confirmed"); candidates wait for the provider's own list at the next sync.
|
|
49
|
+
if attributes[:kind] == "hard_bounce" || (attributes[:kind] == "complaint" && details["certainty"] == "confirmed")
|
|
50
|
+
row.event_blocked_at = time
|
|
51
|
+
row.event_reason = attributes[:kind]
|
|
52
|
+
row.last_event_at = [row.last_event_at, time].compact.max
|
|
53
|
+
row.details = row.details.except("absence_count")
|
|
54
|
+
row.details["exact_email"] = details["exact_email"] if details["exact_email"]
|
|
55
|
+
elsif attributes[:kind] == "soft_bounce" && details["escalation_eligible"] == true
|
|
56
|
+
count_soft_bounce(row, time, details)
|
|
57
|
+
end
|
|
58
|
+
row.summarize!
|
|
59
|
+
details["became_blocked"] = !was_blocked && row.blocked?
|
|
60
|
+
end
|
|
61
|
+
Store.event!(attributes.fetch(:kind), **attributes.except(:kind, :details), source: "webhook",
|
|
62
|
+
provider: Bouncy.configuration.provider.to_s, details: details)
|
|
63
|
+
:accepted
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
# Records this soft bounce against the address and, once config.soft_bounce_threshold of them
|
|
67
|
+
# fall inside config.soft_bounce_window, holds the address for config.soft_bounce_block_for.
|
|
68
|
+
# The occurrence times are kept so the window really rolls: a bounce that ages out stops
|
|
69
|
+
# counting, instead of a running total that only ever grows. Redelivered notifications are
|
|
70
|
+
# already filtered by the event dedupe key before this runs. With no threshold configured a
|
|
71
|
+
# soft bounce only updates the counters, which is the default.
|
|
72
|
+
def count_soft_bounce(row, time, details)
|
|
73
|
+
window = Bouncy.configuration.soft_bounce_window
|
|
74
|
+
previous = Array(row.details["soft_bounces"]).filter_map { |value| parse_time(value) }
|
|
75
|
+
occurrences = (previous + [time]).select { |occurred_at| occurred_at > Time.current - window }.sort
|
|
76
|
+
occurrences = occurrences.last(Bouncy.configuration.soft_bounce_occurrences)
|
|
77
|
+
row.details = row.details.merge("soft_bounces" => occurrences.map { |occurred_at| occurred_at.iso8601(6) })
|
|
78
|
+
row.soft_bounce_count = occurrences.size
|
|
79
|
+
row.last_soft_bounce_at = [row.last_soft_bounce_at, time].compact.max
|
|
80
|
+
row.last_event_at = [row.last_event_at, time].compact.max
|
|
81
|
+
details["soft_bounce_count"] = occurrences.size
|
|
82
|
+
return unless Bouncy.configuration.soft_bounce_escalation? && occurrences.size >= Bouncy.configuration.soft_bounce_threshold
|
|
83
|
+
|
|
84
|
+
row.soft_blocked_until = [row.soft_blocked_until, occurrences.last + Bouncy.configuration.soft_bounce_block_for].compact.max
|
|
85
|
+
details["soft_blocked_until"] = row.soft_blocked_until.iso8601(6)
|
|
86
|
+
end
|
|
87
|
+
|
|
88
|
+
def parse_time(value)
|
|
89
|
+
Time.iso8601(value.to_s)
|
|
90
|
+
rescue ArgumentError, TypeError
|
|
91
|
+
nil
|
|
92
|
+
end
|
|
93
|
+
end
|
|
94
|
+
end
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Bouncy
|
|
4
|
+
class Interceptor
|
|
5
|
+
def self.delivering_email(message)
|
|
6
|
+
return if Bouncy.configuration.interception == :off || ActiveSupport::IsolatedExecutionState[:bouncy_unblocked]
|
|
7
|
+
return Bouncy.unconfigured!("interception") unless Bouncy.configured?
|
|
8
|
+
|
|
9
|
+
new.call(message)
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
def call(message)
|
|
13
|
+
envelope = Array(message.smtp_envelope_to).dup
|
|
14
|
+
headers = %i[to cc bcc].to_h { |field| [field, Array(message.public_send(field)).dup] }
|
|
15
|
+
keys = (envelope + headers.values.flatten).filter_map { |email| normalize(email) }.uniq
|
|
16
|
+
rows = Bouncy.blocked.where(email: keys).to_a
|
|
17
|
+
return if rows.empty?
|
|
18
|
+
|
|
19
|
+
mode = Bouncy.configuration.interception
|
|
20
|
+
# Provider-derived evidence is only enforced while a complete sync is fresh. Manual holds
|
|
21
|
+
# and legacy soft holds are local policy and do not depend on provider freshness. The same
|
|
22
|
+
# rule applies in :log mode so that its preview matches what :drop would do.
|
|
23
|
+
fresh = Bouncy.sync_fresh?
|
|
24
|
+
enforceable = rows.select { |row| fresh || row.manual_blocked_at || (row.soft_blocked_until && row.soft_blocked_until > Time.current) }
|
|
25
|
+
blocked = enforceable.map(&:email)
|
|
26
|
+
remaining = envelope.reject { |email| blocked.include?(normalize(email)) }
|
|
27
|
+
changed_headers = headers.transform_values { |values| values.reject { |email| blocked.include?(normalize(email)) } }
|
|
28
|
+
# Persist all skip evidence before changing the message. A DB outage must leave it intact.
|
|
29
|
+
Event.transaction do
|
|
30
|
+
rows.each do |row|
|
|
31
|
+
would_drop = enforceable.include?(row)
|
|
32
|
+
Store.event!("skipped", email: row.email, source: "interceptor", details: {
|
|
33
|
+
"mode" => mode.to_s, "reasons" => row.reasons.map(&:to_s), "would_drop" => would_drop,
|
|
34
|
+
"dropped" => mode == :drop && would_drop, "stale_provider_evidence" => !would_drop
|
|
35
|
+
})
|
|
36
|
+
end
|
|
37
|
+
end
|
|
38
|
+
return unless mode == :drop && blocked.any?
|
|
39
|
+
|
|
40
|
+
changed_headers.each { |field, values| message.public_send("#{field}=", values.empty? ? nil : values) }
|
|
41
|
+
message.smtp_envelope_to = remaining
|
|
42
|
+
message.perform_deliveries = false if remaining.empty?
|
|
43
|
+
rescue ActiveRecord::ActiveRecordError => e
|
|
44
|
+
raise unless Bouncy.database_unavailable?(e)
|
|
45
|
+
|
|
46
|
+
ActiveSupport::Notifications.instrument("unavailable.bouncy", operation: "interception")
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
private
|
|
50
|
+
|
|
51
|
+
def normalize(value)
|
|
52
|
+
Identity.normalize(Mail::Address.new(value).address)
|
|
53
|
+
rescue InvalidAddress, Mail::Field::ParseError
|
|
54
|
+
nil
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
end
|
data/lib/bouncy/model.rb
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Bouncy
|
|
4
|
+
module Model
|
|
5
|
+
def bouncy(*attributes, normalized_attribute: nil)
|
|
6
|
+
raise ArgumentError, "Provide an email attribute" if attributes.empty?
|
|
7
|
+
raise ArgumentError, "normalized_attribute supports one email attribute at a time" if normalized_attribute && attributes.size != 1
|
|
8
|
+
|
|
9
|
+
attributes.each do |attribute|
|
|
10
|
+
column = normalized_attribute || attribute
|
|
11
|
+
define_method("#{attribute}_status") { Bouncy.status(public_send(column)) }
|
|
12
|
+
define_method("#{attribute}_blocked?") do
|
|
13
|
+
value = public_send(column)
|
|
14
|
+
value.present? && Bouncy.blocked?(value)
|
|
15
|
+
end
|
|
16
|
+
define_method("#{attribute}_bounced?") do
|
|
17
|
+
public_send("#{attribute}_blocked?") && public_send("#{attribute}_status").reasons.intersect?(%i[hard_bounce soft_bounces])
|
|
18
|
+
end
|
|
19
|
+
define_method("#{attribute}_complained?") { public_send("#{attribute}_blocked?") && public_send("#{attribute}_status").reasons.include?(:complaint) }
|
|
20
|
+
|
|
21
|
+
scope "#{attribute}_blocked", -> { where(column => Bouncy.blocked.select(:email)) }
|
|
22
|
+
scope "#{attribute}_bounced", lambda {
|
|
23
|
+
rows = Bouncy.blocked.where("provider_reason = ? OR event_reason = ? OR soft_blocked_until > ?", "hard_bounce", "hard_bounce", Time.current)
|
|
24
|
+
where(column => rows.select(:email))
|
|
25
|
+
}
|
|
26
|
+
scope "#{attribute}_unblocked", -> { where.not(column => [nil, ""]).where.not(column => Bouncy.blocked.select(:email)) }
|
|
27
|
+
end
|
|
28
|
+
end
|
|
29
|
+
end
|
|
30
|
+
end
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Bouncy
|
|
4
|
+
module Providers
|
|
5
|
+
Entry = Data.define(:email, :reason, :updated_at) do
|
|
6
|
+
def to_h
|
|
7
|
+
{ "email" => email, "reason" => reason.to_s, "provider_updated_at" => updated_at.iso8601(6) }
|
|
8
|
+
end
|
|
9
|
+
end
|
|
10
|
+
|
|
11
|
+
Snapshot = Data.define(:entries, :scope, :complete, :policy_verified, :started_at, :finished_at, :policy_reason) do
|
|
12
|
+
def initialize(policy_reason: nil, **members) = super
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
# Result of the sending-policy check: verified, or the plain-language reason it is not.
|
|
16
|
+
PolicyCheck = Data.define(:verified, :reason)
|
|
17
|
+
|
|
18
|
+
class Base
|
|
19
|
+
def snapshot = raise(NotImplementedError)
|
|
20
|
+
def lookup(_entry) = raise(NotImplementedError)
|
|
21
|
+
def release(_entries) = raise(NotImplementedError)
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
|
@@ -0,0 +1,197 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Bouncy
|
|
4
|
+
module Providers
|
|
5
|
+
class Ses < Base
|
|
6
|
+
def initialize(configuration = Bouncy.configuration)
|
|
7
|
+
super()
|
|
8
|
+
@configuration = configuration
|
|
9
|
+
@settings = configuration.ses
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
def snapshot
|
|
13
|
+
started_at = Time.current
|
|
14
|
+
policy = policy_check
|
|
15
|
+
entries = []
|
|
16
|
+
tokens = []
|
|
17
|
+
token = nil
|
|
18
|
+
loop do
|
|
19
|
+
response = request { client.list_suppressed_destinations(page_size: 1000, next_token: token) }
|
|
20
|
+
response.suppressed_destination_summaries.each do |row|
|
|
21
|
+
Identity.normalize(row.email_address)
|
|
22
|
+
entries << Entry.new(email: row.email_address, reason: reason(row.reason), updated_at: row.last_update_time)
|
|
23
|
+
end
|
|
24
|
+
token = response.next_token
|
|
25
|
+
break if token.nil? || token.empty?
|
|
26
|
+
raise ProviderError, "SES repeated a pagination token" if tokens.include?(token)
|
|
27
|
+
|
|
28
|
+
tokens << token
|
|
29
|
+
end
|
|
30
|
+
Snapshot.new(entries: entries, scope: @configuration.scope, complete: true, policy_verified: policy.verified,
|
|
31
|
+
policy_reason: policy.reason, started_at: started_at, finished_at: Time.current)
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
def lookup(email)
|
|
35
|
+
request { client.get_suppressed_destination(email_address: email) }
|
|
36
|
+
:present
|
|
37
|
+
rescue Aws::SESV2::Errors::NotFoundException
|
|
38
|
+
:absent
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
def release(entries)
|
|
42
|
+
outcomes = []
|
|
43
|
+
entries.each do |entry|
|
|
44
|
+
result = "removed"
|
|
45
|
+
begin
|
|
46
|
+
request { client.delete_suppressed_destination(email_address: entry.email) }
|
|
47
|
+
rescue Aws::SESV2::Errors::NotFoundException
|
|
48
|
+
# Only the exact provider identifier is checked; never substitute lowercase.
|
|
49
|
+
result = "already_absent"
|
|
50
|
+
end
|
|
51
|
+
raise ProviderError, "SES restriction remains after recovery" unless lookup(entry.email) == :absent
|
|
52
|
+
|
|
53
|
+
outcomes << { "email" => entry.email, "result" => result }
|
|
54
|
+
rescue ProviderError => e
|
|
55
|
+
outcomes << { "email" => entry.email, "result" => "failed", "error_class" => e.class.name }
|
|
56
|
+
remaining = entries.reject { |candidate| outcomes.any? { |outcome| outcome["email"] == candidate.email } }
|
|
57
|
+
outcomes.concat(remaining.map { |candidate| { "email" => candidate.email, "result" => "not_attempted" } })
|
|
58
|
+
raise ReleaseFailed.new("SES recovery did not finish; review the per-address outcomes", outcomes: outcomes)
|
|
59
|
+
end
|
|
60
|
+
outcomes
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
def authenticate(raw_body:, **)
|
|
64
|
+
require_sdk("sns")
|
|
65
|
+
require_relative "sns_verifier"
|
|
66
|
+
@authenticator ||= SnsVerifier.new(@configuration)
|
|
67
|
+
@authenticator.call(raw_body)
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
def control(envelope) # rubocop:disable Naming/PredicateMethod -- Adapter operation with side effects.
|
|
71
|
+
case envelope.fetch("Type")
|
|
72
|
+
when "SubscriptionConfirmation"
|
|
73
|
+
request { sns_client.confirm_subscription(topic_arn: envelope.fetch("TopicArn"), token: envelope.fetch("Token")) }
|
|
74
|
+
Store.event!("subscription_confirmed", source: "webhook")
|
|
75
|
+
true
|
|
76
|
+
when "UnsubscribeConfirmation"
|
|
77
|
+
Store.event!("subscription_unsubscribed", source: "webhook")
|
|
78
|
+
true
|
|
79
|
+
else
|
|
80
|
+
false
|
|
81
|
+
end
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
def parse(envelope)
|
|
85
|
+
SesParser.new(@configuration).call(envelope)
|
|
86
|
+
end
|
|
87
|
+
|
|
88
|
+
def doctor
|
|
89
|
+
policy = policy_check
|
|
90
|
+
{ "scope" => @configuration.scope, "policy_verified" => policy.verified, "policy_reason" => policy.reason,
|
|
91
|
+
"topic_allowlist" => @settings.topic_arns.any?, "topics" => topic_checks,
|
|
92
|
+
"write_permissions" => "unknown; never tested by mutation" }
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
private
|
|
96
|
+
|
|
97
|
+
def require_sdk(service)
|
|
98
|
+
require "aws-sdk-#{service}"
|
|
99
|
+
rescue LoadError
|
|
100
|
+
raise ConfigurationError, "Install the optional adapter dependency: bundle add aws-sdk-#{service}"
|
|
101
|
+
end
|
|
102
|
+
|
|
103
|
+
def client
|
|
104
|
+
require_sdk("sesv2")
|
|
105
|
+
@client ||= @settings.client || Aws::SESV2::Client.new(**client_options)
|
|
106
|
+
end
|
|
107
|
+
|
|
108
|
+
def sns_client
|
|
109
|
+
require_sdk("sns")
|
|
110
|
+
@sns_client ||= @settings.sns_client || Aws::SNS::Client.new(**client_options)
|
|
111
|
+
end
|
|
112
|
+
|
|
113
|
+
def sts_client
|
|
114
|
+
require_sdk("sts")
|
|
115
|
+
@sts_client ||= @settings.sts_client || Aws::STS::Client.new(**client_options)
|
|
116
|
+
end
|
|
117
|
+
|
|
118
|
+
def client_options
|
|
119
|
+
{ region: @settings.region, credentials: @settings.credentials, retry_limit: 2,
|
|
120
|
+
http_open_timeout: 3, http_read_timeout: 10 }.compact
|
|
121
|
+
end
|
|
122
|
+
|
|
123
|
+
def request
|
|
124
|
+
yield
|
|
125
|
+
rescue Seahorse::Client::NetworkingError => e
|
|
126
|
+
raise ProviderError, "AWS transport failed (#{e.class.name})"
|
|
127
|
+
rescue Aws::Errors::MissingCredentialsError
|
|
128
|
+
raise ProviderError, "AWS credentials are missing; set config.ses.credentials or configure the AWS credential chain"
|
|
129
|
+
rescue Aws::Errors::ServiceError => e
|
|
130
|
+
raise if defined?(Aws::SESV2::Errors::NotFoundException) && e.is_a?(Aws::SESV2::Errors::NotFoundException)
|
|
131
|
+
|
|
132
|
+
raise ProviderError, "AWS request failed (#{e.code})"
|
|
133
|
+
end
|
|
134
|
+
|
|
135
|
+
# Verifies that the configured scope matches the credentials and that every listed sending
|
|
136
|
+
# path uses account-level suppression for both bounces and complaints. Until this passes,
|
|
137
|
+
# sync runs in observation mode: entries are mirrored but nothing is enforced, and the
|
|
138
|
+
# reason is reported by the sync event and by `bouncy:doctor`.
|
|
139
|
+
def policy_check
|
|
140
|
+
region = @settings.region
|
|
141
|
+
raise ConfigurationError, "Set config.ses.region" unless region.to_s.match?(/\A[a-z]{2}(?:-[a-z]+)+-\d\z/)
|
|
142
|
+
|
|
143
|
+
# Resolve clients before request's typed AWS rescue clauses are needed.
|
|
144
|
+
sts = sts_client
|
|
145
|
+
account = request { sts.get_caller_identity }.account
|
|
146
|
+
expected = "ses:#{account}:#{region}:account"
|
|
147
|
+
raise UnsafeSnapshot, "Configured scope does not match AWS credentials and region" unless @configuration.scope == expected
|
|
148
|
+
|
|
149
|
+
ses = client
|
|
150
|
+
raise UnsafeSnapshot, "SES client region differs from configured scope" unless ses.config.region == region
|
|
151
|
+
|
|
152
|
+
reasons = request { ses.get_account }.suppression_attributes&.suppressed_reasons || []
|
|
153
|
+
unless reasons.sort == %w[BOUNCE COMPLAINT]
|
|
154
|
+
return PolicyCheck.new(verified: false, reason: "the SES account-level suppression list covers #{reasons.inspect}; " \
|
|
155
|
+
"Bouncy needs both BOUNCE and COMPLAINT enabled for the account")
|
|
156
|
+
end
|
|
157
|
+
unless @settings.all_sending_paths_listed
|
|
158
|
+
return PolicyCheck.new(verified: false, reason: "config.ses.all_sending_paths_listed is false: list every identity and " \
|
|
159
|
+
"configuration set you send through, then set it to true")
|
|
160
|
+
end
|
|
161
|
+
|
|
162
|
+
sets = @settings.configuration_sets.dup
|
|
163
|
+
@settings.identities.each do |identity|
|
|
164
|
+
response = request { ses.get_email_identity(email_identity: identity) }
|
|
165
|
+
sets << response.configuration_set_name if response.configuration_set_name.present?
|
|
166
|
+
end
|
|
167
|
+
sets.uniq.each do |name|
|
|
168
|
+
suppression = request { ses.get_configuration_set(configuration_set_name: name) }.suppression_options
|
|
169
|
+
next if suppression.nil? || suppression.suppressed_reasons.nil? || suppression.suppressed_reasons.sort == %w[BOUNCE COMPLAINT]
|
|
170
|
+
|
|
171
|
+
return PolicyCheck.new(verified: false, reason: "configuration set #{name} overrides suppression with " \
|
|
172
|
+
"#{suppression.suppressed_reasons.inspect}; Bouncy needs BOUNCE and COMPLAINT")
|
|
173
|
+
end
|
|
174
|
+
PolicyCheck.new(verified: true, reason: nil)
|
|
175
|
+
end
|
|
176
|
+
|
|
177
|
+
def topic_checks
|
|
178
|
+
sns = sns_client
|
|
179
|
+
@settings.topic_arns.to_h do |arn|
|
|
180
|
+
response = request { sns.get_topic_attributes(topic_arn: arn) }
|
|
181
|
+
# Expose an explicit manual check; do not pretend a generic IAM evaluator.
|
|
182
|
+
policy = JSON.parse(response.attributes.fetch("Policy", "{}"))
|
|
183
|
+
[arn, { "readable" => true, "publisher_policy_review_required" => true,
|
|
184
|
+
"statements" => Array(policy["Statement"]).size }]
|
|
185
|
+
end
|
|
186
|
+
end
|
|
187
|
+
|
|
188
|
+
def reason(value)
|
|
189
|
+
case value
|
|
190
|
+
when "BOUNCE" then :hard_bounce
|
|
191
|
+
when "COMPLAINT" then :complaint
|
|
192
|
+
else raise ProviderError, "Unknown SES suppression reason"
|
|
193
|
+
end
|
|
194
|
+
end
|
|
195
|
+
end
|
|
196
|
+
end
|
|
197
|
+
end
|