cloudflare-email 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +73 -1
- data/README.md +265 -388
- data/SECURITY.md +51 -0
- data/app/controllers/cloudflare/email/ingress_controller.rb +62 -6
- data/docs/activerecord-tenanted.md +135 -0
- data/docs/architecture.md +95 -0
- data/docs/delivery-events.md +177 -0
- data/docs/features.md +107 -0
- data/docs/getting-started.md +298 -0
- data/docs/mailboxes.md +299 -0
- data/docs/outbox.md +175 -0
- data/docs/tenant-mailboxes-plan.md +149 -0
- data/docs/thread-correlation.md +27 -0
- data/docs/troubleshooting.md +103 -0
- data/docs/upgrading-0.2.md +96 -0
- data/docs/verification/2026-09-10-followup.md +49 -0
- data/docs/verification/2026-09-10-install-upgrade.md +86 -0
- data/docs/verification/2026-09-10-live.md +102 -0
- data/docs/verification/2026-09-10.md +175 -0
- data/docs/verification/2026-09-11-extraction.md +32 -0
- data/docs/verification/2026-09-11-outbound-ledger.md +60 -0
- data/docs/verification/2026-09-11-security.md +76 -0
- data/docs/verification/2026-09-11-tenant-mailboxes.md +75 -0
- data/examples/plain_ruby.rb +24 -0
- data/lib/cloudflare/email/active_record/base.rb +48 -0
- data/lib/cloudflare/email/active_record/delivery_events.rb +72 -0
- data/lib/cloudflare/email/active_record/event_inbox.rb +77 -0
- data/lib/cloudflare/email/active_record/event_receipt.rb +17 -0
- data/lib/cloudflare/email/active_record/mail_snapshot.rb +24 -0
- data/lib/cloudflare/email/active_record/outbound_delivery.rb +22 -0
- data/lib/cloudflare/email/active_record/outbound_recipient.rb +13 -0
- data/lib/cloudflare/email/active_record/outbound_reconciliation.rb +15 -0
- data/lib/cloudflare/email/active_record/outbox.rb +219 -0
- data/lib/cloudflare/email/active_record/outbox_notifications.rb +34 -0
- data/lib/cloudflare/email/active_record.rb +5 -0
- data/lib/cloudflare/email/client.rb +51 -11
- data/lib/cloudflare/email/consume_events_task.rb +22 -0
- data/lib/cloudflare/email/delivery_event.rb +70 -0
- data/lib/cloudflare/email/delivery_method.rb +3 -0
- data/lib/cloudflare/email/deploy_worker_task.rb +3 -0
- data/lib/cloudflare/email/dev_ingress_guard.rb +27 -0
- data/lib/cloudflare/email/dev_tunnel.rb +31 -5
- data/lib/cloudflare/email/doctor.rb +16 -38
- data/lib/cloudflare/email/endpoint.rb +24 -0
- data/lib/cloudflare/email/engine.rb +28 -0
- data/lib/cloudflare/email/envelope.rb +58 -0
- data/lib/cloudflare/email/event_consumer.rb +86 -0
- data/lib/cloudflare/email/mailboxes/configuration.rb +58 -0
- data/lib/cloudflare/email/mailboxes/events.rb +112 -0
- data/lib/cloudflare/email/mailboxes/inbound_retention.rb +14 -0
- data/lib/cloudflare/email/mailboxes/jobs.rb +34 -0
- data/lib/cloudflare/email/mailboxes/models.rb +148 -0
- data/lib/cloudflare/email/mailboxes/provider_correlation.rb +11 -0
- data/lib/cloudflare/email/mailboxes/service.rb +288 -0
- data/lib/cloudflare/email/mailboxes/shared_event_receipt.rb +14 -0
- data/lib/cloudflare/email/mailboxes.rb +11 -0
- data/lib/cloudflare/email/message_id.rb +12 -0
- data/lib/cloudflare/email/provision_catchall_task.rb +4 -4
- data/lib/cloudflare/email/provision_route_task.rb +2 -1
- data/lib/cloudflare/email/replay_events_job.rb +18 -0
- data/lib/cloudflare/email/response.rb +21 -3
- data/lib/cloudflare/email/routing_provisioner.rb +71 -45
- data/lib/cloudflare/email/send_job.rb +41 -0
- data/lib/cloudflare/email/send_test.rb +5 -29
- data/lib/cloudflare/email/signing.rb +2 -2
- data/lib/cloudflare/email/tenancy.rb +78 -0
- data/lib/cloudflare/email/tenant_job_context.rb +90 -0
- data/lib/cloudflare/email/verification.rb +30 -8
- data/lib/cloudflare/email/version.rb +1 -1
- data/lib/cloudflare/email/worker_deployer.rb +5 -3
- data/lib/cloudflare-email.rb +4 -1
- data/lib/generators/cloudflare/email/install_generator.rb +48 -33
- data/lib/generators/cloudflare/email/mailboxes/mailboxes_generator.rb +55 -0
- data/lib/generators/cloudflare/email/mailboxes/templates/create_cloudflare_email_mailboxes.rb +54 -0
- data/lib/generators/cloudflare/email/mailboxes/templates/create_cloudflare_email_receiving_domains.rb +16 -0
- data/lib/generators/cloudflare/email/mailboxes/templates/create_cloudflare_email_shared_events.rb +35 -0
- data/lib/generators/cloudflare/email/outbox/outbox_generator.rb +28 -0
- data/lib/generators/cloudflare/email/outbox/templates/create_cloudflare_email_outbox.rb +48 -0
- data/lib/generators/cloudflare/email/templates/initializer.rb +3 -3
- data/lib/generators/cloudflare/email/templates/main_mailbox.rb +2 -7
- data/lib/generators/cloudflare/email/tracking/templates/create_cloudflare_email_event_receipts.rb +22 -0
- data/lib/generators/cloudflare/email/tracking/tracking_generator.rb +26 -0
- data/lib/tasks/cloudflare_email.rake +41 -4
- data/templates/worker/README.md +61 -11
- data/templates/worker/package-lock.json +2627 -0
- data/templates/worker/package.json +8 -5
- data/templates/worker/scripts/wrangler.mjs +16 -0
- data/templates/worker/src/index.js +83 -4
- data/templates/worker/test/index.test.ts +117 -5
- data/templates/worker/wrangler.toml +13 -4
- metadata +107 -6
- data/lib/cloudflare/email/secure_message_id.rb +0 -89
data/SECURITY.md
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# Security
|
|
2
|
+
|
|
3
|
+
The hardening described here is included in 0.2.0. Version 0.1.0 does not include
|
|
4
|
+
these fixes. Upgrade Rails and the bundled Worker together using the
|
|
5
|
+
[upgrade guide](docs/upgrading-0.2.md).
|
|
6
|
+
|
|
7
|
+
Rails integration is tested on patched Rails 7.2, 8.0, and 8.1; application
|
|
8
|
+
owners must update their own Rails, database adapter, and other dependencies.
|
|
9
|
+
The gem does not add Rails or SQLite as runtime dependencies.
|
|
10
|
+
|
|
11
|
+
Report sensitive findings privately to the maintainer at cole@dscribeai.com.
|
|
12
|
+
Include affected versions and impact; omit real credentials and private mail.
|
|
13
|
+
Do not place secrets or private message data in public issues.
|
|
14
|
+
|
|
15
|
+
## Deployment responsibilities
|
|
16
|
+
|
|
17
|
+
- Keep API and management tokens least-privileged and separate. Store tokens and
|
|
18
|
+
a strong ingress secret in a secret manager or Rails encrypted credentials.
|
|
19
|
+
Rotate compromised secrets on Rails and the Worker together. HTTPS endpoint
|
|
20
|
+
overrides are trusted operator configuration; never derive them from mail.
|
|
21
|
+
- Set request-size limits, timeouts and rate limits at the proxy/server. Rails
|
|
22
|
+
and the Worker bound MIME reads to 25 MiB by default (`MAX_EMAIL_BYTES`), but
|
|
23
|
+
cannot prevent upstream buffering or slow clients. Limits apply to raw MIME,
|
|
24
|
+
not just decoded attachments.
|
|
25
|
+
- Treat mail and attachments as untrusted content. The gem authenticates Worker
|
|
26
|
+
transport and SMTP envelope metadata, not a human sender's identity. Rendering,
|
|
27
|
+
sanitization, malware scanning and application authorization belong to the app.
|
|
28
|
+
- Protect raw MIME, attachments, outbox snapshots and event receipts with access
|
|
29
|
+
control, storage encryption, backups and retention policies. Avoid logging raw
|
|
30
|
+
responses/errors: provider details and application logs can contain private mail.
|
|
31
|
+
- Restrict queue producers and consumers. Event account/domain checks are
|
|
32
|
+
validation, not cryptographic signatures. Monitor unacknowledged malformed
|
|
33
|
+
messages, processing failures and unmatched receipts.
|
|
34
|
+
- Database administrators and direct SQL writes are trusted. ActiveRecord
|
|
35
|
+
read-only evidence fields do not provide tamper-proof storage. Keep tenant
|
|
36
|
+
authorization around outbox operations and receipt access in the application.
|
|
37
|
+
- The optional mailbox session scopes tenant and mailbox access, but does not
|
|
38
|
+
authenticate users. Keep domain registration/activation administrative, verify
|
|
39
|
+
ownership externally, and use your access resolver to choose tenant keys.
|
|
40
|
+
Ingress selects a registered tenant only after full signature verification.
|
|
41
|
+
Raw ActiveRecord access remains privileged, especially in shared-database mode.
|
|
42
|
+
Private tenant storage, consistent migrations and trusted job payloads are
|
|
43
|
+
required. Drain old framework jobs before enabling database tenancy; jobs
|
|
44
|
+
without the new tenant metadata are rejected rather than guessing a tenant.
|
|
45
|
+
- Development tunnels expose the ingress to the internet. Use the bundled task,
|
|
46
|
+
dedicated development mail routes and test data; stop tunnels when finished.
|
|
47
|
+
The Host guard restricts routing but is not a sandbox for the development app
|
|
48
|
+
or a replacement for safe error pages and request logging.
|
|
49
|
+
|
|
50
|
+
See [the September 11 security review](docs/verification/2026-09-11-security.md)
|
|
51
|
+
for findings, verification, and remaining limits.
|
|
@@ -5,26 +5,46 @@ module Cloudflare
|
|
|
5
5
|
# ActionMailbox ingress for Cloudflare Email Worker forwards.
|
|
6
6
|
#
|
|
7
7
|
# The shipped Worker template signs each forwarded message with HMAC-SHA256
|
|
8
|
-
# over "{timestamp}.{raw_body}" and sends:
|
|
8
|
+
# over "v2.{timestamp}.{encoded_envelope}.{raw_body}" and sends:
|
|
9
9
|
# X-CF-Email-Timestamp: <unix seconds>
|
|
10
10
|
# X-CF-Email-Signature: <hex digest>
|
|
11
|
+
# X-CF-Email-Signature-Version: 2
|
|
12
|
+
# X-CF-Email-Envelope: <unpadded base64url JSON from/to>
|
|
13
|
+
# Requests must include the v2 signature and authenticated SMTP envelope.
|
|
11
14
|
#
|
|
12
15
|
# Set the shared secret in Rails credentials under cloudflare.ingress_secret
|
|
13
16
|
# (or in the CLOUDFLARE_INGRESS_SECRET env var) and as the Worker secret
|
|
14
17
|
# INGRESS_SECRET via `wrangler secret put INGRESS_SECRET`.
|
|
15
18
|
class IngressController < ActionMailbox::BaseController
|
|
19
|
+
DEFAULT_MAX_EMAIL_BYTES = 25 * 1024 * 1024
|
|
16
20
|
param_encoding :create, "raw_email", Encoding::ASCII_8BIT
|
|
17
21
|
|
|
18
22
|
def create
|
|
19
23
|
ActiveSupport::Notifications.instrument(
|
|
20
24
|
"cloudflare_email.ingress",
|
|
21
|
-
bytes:
|
|
25
|
+
bytes: 0,
|
|
22
26
|
) do |payload|
|
|
27
|
+
preflight = Cloudflare::Email::Verification.verify_headers(secret: secret,
|
|
28
|
+
timestamp: request.headers["X-CF-Email-Timestamp"],
|
|
29
|
+
signature: request.headers["X-CF-Email-Signature"],
|
|
30
|
+
version: request.headers["X-CF-Email-Signature-Version"],
|
|
31
|
+
envelope: request.headers["X-CF-Email-Envelope"])
|
|
32
|
+
unless preflight == :ok
|
|
33
|
+
payload[:result] = preflight
|
|
34
|
+
next head(preflight == :stale ? :request_timeout : :unauthorized)
|
|
35
|
+
end
|
|
36
|
+
if request.content_length.to_i > max_email_bytes || raw_body.bytesize > max_email_bytes
|
|
37
|
+
payload[:result] = :too_large
|
|
38
|
+
next head(:payload_too_large)
|
|
39
|
+
end
|
|
40
|
+
payload[:bytes] = raw_body.bytesize
|
|
23
41
|
case Cloudflare::Email::Verification.verify(
|
|
24
42
|
secret: secret,
|
|
25
43
|
body: raw_body,
|
|
26
44
|
timestamp: request.headers["X-CF-Email-Timestamp"],
|
|
27
45
|
signature: request.headers["X-CF-Email-Signature"],
|
|
46
|
+
version: request.headers["X-CF-Email-Signature-Version"],
|
|
47
|
+
envelope: request.headers["X-CF-Email-Envelope"],
|
|
28
48
|
)
|
|
29
49
|
when :stale
|
|
30
50
|
payload[:result] = :stale
|
|
@@ -33,9 +53,19 @@ module Cloudflare
|
|
|
33
53
|
payload[:result] = :bad_signature
|
|
34
54
|
head :unauthorized
|
|
35
55
|
when :ok
|
|
36
|
-
inbound =
|
|
37
|
-
|
|
38
|
-
|
|
56
|
+
inbound = if defined?(Cloudflare::Email::Mailboxes) && Cloudflare::Email::Mailboxes.enabled?
|
|
57
|
+
recipient = Cloudflare::Email::Envelope.decode(request.headers["X-CF-Email-Envelope"]).fetch("to")
|
|
58
|
+
begin
|
|
59
|
+
Cloudflare::Email::Mailboxes.receive(recipient: recipient) { persist_inbound }
|
|
60
|
+
rescue Cloudflare::Email::Mailboxes::Unavailable
|
|
61
|
+
payload[:result] = :unavailable_mailbox
|
|
62
|
+
next head(:unprocessable_entity)
|
|
63
|
+
end
|
|
64
|
+
else
|
|
65
|
+
persist_inbound
|
|
66
|
+
end
|
|
67
|
+
payload[:result] = inbound ? :ok : :duplicate
|
|
68
|
+
payload[:message_id] = inbound&.message_id
|
|
39
69
|
head :ok
|
|
40
70
|
end
|
|
41
71
|
end
|
|
@@ -43,6 +73,24 @@ module Cloudflare
|
|
|
43
73
|
|
|
44
74
|
private
|
|
45
75
|
|
|
76
|
+
def persist_inbound
|
|
77
|
+
envelope = Cloudflare::Email::Envelope.decode(request.headers["X-CF-Email-Envelope"])
|
|
78
|
+
|
|
79
|
+
# Commit trusted routing metadata before ActionMailbox's after_create_commit
|
|
80
|
+
# enqueues routing. Identical MIME for To/Cc/Bcc recipients is independent.
|
|
81
|
+
checksum = OpenSSL::Digest::SHA256.hexdigest("v2\0#{envelope.fetch('to')}\0".b + raw_body.b)
|
|
82
|
+
ActionMailbox::InboundEmail.transaction do
|
|
83
|
+
inbound = ActionMailbox::InboundEmail.create_and_extract_message_id!(raw_body, message_checksum: checksum)
|
|
84
|
+
if inbound
|
|
85
|
+
blob = inbound.raw_email.blob
|
|
86
|
+
blob.update!(metadata: blob.metadata.merge(
|
|
87
|
+
Cloudflare::Email::Envelope::METADATA_KEY => envelope.merge("version" => 2),
|
|
88
|
+
))
|
|
89
|
+
end
|
|
90
|
+
inbound
|
|
91
|
+
end
|
|
92
|
+
end
|
|
93
|
+
|
|
46
94
|
# Override ActionMailbox::BaseController's default name inference so
|
|
47
95
|
# `config.action_mailbox.ingress = :cloudflare` gates this controller.
|
|
48
96
|
def ingress_name
|
|
@@ -52,7 +100,15 @@ module Cloudflare
|
|
|
52
100
|
def raw_body
|
|
53
101
|
@raw_body ||= begin
|
|
54
102
|
request.body.rewind if request.body.respond_to?(:rewind)
|
|
55
|
-
request.body.read
|
|
103
|
+
request.body.read(max_email_bytes + 1).to_s
|
|
104
|
+
end
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
def max_email_bytes
|
|
108
|
+
@max_email_bytes ||= begin
|
|
109
|
+
value = Integer(ENV.fetch("MAX_EMAIL_BYTES", DEFAULT_MAX_EMAIL_BYTES.to_s), 10)
|
|
110
|
+
raise ArgumentError, "MAX_EMAIL_BYTES must be positive" unless value.positive?
|
|
111
|
+
value
|
|
56
112
|
end
|
|
57
113
|
end
|
|
58
114
|
|
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
# SQLite mailboxes with activerecord-tenanted
|
|
2
|
+
|
|
3
|
+
Use this setup when each organization has its own SQLite database. The shared database holds receiving domains and delivery-event routing. Mailboxes, messages, and the outbox live in the organization's database.
|
|
4
|
+
|
|
5
|
+
This setup is entirely opt-in. Skip this guide for a single database: installing
|
|
6
|
+
the gem or running the mailbox generator does not enable database tenancy.
|
|
7
|
+
|
|
8
|
+
`activerecord-tenanted` is an optional application dependency. The gem's integration test uses version 0.8 on Rails 8.1; you do not need it for a single database or another tenant adapter.
|
|
9
|
+
|
|
10
|
+
## 1. Configure your databases
|
|
11
|
+
|
|
12
|
+
Add `gem "activerecord-tenanted", "~> 0.8.0"` to your application's Gemfile and run `bundle install`.
|
|
13
|
+
|
|
14
|
+
For example, in `config/database.yml`:
|
|
15
|
+
|
|
16
|
+
```yaml
|
|
17
|
+
production:
|
|
18
|
+
primary:
|
|
19
|
+
adapter: sqlite3
|
|
20
|
+
database: storage/directory.sqlite3
|
|
21
|
+
migrations_paths: db/migrate
|
|
22
|
+
tenant:
|
|
23
|
+
adapter: sqlite3
|
|
24
|
+
tenanted: true
|
|
25
|
+
database: storage/tenants/%{tenant}/db.sqlite3
|
|
26
|
+
migrations_paths: db/tenant_migrate
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
Use equivalent entries for development and test. Keep tenant keys stable, such as an organization's internal identifier. Customer-supplied hostnames are not tenant authorization.
|
|
30
|
+
|
|
31
|
+
Generate the mailbox migrations in the correct directories:
|
|
32
|
+
|
|
33
|
+
```sh
|
|
34
|
+
bin/rails generate cloudflare:email:mailboxes \
|
|
35
|
+
--directory-migrations-path=db/migrate \
|
|
36
|
+
--tenant-migrations-path=db/tenant_migrate
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Put Action Mailbox and Active Storage migrations in the tenant migration directory too. Apply the shared migrations and the tenant migrations using your application's database provisioning workflow. Keep every existing tenant's schema current before starting workers against a new release.
|
|
40
|
+
|
|
41
|
+
## 2. Load stable connection classes before the mailbox models
|
|
42
|
+
|
|
43
|
+
The optional gem models inherit from your tenant base. That base must keep the same Ruby class identity across Rails development reloads. Define it in a file you explicitly require, outside reloadable model directories. If your app already has these bases, use those same classes; do not create competing versions.
|
|
44
|
+
|
|
45
|
+
For example, `lib/email_database_records.rb`:
|
|
46
|
+
|
|
47
|
+
```ruby
|
|
48
|
+
class TenantRecord < ActiveRecord::Base
|
|
49
|
+
self.abstract_class = true
|
|
50
|
+
tenanted "tenant"
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
class DirectoryRecord < ActiveRecord::Base
|
|
54
|
+
self.abstract_class = true
|
|
55
|
+
connects_to database: { writing: :primary }
|
|
56
|
+
end
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Keep this file outside any paths your application configures for reloadable autoloading. Require it during application initialization after `activerecord-tenanted` has installed its Active Record support, and before the generated mailbox initializer loads the optional models. Add this inside your application class in `config/application.rb`:
|
|
60
|
+
|
|
61
|
+
```ruby
|
|
62
|
+
config.active_record_tenanted.connection_class = "TenantRecord"
|
|
63
|
+
config.active_record_tenanted.tenanted_rails_records = true
|
|
64
|
+
|
|
65
|
+
config.active_record_tenanted.tenant_resolver = ->(request) do
|
|
66
|
+
if request.path == "/rails/action_mailbox/cloudflare/inbound_emails"
|
|
67
|
+
nil
|
|
68
|
+
else
|
|
69
|
+
request.subdomain # Replace with your existing authorized tenant resolver.
|
|
70
|
+
end
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
initializer "my_app.cloudflare_email_tenancy",
|
|
74
|
+
after: "active_record_tenanted.active_record_base",
|
|
75
|
+
before: :load_config_initializers do
|
|
76
|
+
require Rails.root.join("lib/email_database_records").to_s
|
|
77
|
+
require "cloudflare/email/tenancy"
|
|
78
|
+
require "cloudflare/email/mailboxes/configuration"
|
|
79
|
+
|
|
80
|
+
Cloudflare::Email::Tenancy.configure(
|
|
81
|
+
base_class: TenantRecord,
|
|
82
|
+
current: -> { TenantRecord.current_tenant },
|
|
83
|
+
switch: ->(key, &block) {
|
|
84
|
+
unless TenantRecord.tenant_exist?(key)
|
|
85
|
+
raise Cloudflare::Email::Mailboxes::Unavailable,
|
|
86
|
+
"tenant is not provisioned"
|
|
87
|
+
end
|
|
88
|
+
TenantRecord.with_tenant(key, &block)
|
|
89
|
+
}
|
|
90
|
+
)
|
|
91
|
+
Cloudflare::Email::Mailboxes.configure(directory_base: DirectoryRecord)
|
|
92
|
+
end
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
Leave the generated `cloudflare_email_mailboxes.rb` initializer to require `cloudflare/email/mailboxes`. Remove any duplicate tenancy configuration from other initializers.
|
|
96
|
+
|
|
97
|
+
The ingress exception in the resolver matters: `activerecord-tenanted` normally locks a request to the tenant selected from its hostname. Email ingress must select its tenant from the verified recipient and shared domain directory, after the gem verifies the complete message signature. Returning `nil` lets that signed-recipient lookup select the correct database.
|
|
98
|
+
|
|
99
|
+
`tenanted_rails_records = true` puts Action Mailbox and Active Storage on the configured tenant connection. Include their tables in tenant migrations and configure private storage. The gem restores its tenant context for its supported email and attachment jobs. For your own jobs that serialize tenant model arguments, prepend `Cloudflare::Email::TenantJobContext` and enqueue them inside `Cloudflare::Email::Tenancy.with(key)`; ordinary host tenant context alone does not establish the gem's context.
|
|
100
|
+
|
|
101
|
+
## 3. Provision organizations separately from mailboxes
|
|
102
|
+
|
|
103
|
+
Create tenant databases in your trusted organization-provisioning workflow:
|
|
104
|
+
|
|
105
|
+
```ruby
|
|
106
|
+
TenantRecord.create_tenant("organization-123")
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
The switch adapter above refuses missing tenants. Receiving mail or replaying a delivery event will never create a new tenant database implicitly.
|
|
110
|
+
|
|
111
|
+
After registering and activating the organization's receiving domain, use the mailbox API within its context:
|
|
112
|
+
|
|
113
|
+
```ruby
|
|
114
|
+
Cloudflare::Email::Mailboxes.for_tenant("organization-123") do |inboxes|
|
|
115
|
+
mailbox = inboxes.create(
|
|
116
|
+
name: "Support",
|
|
117
|
+
address: "support@customer.example.com",
|
|
118
|
+
owner_ref: "team:42"
|
|
119
|
+
)
|
|
120
|
+
# Provision and activate the address before receiving or sending mail.
|
|
121
|
+
end
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
See the [mailbox guide](mailboxes.md) for domain activation, address provisioning, sending, and event replay. Your application still authorizes which organization, mailbox, and message each user can access.
|
|
125
|
+
|
|
126
|
+
## What is verified
|
|
127
|
+
|
|
128
|
+
The optional CI job boots a real Rails application with `activerecord-tenanted`, runs the gem's tenant migrations in two SQLite databases, and checks identical numeric IDs remain isolated. It also verifies context restoration, refusal to create unknown tenant databases, and GlobalID's missing/wrong-tenant rejection. Separate integration tests exercise mailbox ingress, sending, jobs, and delivery-event replay.
|
|
129
|
+
|
|
130
|
+
Run the library-specific smoke test locally with:
|
|
131
|
+
|
|
132
|
+
```sh
|
|
133
|
+
BUNDLE_GEMFILE=gemfiles/tenanted.gemfile bundle install
|
|
134
|
+
BUNDLE_GEMFILE=gemfiles/tenanted.gemfile bundle exec ruby test/support/activerecord_tenanted.rb
|
|
135
|
+
```
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
# Gem and inbox responsibilities
|
|
2
|
+
|
|
3
|
+
The Cloudflare gem supplies email transport and reusable delivery infrastructure.
|
|
4
|
+
The inbox application supplies a product built on that infrastructure. Plain Ruby
|
|
5
|
+
users should not need Rails or a database. Rails users should be able to opt into
|
|
6
|
+
durable infrastructure without copying the reference inbox's models and services.
|
|
7
|
+
|
|
8
|
+
## Implemented in this extraction
|
|
9
|
+
|
|
10
|
+
| Gem capability | Application responsibility |
|
|
11
|
+
| --- | --- |
|
|
12
|
+
| Optional mailbox/address directory and tenant connection adapter | Authorize domain ownership, provision tenant databases and grant user access |
|
|
13
|
+
| Mailbox memberships, read/archive state and retained raw mail | Inbox UI, retention schedule and explicit purge policy |
|
|
14
|
+
| Shared event intake, tenant correlation and tenant-aware jobs | Durable queue workers, scheduled recovery and schema rollout to every tenant |
|
|
15
|
+
| Structured/raw sending and ActionMailer transport | Compose UI, recipients and send authorization |
|
|
16
|
+
| `Response#accepted?`, plus individual recipient outcomes | Handle partial acceptance without resending accepted recipients |
|
|
17
|
+
| Authenticated v2 ingress and recipient-scoped deduplication | Map trusted recipient to an authorized mailbox |
|
|
18
|
+
| `MessageId.normalize` and returned provider IDs | Store conversation membership and scope reply lookups |
|
|
19
|
+
| Queue decoding, validation and ACK | Queue/subscription setup and recurring execution |
|
|
20
|
+
| Optional ActiveRecord event receipts, deduplication and indexed replay | Configure queue polling and receipt retention |
|
|
21
|
+
| Immutable outbox snapshots, send claims, per-recipient outcomes | Authorize sending and choose a stable operation key |
|
|
22
|
+
| `DeliveryEvents` account/message/recipient correlation and ordering | Transactional callback to update product records |
|
|
23
|
+
| Send/replay jobs, recovery tasks and audited reconciliation | Authorize operators and provide evidence of provider outcome |
|
|
24
|
+
|
|
25
|
+
The ActiveRecord adapter is explicit opt-in. Its generator installs a receipt table
|
|
26
|
+
and initializer; the default plain Ruby client does not load ActiveRecord. Its
|
|
27
|
+
handler applies application writes on the same database connection as the receipt.
|
|
28
|
+
It does not wrap HTTP calls or other external effects in a database transaction.
|
|
29
|
+
|
|
30
|
+
Version 0.2 is a preproduction break: v1 ingress and signed Message-ID tokens are
|
|
31
|
+
removed. Cloudflare's documented queue encodings and recipient-response forms are
|
|
32
|
+
provider contracts, so accepting them is not obsolete application compatibility.
|
|
33
|
+
Historical verification reports describe their dated runs, not the current API.
|
|
34
|
+
|
|
35
|
+
## Implemented: durable outbound delivery
|
|
36
|
+
|
|
37
|
+
The inbox now uses the gem's ledger tables and orchestration. Its model subclasses
|
|
38
|
+
add application message associations and display snapshots; they do not maintain
|
|
39
|
+
a second sending ledger. See the [outbox guide](outbox.md) for installation and API.
|
|
40
|
+
|
|
41
|
+
1. Claim an application-supplied operation key scoped to an account using a unique
|
|
42
|
+
index before sending; repeated or concurrent calls cannot silently resend.
|
|
43
|
+
2. Persist immutable rendered MIME and envelope recipients before the network
|
|
44
|
+
request. Track each recipient separately, including partial acceptance.
|
|
45
|
+
3. Separate preparation failures from ambiguous outcomes. A timeout, process death
|
|
46
|
+
or persistence failure after acceptance keeps the operation blocked. An elapsed
|
|
47
|
+
timeout alone never proves the message was not sent.
|
|
48
|
+
4. Persist validated provider IDs and recipient acceptance outcomes separately
|
|
49
|
+
from later lifecycle events. Replay correlates account, message and recipient;
|
|
50
|
+
conflicting IDs remain uncertain. The network and database are not atomic.
|
|
51
|
+
5. Expose audited reconciliation with caller-supplied actor/reason and evidence.
|
|
52
|
+
The application authorizes the operator; the gem enforces legal transitions.
|
|
53
|
+
6. Provide jobs/tasks and notifications that preserve the same claim on retries.
|
|
54
|
+
Include migration/import support for the reference inbox's existing attempts.
|
|
55
|
+
|
|
56
|
+
The suite exercises concurrent senders, actual process termination, acceptance
|
|
57
|
+
followed by persistence failure, partial recipients, event-before-response,
|
|
58
|
+
out-of-order events, callbacks that roll back, and recovery without a duplicate
|
|
59
|
+
send. PostgreSQL and SQLite run separate concurrency checks. Inbox HTTP and browser
|
|
60
|
+
workflows use the shared implementation, with controlled local provider responses.
|
|
61
|
+
|
|
62
|
+
Historic attempts did not preserve rendered MIME. Their import retains available
|
|
63
|
+
bodies, IDs, outcomes and operator evidence and marks reconstructed snapshots.
|
|
64
|
+
Unknown historical outcomes remain blocked. No migration claims to recreate the
|
|
65
|
+
exact bytes of a previously sent message.
|
|
66
|
+
|
|
67
|
+
## Operational completion
|
|
68
|
+
|
|
69
|
+
The reference inbox supplies a functional mailbox product over these APIs. Before
|
|
70
|
+
production use, configure sending DNS and receiving routes, queue subscriptions
|
|
71
|
+
and dead-letter handling, durable Rails jobs, receipt/MIME retention and backups,
|
|
72
|
+
and monitoring for prepared/sending/unknown/partial operations. Exercise live
|
|
73
|
+
external mailbox delivery and recovery with the deployed revision. Installation
|
|
74
|
+
does not automatically provision domains or deploy infrastructure.
|
|
75
|
+
|
|
76
|
+
Current limitations are deliberate and visible: no provider exactly-once API,
|
|
77
|
+
no automatic resend of uncertain or rejected operations, no multi-provider-ID
|
|
78
|
+
operation (conflicting IDs require review), and no durable inbound Worker buffer.
|
|
79
|
+
A prepared operation can be dispatched using its existing identity after a job
|
|
80
|
+
enqueue failure; sending/unknown operations require evidence-based reconciliation.
|
|
81
|
+
Notifications indicate method execution and may run inside an outer transaction;
|
|
82
|
+
the durable database record is authoritative.
|
|
83
|
+
|
|
84
|
+
## Keep in the inbox product
|
|
85
|
+
|
|
86
|
+
Users, tenancy, mailbox ownership, conversations, folders, search and UI remain in
|
|
87
|
+
the inbox. AI context, review flags, approval thresholds, auto-send policy and
|
|
88
|
+
operator permissions also stay there. Suppression/unsubscribe requirements depend
|
|
89
|
+
on message purpose and application policy; expose provider outcomes, but do not
|
|
90
|
+
silently impose a marketing subscription model on transactional mail.
|
|
91
|
+
|
|
92
|
+
The optional [mailbox module](mailboxes.md) now packages mailbox persistence and
|
|
93
|
+
address management over the existing transport/receipt APIs. It includes a
|
|
94
|
+
[tenant adapter](activerecord-tenanted.md), without adding a tenant library to
|
|
95
|
+
the plain Ruby client. A future inbox UI or starter should consume these APIs.
|
|
@@ -0,0 +1,177 @@
|
|
|
1
|
+
# Delivery events
|
|
2
|
+
|
|
3
|
+
Cloudflare Email Sending can publish outbound lifecycle events to Queues. This is separate from inbound Email Routing and the ActionMailbox ingress.
|
|
4
|
+
|
|
5
|
+
## Provision once
|
|
6
|
+
|
|
7
|
+
1. Create a dedicated Cloudflare Queue for your application's sending events.
|
|
8
|
+
2. In that queue's **Subscriptions** tab, subscribe to Email Sending events for your verified sending domain. Subscriptions are domain-specific; add another for each sending domain you need.
|
|
9
|
+
3. Select delivered, deferred, bounced, failed, rejected, and complained events.
|
|
10
|
+
4. Enable **HTTP pull** as the queue consumer. Do not also attach a Worker push consumer.
|
|
11
|
+
5. Configure queue retries and a dead-letter queue so failed or unsupported events can be investigated instead of disappearing after retry exhaustion.
|
|
12
|
+
6. Create an account-scoped **Queues Read + Write** token (dashboard Queues Edit). Writing is necessary to acknowledge messages.
|
|
13
|
+
|
|
14
|
+
These are explicit Cloudflare setup steps, not actions performed by installing this gem. See [subscription management](https://developers.cloudflare.com/queues/event-subscriptions/manage-event-subscriptions/) and [HTTP pull consumers](https://developers.cloudflare.com/queues/configuration/pull-consumers/).
|
|
15
|
+
|
|
16
|
+
## Rails setup
|
|
17
|
+
|
|
18
|
+
Add credentials:
|
|
19
|
+
|
|
20
|
+
```yaml
|
|
21
|
+
cloudflare:
|
|
22
|
+
account_id: your-account-id
|
|
23
|
+
queues_token: your-queues-read-write-token
|
|
24
|
+
event_queue_id: your-queue-id
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Environment alternatives are `CLOUDFLARE_ACCOUNT_ID`, `CLOUDFLARE_QUEUES_TOKEN`, and `CLOUDFLARE_EVENT_QUEUE_ID`. Queue consumption uses its own token; it does not use the management-token fallback.
|
|
28
|
+
|
|
29
|
+
Configure a callable application handler:
|
|
30
|
+
|
|
31
|
+
```ruby
|
|
32
|
+
# config/initializers/cloudflare_delivery_events.rb
|
|
33
|
+
Rails.application.configure do
|
|
34
|
+
config.x.cloudflare_email.event_domains = ["mail.example.com"]
|
|
35
|
+
config.x.cloudflare_email.event_handler = ->(event) {
|
|
36
|
+
DeliveryEventProcessor.call(event)
|
|
37
|
+
}
|
|
38
|
+
end
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
`DeliveryEventProcessor` supplies application correlation and policy. For Rails,
|
|
42
|
+
use the optional durable receipt adapter below to avoid implementing receipt
|
|
43
|
+
storage, deduplication and replay yourself. Custom handlers must persist durably
|
|
44
|
+
before returning; do not merely enqueue a non-durable job.
|
|
45
|
+
|
|
46
|
+
### Durable Rails receipts
|
|
47
|
+
|
|
48
|
+
If you also use the [durable outbox](outbox.md), prefer `DeliveryEvents.record`
|
|
49
|
+
and `DeliveryEvents.replay`: they provide account/message/recipient correlation
|
|
50
|
+
and state ordering automatically. The lower-level example below supports an
|
|
51
|
+
application's own delivery schema.
|
|
52
|
+
|
|
53
|
+
```sh
|
|
54
|
+
bin/rails generate cloudflare:email:tracking
|
|
55
|
+
bin/rails db:migrate
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
The generator installs a receipt table and an initializer that explicitly loads
|
|
59
|
+
`cloudflare/email/active_record/event_inbox`. ActiveRecord is not loaded by the
|
|
60
|
+
plain Ruby client. Configure the queue handler to commit receipts:
|
|
61
|
+
|
|
62
|
+
```ruby
|
|
63
|
+
Rails.application.config.x.cloudflare_email.event_handler = ->(event) {
|
|
64
|
+
Cloudflare::Email::ActiveRecord::EventInbox.record(event)
|
|
65
|
+
}
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
Run replay from your recurring job and after saving a send's provider ID:
|
|
69
|
+
|
|
70
|
+
```ruby
|
|
71
|
+
Cloudflare::Email::ActiveRecord::EventInbox.replay(account_id: account_id) do |event|
|
|
72
|
+
# Your own model; always scope by account, message AND recipient.
|
|
73
|
+
delivery = RecipientDelivery.find_by(
|
|
74
|
+
account_id: event.account_id,
|
|
75
|
+
provider_message_id: Cloudflare::Email::MessageId.normalize(event.message_id),
|
|
76
|
+
recipient: event.recipient,
|
|
77
|
+
)
|
|
78
|
+
next :unmatched unless delivery
|
|
79
|
+
|
|
80
|
+
delivery.with_lock do
|
|
81
|
+
if event.supersedes?(occurred_at: delivery.occurred_at, terminal: delivery.terminal?)
|
|
82
|
+
delivery.update!(status: event.status, occurred_at: event.occurred_at,
|
|
83
|
+
terminal: event.terminal?)
|
|
84
|
+
end
|
|
85
|
+
end
|
|
86
|
+
:applied
|
|
87
|
+
end
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
`record` stores the raw event and enforces uniqueness by account/event ID. An ID
|
|
91
|
+
repeated with different content raises. Recording inside an existing database
|
|
92
|
+
transaction raises, because returning before that transaction commits would allow
|
|
93
|
+
premature acknowledgment. An acknowledged receipt can still be pending or unmatched;
|
|
94
|
+
monitor and schedule replay separately from queue polling.
|
|
95
|
+
|
|
96
|
+
`apply(receipt) { |event| ... }` and `replay` lock each receipt and skip applied
|
|
97
|
+
records. Handlers must return `:applied` or `:unmatched`; exceptions and other return
|
|
98
|
+
values roll back the receipt transition and writes on the same database connection.
|
|
99
|
+
`:unmatched` is a successful transaction outcome, so avoid business writes before
|
|
100
|
+
returning it. External effects and writes to other databases cannot be rolled back.
|
|
101
|
+
Concurrent database conflicts raise; retry the job. Do not send email inside this
|
|
102
|
+
handler; use durable outbound orchestration.
|
|
103
|
+
|
|
104
|
+
Replay accepts optional `account_id:`, normalized provider `message_id:`, and
|
|
105
|
+
`batch_size:` filters. The batch size controls database fetch size, not the total
|
|
106
|
+
number processed. `DeliveryEvent#supersedes?` accepts a Time or ISO8601 previous
|
|
107
|
+
timestamp: only a newer known status replaces current state, equal timestamps keep
|
|
108
|
+
existing state, and a nonterminal event cannot replace terminal state. A later
|
|
109
|
+
terminal complaint can replace delivery. Invalid timestamps raise. Match records
|
|
110
|
+
and lock recipient state before using this ordering helper.
|
|
111
|
+
|
|
112
|
+
Receipt retention, recurring jobs, and monitoring are application decisions. See
|
|
113
|
+
the [gem/inbox boundary](architecture.md) for outbound-ledger scope. Existing
|
|
114
|
+
receipt tables from the earlier preproduction adapter must normalize stored
|
|
115
|
+
`message_id` values with `MessageId.normalize` before using filtered replay;
|
|
116
|
+
raw payloads remain unchanged. The reference inbox's outbox migration does this.
|
|
117
|
+
|
|
118
|
+
Poll a single batch:
|
|
119
|
+
|
|
120
|
+
```sh
|
|
121
|
+
bin/rails cloudflare:email:consume_events
|
|
122
|
+
BATCH_SIZE=20 bin/rails cloudflare:email:consume_events
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Run the task from your existing scheduler, or invoke `EventConsumer#poll` from a recurring job. It short-polls once and exits; an empty queue is a successful result. No handler means an error before any queue request, not automatic discard.
|
|
126
|
+
|
|
127
|
+
For more control, use the API directly:
|
|
128
|
+
|
|
129
|
+
```ruby
|
|
130
|
+
consumer = Cloudflare::Email::EventConsumer.new(
|
|
131
|
+
account_id: Cloudflare::Email::Credentials.account_id,
|
|
132
|
+
api_token: Cloudflare::Email::Credentials.fetch(:queues_token),
|
|
133
|
+
queue_id: Cloudflare::Email::Credentials.fetch(:event_queue_id),
|
|
134
|
+
domains: ["mail.example.com"],
|
|
135
|
+
)
|
|
136
|
+
|
|
137
|
+
count = consumer.poll(batch_size: 5, visibility_timeout_ms: 300_000) do |event|
|
|
138
|
+
DeliveryEventProcessor.call(event)
|
|
139
|
+
end
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
The same API works in plain Ruby with explicit credentials.
|
|
143
|
+
|
|
144
|
+
## Event data
|
|
145
|
+
|
|
146
|
+
| Method | Meaning |
|
|
147
|
+
|---|---|
|
|
148
|
+
| `event_id` | Durable application deduplication key |
|
|
149
|
+
| `message_id` | Cloudflare message ID for correlation with the send response |
|
|
150
|
+
| `status` | delivered, deferred, bounced, failed, rejected, complained |
|
|
151
|
+
| `known?` | Whether status is one of those six current types |
|
|
152
|
+
| `terminal?` | Cloudflare's terminal flag; not a guarantee no later complaint arrives |
|
|
153
|
+
| `sender`, `recipient` | Addresses from the event |
|
|
154
|
+
| `account_id`, `domain` | Event source, checked against consumer configuration |
|
|
155
|
+
| `occurred_at` | Original event timestamp string |
|
|
156
|
+
| `delivery`, `bounce`, `rejection`, `failure`, `complaint` | Detail hashes |
|
|
157
|
+
| `raw`, `payload`, `type` | Original provider data |
|
|
158
|
+
|
|
159
|
+
Record outcomes per message **and recipient**. A delivery event means the recipient's server accepted the message, not that the person read it. Use bounce/complaint events to maintain your application's recipient eligibility and unsubscribe state. Cloudflare's own suppression behavior remains controlled by Cloudflare; the consumer does not automatically edit provider suppression lists or your database.
|
|
160
|
+
|
|
161
|
+
Events can be duplicated or arrive out of order. Do not blindly overwrite a newer terminal state with an older deferred event. Decide how your application handles a complaint after delivery. Provider IDs may be absent from older send responses; retain event data even when an application record cannot yet be found.
|
|
162
|
+
|
|
163
|
+
## Processing guarantees
|
|
164
|
+
|
|
165
|
+
For each message, the consumer decodes the body, validates the event, invokes your handler, and then acknowledges its lease. Email Sending subscriptions can return plain JSON strings with `CF-Content-Type: json`; the consumer also accepts base64-encoded JSON for that content type. `bytes` bodies contain base64-encoded JSON, while `text` bodies contain JSON directly. Invalid bodies and events remain unacknowledged.
|
|
166
|
+
|
|
167
|
+
- A handler that returns normally counts as successful, even if its return value is `false` or `nil`. **Raise** to prevent acknowledgement.
|
|
168
|
+
- A parse error, mismatched account/domain, handler exception, or acknowledgement failure stops the batch. Previously acknowledged messages remain acknowledged; the current and remaining leases become available after the visibility timeout.
|
|
169
|
+
- If the handler commits and acknowledgement fails, the event can be processed again. This is why the handler needs durable idempotency.
|
|
170
|
+
- Automatic HTTP retries are disabled for queue calls. Your scheduler can retry the next poll.
|
|
171
|
+
- The default batch size is 5 (max 100); the default visibility window is five minutes. Choose a window longer than the **whole batch's** worst-case processing time, up to Cloudflare's 12-hour maximum.
|
|
172
|
+
- Unknown future status names in schema version 1 are passed through with `known? == false`. Decide whether to store or raise. Unsupported schema versions are rejected and left unacknowledged.
|
|
173
|
+
- Configure dead-letter retention/alerts. Repeated failure eventually exhausts the queue's retry policy.
|
|
174
|
+
|
|
175
|
+
The `cloudflare_email.delivery_event` notification wraps handler execution with event ID, message ID, and status. It does not mean acknowledgement succeeded. Monitor task/job failures as well.
|
|
176
|
+
|
|
177
|
+
Source schemas: [Email Sending event subscriptions](https://developers.cloudflare.com/email-service/platform/event-subscriptions/), [pull API](https://developers.cloudflare.com/api/resources/queues/subresources/messages/methods/pull/), [ack API](https://developers.cloudflare.com/api/resources/queues/subresources/messages/methods/ack/).
|