cloudflare-email 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +149 -1
- data/README.md +287 -385
- data/SECURITY.md +51 -0
- data/app/controllers/cloudflare/email/ingress_controller.rb +33 -21
- data/app/controllers/cloudflare/email/management/mailboxes_controller.rb +173 -0
- data/app/controllers/cloudflare/email/management/styles_controller.rb +13 -0
- data/app/views/cloudflare/email/management/mailboxes/index.html.erb +65 -0
- data/app/views/cloudflare/email/management/mailboxes/message.html.erb +34 -0
- data/app/views/cloudflare/email/management/mailboxes/show.html.erb +86 -0
- data/app/views/layouts/cloudflare/email/management.html.erb +29 -0
- data/docs/activerecord-tenanted.md +181 -0
- data/docs/architecture.md +105 -0
- data/docs/custom-ingress.md +207 -0
- data/docs/delivery-events.md +186 -0
- data/docs/features.md +138 -0
- data/docs/getting-started.md +308 -0
- data/docs/mailboxes.md +363 -0
- data/docs/management-engine.md +190 -0
- data/docs/outbox.md +175 -0
- data/docs/routing-deliveries.md +94 -0
- data/docs/routing-diagnostics.md +68 -0
- data/docs/tenant-mailboxes-plan.md +149 -0
- data/docs/thread-correlation.md +27 -0
- data/docs/troubleshooting.md +104 -0
- data/docs/upgrading-0.2.md +101 -0
- data/docs/upgrading-0.3.md +55 -0
- data/docs/verification/2026-09-10-followup.md +49 -0
- data/docs/verification/2026-09-10-install-upgrade.md +86 -0
- data/docs/verification/2026-09-10-live.md +102 -0
- data/docs/verification/2026-09-10.md +175 -0
- data/docs/verification/2026-09-11-custom-ingress.md +49 -0
- data/docs/verification/2026-09-11-extraction.md +32 -0
- data/docs/verification/2026-09-11-outbound-ledger.md +60 -0
- data/docs/verification/2026-09-11-security.md +76 -0
- data/docs/verification/2026-09-11-tenant-mailboxes.md +75 -0
- data/docs/verification/2026-09-12-destination-context.md +46 -0
- data/docs/verification/2026-09-13-durable-inbound-and-routing.md +71 -0
- data/examples/plain_ruby.rb +24 -0
- data/lib/cloudflare/email/active_record/base.rb +60 -0
- data/lib/cloudflare/email/active_record/delivery_events.rb +72 -0
- data/lib/cloudflare/email/active_record/event_inbox.rb +77 -0
- data/lib/cloudflare/email/active_record/event_receipt.rb +17 -0
- data/lib/cloudflare/email/active_record/mail_snapshot.rb +24 -0
- data/lib/cloudflare/email/active_record/outbound_delivery.rb +22 -0
- data/lib/cloudflare/email/active_record/outbound_recipient.rb +13 -0
- data/lib/cloudflare/email/active_record/outbound_reconciliation.rb +15 -0
- data/lib/cloudflare/email/active_record/outbox.rb +219 -0
- data/lib/cloudflare/email/active_record/outbox_notifications.rb +34 -0
- data/lib/cloudflare/email/active_record/routing_deliveries.rb +128 -0
- data/lib/cloudflare/email/active_record.rb +5 -0
- data/lib/cloudflare/email/check_route_task.rb +21 -0
- data/lib/cloudflare/email/client.rb +93 -18
- data/lib/cloudflare/email/consume_events_task.rb +22 -0
- data/lib/cloudflare/email/delivery_event.rb +70 -0
- data/lib/cloudflare/email/delivery_method.rb +5 -0
- data/lib/cloudflare/email/deploy_worker_task.rb +6 -3
- data/lib/cloudflare/email/dev_ingress_guard.rb +27 -0
- data/lib/cloudflare/email/dev_tunnel.rb +31 -5
- data/lib/cloudflare/email/doctor.rb +16 -38
- data/lib/cloudflare/email/endpoint.rb +24 -0
- data/lib/cloudflare/email/engine.rb +33 -0
- data/lib/cloudflare/email/envelope.rb +58 -0
- data/lib/cloudflare/email/event_consumer.rb +95 -0
- data/lib/cloudflare/email/ingress.rb +115 -0
- data/lib/cloudflare/email/mailboxes/configuration.rb +58 -0
- data/lib/cloudflare/email/mailboxes/events.rb +112 -0
- data/lib/cloudflare/email/mailboxes/inbound_retention.rb +14 -0
- data/lib/cloudflare/email/mailboxes/jobs.rb +34 -0
- data/lib/cloudflare/email/mailboxes/models.rb +165 -0
- data/lib/cloudflare/email/mailboxes/provider_correlation.rb +11 -0
- data/lib/cloudflare/email/mailboxes/service.rb +372 -0
- data/lib/cloudflare/email/mailboxes/shared_event_receipt.rb +14 -0
- data/lib/cloudflare/email/mailboxes.rb +11 -0
- data/lib/cloudflare/email/management/adapter.rb +31 -0
- data/lib/cloudflare/email/management/configuration.rb +17 -0
- data/lib/cloudflare/email/management/engine.rb +15 -0
- data/lib/cloudflare/email/management/management.css +68 -0
- data/lib/cloudflare/email/management/routes.rb +14 -0
- data/lib/cloudflare/email/management.rb +7 -0
- data/lib/cloudflare/email/message_id.rb +12 -0
- data/lib/cloudflare/email/provider_metadata.rb +77 -0
- data/lib/cloudflare/email/provision_catchall_task.rb +4 -4
- data/lib/cloudflare/email/provision_route_task.rb +2 -1
- data/lib/cloudflare/email/replay_events_job.rb +18 -0
- data/lib/cloudflare/email/response.rb +21 -3
- data/lib/cloudflare/email/routing_analytics.rb +196 -0
- data/lib/cloudflare/email/routing_diagnostics.rb +206 -0
- data/lib/cloudflare/email/routing_provisioner.rb +71 -45
- data/lib/cloudflare/email/send_job.rb +41 -0
- data/lib/cloudflare/email/send_test.rb +5 -29
- data/lib/cloudflare/email/signing.rb +2 -2
- data/lib/cloudflare/email/tenancy.rb +78 -0
- data/lib/cloudflare/email/tenant_job_context.rb +90 -0
- data/lib/cloudflare/email/verification.rb +38 -8
- data/lib/cloudflare/email/version.rb +1 -1
- data/lib/cloudflare/email/worker_deployer.rb +41 -8
- data/lib/cloudflare-email.rb +4 -1
- data/lib/generators/cloudflare/email/install_generator.rb +53 -37
- data/lib/generators/cloudflare/email/mailboxes/catch_all/catch_all_generator.rb +21 -0
- data/lib/generators/cloudflare/email/mailboxes/catch_all/templates/add_cloudflare_email_catch_all.rb +18 -0
- data/lib/generators/cloudflare/email/mailboxes/mailboxes_generator.rb +55 -0
- data/lib/generators/cloudflare/email/mailboxes/templates/create_cloudflare_email_mailboxes.rb +58 -0
- data/lib/generators/cloudflare/email/mailboxes/templates/create_cloudflare_email_receiving_domains.rb +16 -0
- data/lib/generators/cloudflare/email/mailboxes/templates/create_cloudflare_email_shared_events.rb +35 -0
- data/lib/generators/cloudflare/email/outbox/outbox_generator.rb +28 -0
- data/lib/generators/cloudflare/email/outbox/templates/create_cloudflare_email_outbox.rb +48 -0
- data/lib/generators/cloudflare/email/routing_tracking/routing_tracking_generator.rb +28 -0
- data/lib/generators/cloudflare/email/routing_tracking/templates/create_cloudflare_email_routing_delivery_receipts.rb +22 -0
- data/lib/generators/cloudflare/email/templates/initializer.rb +3 -3
- data/lib/generators/cloudflare/email/templates/main_mailbox.rb +2 -7
- data/lib/generators/cloudflare/email/tracking/templates/create_cloudflare_email_event_receipts.rb +22 -0
- data/lib/generators/cloudflare/email/tracking/tracking_generator.rb +26 -0
- data/lib/tasks/cloudflare_email.rake +42 -4
- data/lib/tasks/cloudflare_email_diagnostics.rake +11 -0
- data/templates/deploy-to-cloudflare/.dev.vars.example +3 -0
- data/templates/deploy-to-cloudflare/.gitignore +6 -0
- data/templates/deploy-to-cloudflare/LICENSE.txt +21 -0
- data/templates/deploy-to-cloudflare/README.md +84 -0
- data/templates/deploy-to-cloudflare/package-lock.json +2627 -0
- data/templates/deploy-to-cloudflare/package.json +36 -0
- data/templates/deploy-to-cloudflare/src/index.js +445 -0
- data/templates/deploy-to-cloudflare/test/durable.test.ts +193 -0
- data/templates/deploy-to-cloudflare/test/index.test.ts +309 -0
- data/templates/deploy-to-cloudflare/test/relay.test.ts +226 -0
- data/templates/deploy-to-cloudflare/vitest.config.ts +8 -0
- data/templates/deploy-to-cloudflare/wrangler.toml +22 -0
- data/templates/worker/README.md +231 -16
- data/templates/worker/docs/durable-inbound.md +149 -0
- data/templates/worker/package-lock.json +2627 -0
- data/templates/worker/package.json +8 -5
- data/templates/worker/scripts/wrangler.mjs +16 -0
- data/templates/worker/src/index.js +397 -25
- data/templates/worker/test/durable.test.ts +193 -0
- data/templates/worker/test/index.test.ts +183 -13
- data/templates/worker/test/relay.test.ts +226 -0
- data/templates/worker/wrangler.toml +63 -4
- metadata +154 -7
- data/lib/cloudflare/email/secure_message_id.rb +0 -89
|
@@ -0,0 +1,181 @@
|
|
|
1
|
+
# SQLite mailboxes with activerecord-tenanted
|
|
2
|
+
|
|
3
|
+
Use this setup when each organization has its own SQLite database. The shared database holds receiving domains and delivery-event routing. Mailboxes, messages, and the outbox live in the organization's database.
|
|
4
|
+
|
|
5
|
+
This setup is entirely opt-in. Skip this guide for a single database: installing
|
|
6
|
+
the gem or running the mailbox generator does not enable database tenancy.
|
|
7
|
+
|
|
8
|
+
`activerecord-tenanted` is an optional application dependency. The gem's integration test uses version 0.8 on Rails 8.1; you do not need it for a single database or another tenant adapter.
|
|
9
|
+
|
|
10
|
+
## 1. Configure your databases
|
|
11
|
+
|
|
12
|
+
Add `gem "activerecord-tenanted", "~> 0.8.0"` to your application's Gemfile and run `bundle install`.
|
|
13
|
+
|
|
14
|
+
For example, in `config/database.yml`:
|
|
15
|
+
|
|
16
|
+
```yaml
|
|
17
|
+
production:
|
|
18
|
+
primary:
|
|
19
|
+
adapter: sqlite3
|
|
20
|
+
database: storage/directory.sqlite3
|
|
21
|
+
migrations_paths: db/migrate
|
|
22
|
+
tenant:
|
|
23
|
+
adapter: sqlite3
|
|
24
|
+
tenanted: true
|
|
25
|
+
database: storage/tenants/%{tenant}/db.sqlite3
|
|
26
|
+
migrations_paths: db/tenant_migrate
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
Use equivalent entries for development and test. Keep tenant keys stable, such as an organization's internal identifier. Customer-supplied hostnames are not tenant authorization.
|
|
30
|
+
|
|
31
|
+
Generate the mailbox migrations in the correct directories:
|
|
32
|
+
|
|
33
|
+
```sh
|
|
34
|
+
bin/rails generate cloudflare:email:mailboxes \
|
|
35
|
+
--directory-migrations-path=db/migrate \
|
|
36
|
+
--tenant-migrations-path=db/tenant_migrate
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Put Action Mailbox and Active Storage migrations in the tenant migration directory too. Apply the shared migrations and the tenant migrations using your application's database provisioning workflow. Keep every existing tenant's schema current before starting workers against a new release.
|
|
40
|
+
|
|
41
|
+
## 2. Load stable connection classes before the mailbox models
|
|
42
|
+
|
|
43
|
+
The optional gem models inherit from your tenant base. That base must keep the same Ruby class identity across Rails development reloads. Define it in a file you explicitly require, outside reloadable model directories. If your app already has these bases, use those same classes; do not create competing versions.
|
|
44
|
+
|
|
45
|
+
For example, `lib/email_database_records.rb`:
|
|
46
|
+
|
|
47
|
+
```ruby
|
|
48
|
+
class TenantRecord < ActiveRecord::Base
|
|
49
|
+
self.abstract_class = true
|
|
50
|
+
tenanted "tenant"
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
class DirectoryRecord < ActiveRecord::Base
|
|
54
|
+
self.abstract_class = true
|
|
55
|
+
connects_to database: { writing: :primary }
|
|
56
|
+
end
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Keep this file outside any paths your application configures for reloadable autoloading. Require it during application initialization after `activerecord-tenanted` has installed its Active Record support, and before the generated mailbox initializer loads the optional models. Add this inside your application class in `config/application.rb`:
|
|
60
|
+
|
|
61
|
+
```ruby
|
|
62
|
+
config.active_record_tenanted.connection_class = "TenantRecord"
|
|
63
|
+
config.active_record_tenanted.tenanted_rails_records = true
|
|
64
|
+
|
|
65
|
+
config.active_record_tenanted.tenant_resolver = ->(request) do
|
|
66
|
+
if request.path == "/rails/action_mailbox/cloudflare/inbound_emails"
|
|
67
|
+
nil
|
|
68
|
+
else
|
|
69
|
+
request.subdomain # Replace with your existing authorized tenant resolver.
|
|
70
|
+
end
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
initializer "my_app.cloudflare_email_tenancy",
|
|
74
|
+
after: "active_record_tenanted.active_record_base",
|
|
75
|
+
before: :load_config_initializers do
|
|
76
|
+
require Rails.root.join("lib/email_database_records").to_s
|
|
77
|
+
require "cloudflare/email/tenancy"
|
|
78
|
+
require "cloudflare/email/mailboxes/configuration"
|
|
79
|
+
|
|
80
|
+
Cloudflare::Email::Tenancy.configure(
|
|
81
|
+
base_class: TenantRecord,
|
|
82
|
+
current: -> { TenantRecord.current_tenant },
|
|
83
|
+
switch: ->(key, &block) {
|
|
84
|
+
unless TenantRecord.tenant_exist?(key)
|
|
85
|
+
raise Cloudflare::Email::Mailboxes::Unavailable,
|
|
86
|
+
"tenant is not provisioned"
|
|
87
|
+
end
|
|
88
|
+
TenantRecord.with_tenant(key, &block)
|
|
89
|
+
}
|
|
90
|
+
)
|
|
91
|
+
Cloudflare::Email::Mailboxes.configure(directory_base: DirectoryRecord)
|
|
92
|
+
end
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
Leave the generated `cloudflare_email_mailboxes.rb` initializer to require `cloudflare/email/mailboxes`. Remove any duplicate tenancy configuration from other initializers.
|
|
96
|
+
|
|
97
|
+
The ingress exception in the resolver matters: `activerecord-tenanted` normally locks a request to the tenant selected from its hostname. Email ingress must select its tenant from the verified recipient and shared domain directory, after the gem verifies the complete message signature. Returning `nil` lets that signed-recipient lookup select the correct database.
|
|
98
|
+
|
|
99
|
+
`tenanted_rails_records = true` puts Action Mailbox and Active Storage on the configured tenant connection. Include their tables in tenant migrations and configure private storage. The gem restores its tenant context for its supported email and attachment jobs. For your own jobs that serialize tenant model arguments, prepend `Cloudflare::Email::TenantJobContext` and enqueue them inside `Cloudflare::Email::Tenancy.with(key)`; ordinary host tenant context alone does not establish the gem's context.
|
|
100
|
+
|
|
101
|
+
## 3. Provision organizations separately from mailboxes
|
|
102
|
+
|
|
103
|
+
Create tenant databases in your trusted organization-provisioning workflow:
|
|
104
|
+
|
|
105
|
+
```ruby
|
|
106
|
+
TenantRecord.create_tenant("organization-123")
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
The switch adapter above refuses missing tenants. Receiving mail or replaying a delivery event will never create a new tenant database implicitly.
|
|
110
|
+
|
|
111
|
+
After registering and activating the organization's receiving domain, use the mailbox API within its context:
|
|
112
|
+
|
|
113
|
+
```ruby
|
|
114
|
+
Cloudflare::Email::Mailboxes.for_tenant("organization-123") do |inboxes|
|
|
115
|
+
mailbox = inboxes.create(
|
|
116
|
+
name: "Support",
|
|
117
|
+
address: "support@customer.example.com",
|
|
118
|
+
owner_ref: "team:42"
|
|
119
|
+
)
|
|
120
|
+
# Provision and activate the address before receiving or sending mail.
|
|
121
|
+
end
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
See the [mailbox guide](mailboxes.md) for domain activation, address provisioning, sending, and event replay. Your application still authorizes which organization, mailbox, and message each user can access.
|
|
125
|
+
|
|
126
|
+
## Account emails without an organization
|
|
127
|
+
|
|
128
|
+
Password resets and portal invitations can belong to the application rather
|
|
129
|
+
than an organization. If you enabled separate tenant databases, give those
|
|
130
|
+
outbox records an explicitly provisioned, host-reserved database too. Keep the
|
|
131
|
+
business record's organization empty; a storage key is not an organization or
|
|
132
|
+
an authorization grant.
|
|
133
|
+
|
|
134
|
+
For example, after configuring the adapter above, provision and migrate a
|
|
135
|
+
reserved database through your deployment workflow:
|
|
136
|
+
|
|
137
|
+
```ruby
|
|
138
|
+
# Run once through trusted provisioning, not during sending or event replay.
|
|
139
|
+
TenantRecord.create_tenant("_system_outbound")
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
Your own service can select storage consistently for sending and feedback:
|
|
143
|
+
|
|
144
|
+
```ruby
|
|
145
|
+
module EmailStorage
|
|
146
|
+
SYSTEM_KEY = "_system_outbound".freeze
|
|
147
|
+
|
|
148
|
+
def self.with_outbound_storage(organization_key, &block)
|
|
149
|
+
# The caller must supply an already-authorized organization key.
|
|
150
|
+
key = organization_key.nil? ? SYSTEM_KEY : organization_key
|
|
151
|
+
Cloudflare::Email::Tenancy.with(key, &block)
|
|
152
|
+
end
|
|
153
|
+
end
|
|
154
|
+
|
|
155
|
+
EmailStorage.with_outbound_storage(nil) do
|
|
156
|
+
# Prepare/read/reconcile the system email's saved outbox operation here.
|
|
157
|
+
end
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
Reserve the key against customer selection, include this database in migrations,
|
|
161
|
+
backups and restoration drills, and use the same selection for receipt replay.
|
|
162
|
+
If your shared business records use another database, retain their own durable
|
|
163
|
+
receipt/projection boundary: a tenant commit and a shared commit are not atomic.
|
|
164
|
+
Replay a committed gem operation rather than sending it again.
|
|
165
|
+
|
|
166
|
+
Never modify the tenancy adapter to treat every missing context as the system
|
|
167
|
+
database. Missing context must continue to fail closed. Your organization's job
|
|
168
|
+
fan-out should enumerate real organizations, not every database file. Applications
|
|
169
|
+
without configured multi-tenancy can keep ordinary system and organization mail
|
|
170
|
+
in their existing single database; this recipe does not enable tenancy by default.
|
|
171
|
+
|
|
172
|
+
## What is verified
|
|
173
|
+
|
|
174
|
+
The optional CI job boots a real Rails application with `activerecord-tenanted`, runs the gem's tenant migrations in two SQLite databases, and checks identical numeric IDs remain isolated. It also verifies context restoration, refusal to create unknown tenant databases, and GlobalID's missing/wrong-tenant rejection. Separate integration tests exercise mailbox ingress, sending, jobs, and delivery-event replay.
|
|
175
|
+
|
|
176
|
+
Run the library-specific smoke test locally with:
|
|
177
|
+
|
|
178
|
+
```sh
|
|
179
|
+
BUNDLE_GEMFILE=gemfiles/tenanted.gemfile bundle install
|
|
180
|
+
BUNDLE_GEMFILE=gemfiles/tenanted.gemfile bundle exec ruby test/support/activerecord_tenanted.rb
|
|
181
|
+
```
|
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
# Gem and inbox responsibilities
|
|
2
|
+
|
|
3
|
+
The Cloudflare gem supplies email transport and reusable delivery infrastructure.
|
|
4
|
+
The inbox application supplies a product built on that infrastructure. Plain Ruby
|
|
5
|
+
users should not need Rails or a database. Rails users should be able to opt into
|
|
6
|
+
durable infrastructure without copying the reference inbox's models and services.
|
|
7
|
+
|
|
8
|
+
## Implemented in this extraction
|
|
9
|
+
|
|
10
|
+
| Gem capability | Application responsibility |
|
|
11
|
+
| --- | --- |
|
|
12
|
+
| Optional mailbox/address directory and tenant connection adapter | Authorize domain ownership, provision tenant databases and grant user access |
|
|
13
|
+
| Mailbox memberships, read/archive state and retained raw mail | Inbox UI, retention schedule and explicit purge policy |
|
|
14
|
+
| Shared event intake, tenant correlation and tenant-aware jobs | Durable queue workers, scheduled recovery and schema rollout to every tenant |
|
|
15
|
+
| Structured/raw sending and ActionMailer transport | Compose UI, recipients and send authorization |
|
|
16
|
+
| `Response#accepted?`, plus individual recipient outcomes | Handle partial acceptance without resending accepted recipients |
|
|
17
|
+
| Authenticated ingress (default v2; opt-in v3 [metadata](custom-ingress.md)) and context-scoped deduplication | Map trusted recipient to an authorized mailbox; apply sender and document policy |
|
|
18
|
+
| `MessageId.normalize` and returned provider IDs | Store conversation membership and scope reply lookups |
|
|
19
|
+
| Queue decoding, validation and ACK | Queue/subscription setup and recurring execution |
|
|
20
|
+
| Optional ActiveRecord event receipts, deduplication and indexed replay | Configure queue polling and receipt retention |
|
|
21
|
+
| Immutable outbox snapshots, send claims, per-recipient outcomes | Authorize sending and choose a stable operation key |
|
|
22
|
+
| `DeliveryEvents` account/message/recipient correlation and ordering | Transactional callback to update product records |
|
|
23
|
+
| Send/replay jobs, recovery tasks and audited reconciliation | Authorize operators and provide evidence of provider outcome |
|
|
24
|
+
|
|
25
|
+
The ActiveRecord adapter is explicit opt-in. Its generator installs a receipt table
|
|
26
|
+
and initializer; the default plain Ruby client does not load ActiveRecord. Its
|
|
27
|
+
handler applies application writes on the same database connection as the receipt.
|
|
28
|
+
It does not wrap HTTP calls or other external effects in a database transaction.
|
|
29
|
+
|
|
30
|
+
Version 0.2 is a preproduction break: v1 ingress and signed Message-ID tokens are
|
|
31
|
+
removed. Cloudflare's documented queue encodings and recipient-response forms are
|
|
32
|
+
provider contracts, so accepting them is not obsolete application compatibility.
|
|
33
|
+
Historical verification reports describe their dated runs, not the current API.
|
|
34
|
+
|
|
35
|
+
## Implemented: durable outbound delivery
|
|
36
|
+
|
|
37
|
+
The inbox now uses the gem's ledger tables and orchestration. Its model subclasses
|
|
38
|
+
add application message associations and display snapshots; they do not maintain
|
|
39
|
+
a second sending ledger. See the [outbox guide](outbox.md) for installation and API.
|
|
40
|
+
|
|
41
|
+
1. Claim an application-supplied operation key scoped to an account using a unique
|
|
42
|
+
index before sending; repeated or concurrent calls cannot silently resend.
|
|
43
|
+
2. Persist immutable rendered MIME and envelope recipients before the network
|
|
44
|
+
request. Track each recipient separately, including partial acceptance.
|
|
45
|
+
3. Separate preparation failures from ambiguous outcomes. A timeout, process death
|
|
46
|
+
or persistence failure after acceptance keeps the operation blocked. An elapsed
|
|
47
|
+
timeout alone never proves the message was not sent.
|
|
48
|
+
4. Persist validated provider IDs and recipient acceptance outcomes separately
|
|
49
|
+
from later lifecycle events. Replay correlates account, message and recipient;
|
|
50
|
+
conflicting IDs remain uncertain. The network and database are not atomic.
|
|
51
|
+
5. Expose audited reconciliation with caller-supplied actor/reason and evidence.
|
|
52
|
+
The application authorizes the operator; the gem enforces legal transitions.
|
|
53
|
+
6. Provide jobs/tasks and notifications that preserve the same claim on retries.
|
|
54
|
+
Include migration/import support for the reference inbox's existing attempts.
|
|
55
|
+
|
|
56
|
+
The suite exercises concurrent senders, actual process termination, acceptance
|
|
57
|
+
followed by persistence failure, partial recipients, event-before-response,
|
|
58
|
+
out-of-order events, callbacks that roll back, and recovery without a duplicate
|
|
59
|
+
send. PostgreSQL and SQLite run separate concurrency checks. Inbox HTTP and browser
|
|
60
|
+
workflows use the shared implementation, with controlled local provider responses.
|
|
61
|
+
|
|
62
|
+
Historic attempts did not preserve rendered MIME. Their import retains available
|
|
63
|
+
bodies, IDs, outcomes and operator evidence and marks reconstructed snapshots.
|
|
64
|
+
Unknown historical outcomes remain blocked. No migration claims to recreate the
|
|
65
|
+
exact bytes of a previously sent message.
|
|
66
|
+
|
|
67
|
+
## Operational completion
|
|
68
|
+
|
|
69
|
+
The reference inbox supplies a functional mailbox product over these APIs. Before
|
|
70
|
+
production use, configure sending DNS and receiving routes, queue subscriptions
|
|
71
|
+
and dead-letter handling, durable Rails jobs, receipt/MIME retention and backups,
|
|
72
|
+
and monitoring for prepared/sending/unknown/partial operations. Exercise live
|
|
73
|
+
external mailbox delivery and recovery with the deployed revision. Installation
|
|
74
|
+
does not automatically provision domains or deploy infrastructure.
|
|
75
|
+
|
|
76
|
+
Current limitations are deliberate and visible: no provider exactly-once API,
|
|
77
|
+
no automatic resend of uncertain or rejected operations, no multi-provider-ID
|
|
78
|
+
operation (conflicting IDs require review). Direct inbound forwarding can reject
|
|
79
|
+
mail during a Rails outage. The default [durable Worker path](../templates/worker/README.md#durable-inbound-delivery)
|
|
80
|
+
commits raw mail and stable envelope context to R2 before returning successfully.
|
|
81
|
+
Queue handoffs and a scheduled recovery pass retry pending mail; a Rails 2xx
|
|
82
|
+
allows removal from the pending store. Queue expiration does not remove R2 mail.
|
|
83
|
+
This boundary requires durable Rails persistence before 2xx, idempotent replay,
|
|
84
|
+
private R2 storage without expiration of pending objects, and operational
|
|
85
|
+
monitoring. It cannot guarantee acceptance when Cloudflare itself cannot store
|
|
86
|
+
the incoming message.
|
|
87
|
+
A prepared operation can be dispatched using its existing identity after a job
|
|
88
|
+
enqueue failure; sending/unknown operations require evidence-based reconciliation.
|
|
89
|
+
Notifications indicate method execution and may run inside an outer transaction;
|
|
90
|
+
the durable database record is authoritative.
|
|
91
|
+
|
|
92
|
+
## Keep in the inbox product
|
|
93
|
+
|
|
94
|
+
Users, tenancy, mailbox ownership, conversations, folders, search and UI remain in
|
|
95
|
+
the inbox. AI context, review flags, approval thresholds, auto-send policy and
|
|
96
|
+
operator permissions also stay there. Suppression/unsubscribe requirements depend
|
|
97
|
+
on message purpose and application policy; expose provider outcomes, but do not
|
|
98
|
+
silently impose a marketing subscription model on transactional mail.
|
|
99
|
+
|
|
100
|
+
The optional [mailbox module](mailboxes.md) now packages mailbox persistence and
|
|
101
|
+
address management over the existing transport/receipt APIs. It includes a
|
|
102
|
+
[tenant adapter](activerecord-tenanted.md), without adding a tenant library to
|
|
103
|
+
the plain Ruby client. The optional [management engine](management-engine.md)
|
|
104
|
+
consumes these same APIs with host-supplied authentication and authorization.
|
|
105
|
+
Full inbox products can continue using them through their own frontends.
|
|
@@ -0,0 +1,207 @@
|
|
|
1
|
+
# Use your existing inbound email pipeline
|
|
2
|
+
|
|
3
|
+
This integration API is available in gem version 0.3.0.
|
|
4
|
+
|
|
5
|
+
You can adopt mailbox registration without replacing your Worker, document processing, sender review, or archive. Use the gem to authenticate the request and resolve an accepted address, then keep your application's policy and processing.
|
|
6
|
+
|
|
7
|
+
A useful starting point is one mailbox per site: an organization owns a receiving domain, a site owns a mailbox, and that mailbox can have several accepted addresses. This is an application convention, not a requirement of the gem. Database tenancy is optional; a single SQLite database works too.
|
|
8
|
+
|
|
9
|
+
## Register addresses before accepting mail
|
|
10
|
+
|
|
11
|
+
Follow [mailbox setup](mailboxes.md) to register and activate the receiving domain and accepted addresses. With existing catch-all Worker routing, address creation can be a database operation once an administrator has confirmed that routing covers the exact domain. Registering a domain does not create DNS records or extend wildcard coverage.
|
|
12
|
+
|
|
13
|
+
Use the same provisioning service from organization/site onboarding and from the management UI. Keep a stable site reference in the mailbox's `owner_ref`; keep site permissions, slug changes, document workflows, and sender allowlists in your app. Treat a rename as an explicit alias/lifecycle decision so old addresses are not accidentally lost.
|
|
14
|
+
|
|
15
|
+
For separate organization databases, configure [the SQLite tenant adapter](activerecord-tenanted.md) before loading mailbox models. The shared directory resolves the signed recipient to a tenant. Action Mailbox, Active Storage, and the gem's mailbox records must all use that tenant's connection.
|
|
16
|
+
|
|
17
|
+
## Verify a custom endpoint before writing anything
|
|
18
|
+
|
|
19
|
+
`Cloudflare::Email::Ingress.verify` is usable outside the bundled controller. It accepts a String or IO body, bounds the raw body size, and returns an immutable result. Successful verification exposes the exact binary body and authenticated envelope; rejected requests expose no verified message.
|
|
20
|
+
|
|
21
|
+
The following controller is a starting point for a host with registered mailboxes:
|
|
22
|
+
|
|
23
|
+
```ruby
|
|
24
|
+
require "cloudflare/email/ingress"
|
|
25
|
+
|
|
26
|
+
class InboundEmailsController < ActionController::API
|
|
27
|
+
def create
|
|
28
|
+
result = Cloudflare::Email::Ingress.verify(
|
|
29
|
+
secret: Rails.application.credentials.dig(:cloudflare, :ingress_secret),
|
|
30
|
+
headers: request.headers,
|
|
31
|
+
body: request.body,
|
|
32
|
+
content_length: request.content_length,
|
|
33
|
+
max_email_bytes: 25 * 1024 * 1024
|
|
34
|
+
)
|
|
35
|
+
|
|
36
|
+
case result.status
|
|
37
|
+
when :too_large
|
|
38
|
+
return head :payload_too_large
|
|
39
|
+
when :bad_signature, :stale
|
|
40
|
+
return head :unauthorized
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
verified = result.message
|
|
44
|
+
|
|
45
|
+
Cloudflare::Email::Mailboxes.receive(
|
|
46
|
+
recipient: verified.envelope.fetch("to")
|
|
47
|
+
) do |destination|
|
|
48
|
+
# This block runs in the recipient's tenant. Apply your sender/review
|
|
49
|
+
# policy here, before persistence schedules Action Mailbox routing.
|
|
50
|
+
# Your policy may raise a host error or return nil instead of persisting.
|
|
51
|
+
# destination.owner_ref identifies your site/team; validate it using
|
|
52
|
+
# your application's policy in this tenant context.
|
|
53
|
+
verified.persist_action_mailbox!
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
head :ok
|
|
57
|
+
rescue Cloudflare::Email::Mailboxes::Unavailable
|
|
58
|
+
head :unprocessable_entity
|
|
59
|
+
end
|
|
60
|
+
end
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
Mount this controller at your own route and point your existing Worker there. The host owns response codes and retry policy. If your policy returns nil and you acknowledge the request, you are accepting responsibility for that email: persist a durable review record or deliberately discard it according to your application's policy. A successful verification alone does not queue processing or create a mailbox.
|
|
64
|
+
|
|
65
|
+
Do not use the MIME `To` header or an unauthenticated URL subdomain to select storage. `verified.envelope.fetch("to")` is the signed SMTP recipient. `Mailboxes.receive` checks that its domain, mailbox, and accepted address are active before entering the persistence block.
|
|
66
|
+
|
|
67
|
+
`persist_action_mailbox!` stores the verified raw bytes and metadata in Action Mailbox. The receiving block adds mailbox membership in the same tenant transaction. Rails schedules normal routing after the transaction commits. Apply policies that must stop processing before calling persistence; a check performed after receiving returns can be too late.
|
|
68
|
+
|
|
69
|
+
For an application that owns a different raw-email store, use `verified.body`, `verified.envelope`, `verified.provider_metadata`, `verified.message_checksum`, and `verified.storage_metadata` with your own persistence/transaction system. `Mailboxes.receive` specifically expects an Action Mailbox inbound email record (or nil) from its block; do not pass an unrelated processing record. The gem does not choose your archive retention, held-message model, or document queue.
|
|
70
|
+
|
|
71
|
+
## Persist the email and link your business record
|
|
72
|
+
|
|
73
|
+
Email persistence remains part of the gem's supported path. Action Mailbox stores
|
|
74
|
+
the raw MIME using Active Storage, the verifier saves authenticated metadata, and
|
|
75
|
+
`Mailboxes.receive` adds the inbox membership. Your processing/review record can
|
|
76
|
+
refer to that email rather than storing a second copy of its raw content:
|
|
77
|
+
|
|
78
|
+
```ruby
|
|
79
|
+
Cloudflare::Email::Mailboxes.receive(
|
|
80
|
+
recipient: verified.envelope.fetch("to")
|
|
81
|
+
) do |destination|
|
|
82
|
+
# Validate owner_ref and sender policy here, before any routing is scheduled.
|
|
83
|
+
inbound = verified.persist_action_mailbox!
|
|
84
|
+
if inbound
|
|
85
|
+
# Example host model, on the SAME tenant connection as the gem records.
|
|
86
|
+
EmailIntake.create!(
|
|
87
|
+
inbound_email_id: inbound.id,
|
|
88
|
+
mailbox_id: destination.mailbox_id,
|
|
89
|
+
owner_ref: destination.owner_ref
|
|
90
|
+
)
|
|
91
|
+
end
|
|
92
|
+
inbound # Return the ActionMailbox record; duplicate deliveries return nil.
|
|
93
|
+
end
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
The database writes commit together. A host failure rolls back the new email,
|
|
97
|
+
membership and host link and prevents the routing job from being enqueued. This
|
|
98
|
+
does not make an R2 archive, another database, or external API calls transactional.
|
|
99
|
+
Host exceptions propagate; only unavailable registry destinations are translated
|
|
100
|
+
to `Mailboxes::Unavailable`. Existing blocks without a destination argument still
|
|
101
|
+
work.
|
|
102
|
+
|
|
103
|
+
Persisting through ActionMailbox schedules its normal routing after commit. Put
|
|
104
|
+
your processing in that route or a deliberately coordinated job pipeline. If an
|
|
105
|
+
email must be held, implement a routing policy that cannot process it before
|
|
106
|
+
approval; storing it is not itself a review gate. A routing job failure also needs
|
|
107
|
+
normal job retry/recovery. Receipt deduplication does not make every downstream
|
|
108
|
+
business side effect idempotent.
|
|
109
|
+
|
|
110
|
+
Managed mailbox membership protects raw email from ActionMailbox's normal
|
|
111
|
+
automatic incineration. Archiving a mailbox message keeps its content; deliberate
|
|
112
|
+
purging can remove it once no other membership references the email. Raw email
|
|
113
|
+
saved without membership follows the normal ActionMailbox lifecycle. Keep host
|
|
114
|
+
references, backup and R2 retention consistent with your chosen deletion policy.
|
|
115
|
+
Multi-tenancy remains opt-in.
|
|
116
|
+
|
|
117
|
+
## Resolve an address with your own persistence
|
|
118
|
+
|
|
119
|
+
If your application already owns its raw-email store, use the standalone API:
|
|
120
|
+
|
|
121
|
+
```ruby
|
|
122
|
+
Cloudflare::Email::Mailboxes.with_recipient(
|
|
123
|
+
recipient: verified.envelope.fetch("to")
|
|
124
|
+
) do |destination|
|
|
125
|
+
# Example host-owned persistence service; it must commit durably before ACK.
|
|
126
|
+
ExistingEmailStore.save!(
|
|
127
|
+
tenant_key: destination.tenant_key,
|
|
128
|
+
mailbox_id: destination.mailbox_id,
|
|
129
|
+
recipient: destination.recipient,
|
|
130
|
+
raw: verified.body,
|
|
131
|
+
metadata: verified.storage_metadata
|
|
132
|
+
)
|
|
133
|
+
end
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
This checks the active domain, address and mailbox, enters the configured tenant,
|
|
137
|
+
and yields an immutable `Destination` with `tenant_key`, `mailbox_id`, `address_id`,
|
|
138
|
+
`receiving_domain_id`, `recipient` and `owner_ref`. It returns the block's result,
|
|
139
|
+
creates no email or membership, starts no storage transaction, and requires no
|
|
140
|
+
ActionMailbox. Configure the optional mailbox registry and tenant adapter as usual.
|
|
141
|
+
|
|
142
|
+
Both APIs must be called after complete request verification. Destination values
|
|
143
|
+
are routing snapshots, not authorization tokens: an owner reference does not
|
|
144
|
+
prove a site still exists or that a sender may submit to it. Use records inside
|
|
145
|
+
their tenant context and recheck lifecycle/permissions when later work requires
|
|
146
|
+
it. Both APIs restore the previous tenant context when the block returns or raises.
|
|
147
|
+
|
|
148
|
+
## Carry Worker metadata with authenticated provenance
|
|
149
|
+
|
|
150
|
+
The default Worker protocol remains v2: it authenticates the raw message and SMTP envelope. Optional v3 also authenticates a bounded metadata object:
|
|
151
|
+
|
|
152
|
+
```json
|
|
153
|
+
{
|
|
154
|
+
"source": "cloudflare",
|
|
155
|
+
"data": {
|
|
156
|
+
"archive_key": "inbound/example.eml"
|
|
157
|
+
}
|
|
158
|
+
}
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
The `archive_key` above is an illustrative application-defined value. The gem does not fetch that object or verify an archive exists. Your trusted Worker constructs the data from sources whose provenance you understand.
|
|
162
|
+
|
|
163
|
+
V3 binds the encoded metadata to the timestamp, envelope, and exact raw bytes. The `X-CF-Email-Metadata` header is unpadded base64url UTF-8 JSON. Metadata attached to a v2 request is rejected rather than treated as authenticated. Existing v2 requests without metadata continue to work.
|
|
164
|
+
|
|
165
|
+
The Worker template exports `signedEmailHeaders` for an existing transport and `forwardEmail` for the bundled forwarding behavior. Keep your current archive and retry handling, and use the header builder to authenticate the original raw bytes:
|
|
166
|
+
|
|
167
|
+
```js
|
|
168
|
+
import { signedEmailHeaders } from "./cloudflare-ingress.js";
|
|
169
|
+
|
|
170
|
+
const headers = await signedEmailHeaders({
|
|
171
|
+
secret: env.INGRESS_SECRET,
|
|
172
|
+
raw, // Uint8Array from your bounded reader
|
|
173
|
+
from: message.from,
|
|
174
|
+
to: message.to,
|
|
175
|
+
metadata: { source: "cloudflare", data: { archive_key: archiveKey } },
|
|
176
|
+
});
|
|
177
|
+
// Send exactly raw with these headers using your existing transport.
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
Here `cloudflare-ingress.js` is your local copy of the template's `src/index.js`. `raw` and `archiveKey` come from your existing bounded reader and archive operation. See the [Worker integration instructions](../templates/worker/README.md) for supported metadata values and forwarding behavior. Upgrade the Rails receiver before enabling v3 in the Worker.
|
|
181
|
+
|
|
182
|
+
In your Action Mailbox handler, read persisted metadata through the gem:
|
|
183
|
+
|
|
184
|
+
```ruby
|
|
185
|
+
require "cloudflare/email/provider_metadata"
|
|
186
|
+
|
|
187
|
+
metadata = Cloudflare::Email::ProviderMetadata.for(inbound_email)
|
|
188
|
+
archive_key = metadata&.dig("data", "archive_key")
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
This reads verified Active Storage metadata, not similarly named MIME headers. Authenticate your Worker first and preserve the verified object unchanged. Avoid adding secrets or unnecessary personal information to provider metadata.
|
|
192
|
+
|
|
193
|
+
**A valid Worker signature proves that your trusted Worker submitted the request. It does not prove the original email sender is authentic.** Even metadata labelled `source: "cloudflare"` is an assertion made by the holder of the ingress secret. Do not copy sender-controlled MIME headers into a “trusted authentication” field. If your existing Worker has reliable authentication evidence, preserve its documented provenance and let your application decide how that evidence affects acceptance. The gem intentionally does not infer SPF/DMARC results or implement a sender allowlist.
|
|
194
|
+
|
|
195
|
+
Retries with the same recipient, raw bytes, and metadata context reuse the inbound record. Aliases receive distinct membership context. Changing signed metadata produces a distinct checksum: stable values are preferable to per-attempt timestamps or random IDs. Keep downstream document extraction and external side effects idempotent too.
|
|
196
|
+
|
|
197
|
+
## Adopt incrementally
|
|
198
|
+
|
|
199
|
+
1. Inventory existing accepted addresses, organization/subdomain mappings, fallback destinations, and disabled sites. Backfill records explicitly; do not silently create a mailbox for every incoming address.
|
|
200
|
+
2. Confirm existing Worker routing for one organization's exact domain and activate only verified addresses. Exercise an alias, unknown address, suspended mailbox, and cross-tenant destination with synthetic mail.
|
|
201
|
+
3. Add request verification and preserve your current sender review, archive behavior, error responses, and size limits. Test that rejected or held mail cannot reach the processing queue.
|
|
202
|
+
4. Connect site provisioning to the same mailbox API used by the [management engine](management-engine.md). Scope the engine through existing host authentication and authorization.
|
|
203
|
+
5. Retire duplicate infrastructure only after comparing delivery and recovery behavior. Inbound adoption does not require switching your outbound provider.
|
|
204
|
+
|
|
205
|
+
Keep the shared operational communications history separate from tenant-local raw emails and mailbox membership when those serve different purposes. Define who owns deletion, archive retention, deduplication, and processing status; two competing delivery ledgers make recovery harder.
|
|
206
|
+
|
|
207
|
+
The gem's custom-ingress integration test exercises a custom controller against two real SQLite tenant databases, preserving binary MIME bytes and signed metadata before routing jobs run. It also verifies rejected metadata, host review, accepted-address enforcement, and retry behavior. It does not verify a deployed customer's Cloudflare DNS, Worker authentication evidence, R2 archive, or document-processing pipeline; those require that host's end-to-end tests.
|