cloudflare-email 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (93) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +73 -1
  3. data/README.md +265 -388
  4. data/SECURITY.md +51 -0
  5. data/app/controllers/cloudflare/email/ingress_controller.rb +62 -6
  6. data/docs/activerecord-tenanted.md +135 -0
  7. data/docs/architecture.md +95 -0
  8. data/docs/delivery-events.md +177 -0
  9. data/docs/features.md +107 -0
  10. data/docs/getting-started.md +298 -0
  11. data/docs/mailboxes.md +299 -0
  12. data/docs/outbox.md +175 -0
  13. data/docs/tenant-mailboxes-plan.md +149 -0
  14. data/docs/thread-correlation.md +27 -0
  15. data/docs/troubleshooting.md +103 -0
  16. data/docs/upgrading-0.2.md +96 -0
  17. data/docs/verification/2026-09-10-followup.md +49 -0
  18. data/docs/verification/2026-09-10-install-upgrade.md +86 -0
  19. data/docs/verification/2026-09-10-live.md +102 -0
  20. data/docs/verification/2026-09-10.md +175 -0
  21. data/docs/verification/2026-09-11-extraction.md +32 -0
  22. data/docs/verification/2026-09-11-outbound-ledger.md +60 -0
  23. data/docs/verification/2026-09-11-security.md +76 -0
  24. data/docs/verification/2026-09-11-tenant-mailboxes.md +75 -0
  25. data/examples/plain_ruby.rb +24 -0
  26. data/lib/cloudflare/email/active_record/base.rb +48 -0
  27. data/lib/cloudflare/email/active_record/delivery_events.rb +72 -0
  28. data/lib/cloudflare/email/active_record/event_inbox.rb +77 -0
  29. data/lib/cloudflare/email/active_record/event_receipt.rb +17 -0
  30. data/lib/cloudflare/email/active_record/mail_snapshot.rb +24 -0
  31. data/lib/cloudflare/email/active_record/outbound_delivery.rb +22 -0
  32. data/lib/cloudflare/email/active_record/outbound_recipient.rb +13 -0
  33. data/lib/cloudflare/email/active_record/outbound_reconciliation.rb +15 -0
  34. data/lib/cloudflare/email/active_record/outbox.rb +219 -0
  35. data/lib/cloudflare/email/active_record/outbox_notifications.rb +34 -0
  36. data/lib/cloudflare/email/active_record.rb +5 -0
  37. data/lib/cloudflare/email/client.rb +51 -11
  38. data/lib/cloudflare/email/consume_events_task.rb +22 -0
  39. data/lib/cloudflare/email/delivery_event.rb +70 -0
  40. data/lib/cloudflare/email/delivery_method.rb +3 -0
  41. data/lib/cloudflare/email/deploy_worker_task.rb +3 -0
  42. data/lib/cloudflare/email/dev_ingress_guard.rb +27 -0
  43. data/lib/cloudflare/email/dev_tunnel.rb +31 -5
  44. data/lib/cloudflare/email/doctor.rb +16 -38
  45. data/lib/cloudflare/email/endpoint.rb +24 -0
  46. data/lib/cloudflare/email/engine.rb +28 -0
  47. data/lib/cloudflare/email/envelope.rb +58 -0
  48. data/lib/cloudflare/email/event_consumer.rb +86 -0
  49. data/lib/cloudflare/email/mailboxes/configuration.rb +58 -0
  50. data/lib/cloudflare/email/mailboxes/events.rb +112 -0
  51. data/lib/cloudflare/email/mailboxes/inbound_retention.rb +14 -0
  52. data/lib/cloudflare/email/mailboxes/jobs.rb +34 -0
  53. data/lib/cloudflare/email/mailboxes/models.rb +148 -0
  54. data/lib/cloudflare/email/mailboxes/provider_correlation.rb +11 -0
  55. data/lib/cloudflare/email/mailboxes/service.rb +288 -0
  56. data/lib/cloudflare/email/mailboxes/shared_event_receipt.rb +14 -0
  57. data/lib/cloudflare/email/mailboxes.rb +11 -0
  58. data/lib/cloudflare/email/message_id.rb +12 -0
  59. data/lib/cloudflare/email/provision_catchall_task.rb +4 -4
  60. data/lib/cloudflare/email/provision_route_task.rb +2 -1
  61. data/lib/cloudflare/email/replay_events_job.rb +18 -0
  62. data/lib/cloudflare/email/response.rb +21 -3
  63. data/lib/cloudflare/email/routing_provisioner.rb +71 -45
  64. data/lib/cloudflare/email/send_job.rb +41 -0
  65. data/lib/cloudflare/email/send_test.rb +5 -29
  66. data/lib/cloudflare/email/signing.rb +2 -2
  67. data/lib/cloudflare/email/tenancy.rb +78 -0
  68. data/lib/cloudflare/email/tenant_job_context.rb +90 -0
  69. data/lib/cloudflare/email/verification.rb +30 -8
  70. data/lib/cloudflare/email/version.rb +1 -1
  71. data/lib/cloudflare/email/worker_deployer.rb +5 -3
  72. data/lib/cloudflare-email.rb +4 -1
  73. data/lib/generators/cloudflare/email/install_generator.rb +48 -33
  74. data/lib/generators/cloudflare/email/mailboxes/mailboxes_generator.rb +55 -0
  75. data/lib/generators/cloudflare/email/mailboxes/templates/create_cloudflare_email_mailboxes.rb +54 -0
  76. data/lib/generators/cloudflare/email/mailboxes/templates/create_cloudflare_email_receiving_domains.rb +16 -0
  77. data/lib/generators/cloudflare/email/mailboxes/templates/create_cloudflare_email_shared_events.rb +35 -0
  78. data/lib/generators/cloudflare/email/outbox/outbox_generator.rb +28 -0
  79. data/lib/generators/cloudflare/email/outbox/templates/create_cloudflare_email_outbox.rb +48 -0
  80. data/lib/generators/cloudflare/email/templates/initializer.rb +3 -3
  81. data/lib/generators/cloudflare/email/templates/main_mailbox.rb +2 -7
  82. data/lib/generators/cloudflare/email/tracking/templates/create_cloudflare_email_event_receipts.rb +22 -0
  83. data/lib/generators/cloudflare/email/tracking/tracking_generator.rb +26 -0
  84. data/lib/tasks/cloudflare_email.rake +41 -4
  85. data/templates/worker/README.md +61 -11
  86. data/templates/worker/package-lock.json +2627 -0
  87. data/templates/worker/package.json +8 -5
  88. data/templates/worker/scripts/wrangler.mjs +16 -0
  89. data/templates/worker/src/index.js +83 -4
  90. data/templates/worker/test/index.test.ts +117 -5
  91. data/templates/worker/wrangler.toml +13 -4
  92. metadata +107 -6
  93. data/lib/cloudflare/email/secure_message_id.rb +0 -89
data/SECURITY.md ADDED
@@ -0,0 +1,51 @@
1
+ # Security
2
+
3
+ The hardening described here is included in 0.2.0. Version 0.1.0 does not include
4
+ these fixes. Upgrade Rails and the bundled Worker together using the
5
+ [upgrade guide](docs/upgrading-0.2.md).
6
+
7
+ Rails integration is tested on patched Rails 7.2, 8.0, and 8.1; application
8
+ owners must update their own Rails, database adapter, and other dependencies.
9
+ The gem does not add Rails or SQLite as runtime dependencies.
10
+
11
+ Report sensitive findings privately to the maintainer at cole@dscribeai.com.
12
+ Include affected versions and impact; omit real credentials and private mail.
13
+ Do not place secrets or private message data in public issues.
14
+
15
+ ## Deployment responsibilities
16
+
17
+ - Keep API and management tokens least-privileged and separate. Store tokens and
18
+ a strong ingress secret in a secret manager or Rails encrypted credentials.
19
+ Rotate compromised secrets on Rails and the Worker together. HTTPS endpoint
20
+ overrides are trusted operator configuration; never derive them from mail.
21
+ - Set request-size limits, timeouts and rate limits at the proxy/server. Rails
22
+ and the Worker bound MIME reads to 25 MiB by default (`MAX_EMAIL_BYTES`), but
23
+ cannot prevent upstream buffering or slow clients. Limits apply to raw MIME,
24
+ not just decoded attachments.
25
+ - Treat mail and attachments as untrusted content. The gem authenticates Worker
26
+ transport and SMTP envelope metadata, not a human sender's identity. Rendering,
27
+ sanitization, malware scanning and application authorization belong to the app.
28
+ - Protect raw MIME, attachments, outbox snapshots and event receipts with access
29
+ control, storage encryption, backups and retention policies. Avoid logging raw
30
+ responses/errors: provider details and application logs can contain private mail.
31
+ - Restrict queue producers and consumers. Event account/domain checks are
32
+ validation, not cryptographic signatures. Monitor unacknowledged malformed
33
+ messages, processing failures and unmatched receipts.
34
+ - Database administrators and direct SQL writes are trusted. ActiveRecord
35
+ read-only evidence fields do not provide tamper-proof storage. Keep tenant
36
+ authorization around outbox operations and receipt access in the application.
37
+ - The optional mailbox session scopes tenant and mailbox access, but does not
38
+ authenticate users. Keep domain registration/activation administrative, verify
39
+ ownership externally, and use your access resolver to choose tenant keys.
40
+ Ingress selects a registered tenant only after full signature verification.
41
+ Raw ActiveRecord access remains privileged, especially in shared-database mode.
42
+ Private tenant storage, consistent migrations and trusted job payloads are
43
+ required. Drain old framework jobs before enabling database tenancy; jobs
44
+ without the new tenant metadata are rejected rather than guessing a tenant.
45
+ - Development tunnels expose the ingress to the internet. Use the bundled task,
46
+ dedicated development mail routes and test data; stop tunnels when finished.
47
+ The Host guard restricts routing but is not a sandbox for the development app
48
+ or a replacement for safe error pages and request logging.
49
+
50
+ See [the September 11 security review](docs/verification/2026-09-11-security.md)
51
+ for findings, verification, and remaining limits.
@@ -5,26 +5,46 @@ module Cloudflare
5
5
  # ActionMailbox ingress for Cloudflare Email Worker forwards.
6
6
  #
7
7
  # The shipped Worker template signs each forwarded message with HMAC-SHA256
8
- # over "{timestamp}.{raw_body}" and sends:
8
+ # over "v2.{timestamp}.{encoded_envelope}.{raw_body}" and sends:
9
9
  # X-CF-Email-Timestamp: <unix seconds>
10
10
  # X-CF-Email-Signature: <hex digest>
11
+ # X-CF-Email-Signature-Version: 2
12
+ # X-CF-Email-Envelope: <unpadded base64url JSON from/to>
13
+ # Requests must include the v2 signature and authenticated SMTP envelope.
11
14
  #
12
15
  # Set the shared secret in Rails credentials under cloudflare.ingress_secret
13
16
  # (or in the CLOUDFLARE_INGRESS_SECRET env var) and as the Worker secret
14
17
  # INGRESS_SECRET via `wrangler secret put INGRESS_SECRET`.
15
18
  class IngressController < ActionMailbox::BaseController
19
+ DEFAULT_MAX_EMAIL_BYTES = 25 * 1024 * 1024
16
20
  param_encoding :create, "raw_email", Encoding::ASCII_8BIT
17
21
 
18
22
  def create
19
23
  ActiveSupport::Notifications.instrument(
20
24
  "cloudflare_email.ingress",
21
- bytes: raw_body.bytesize,
25
+ bytes: 0,
22
26
  ) do |payload|
27
+ preflight = Cloudflare::Email::Verification.verify_headers(secret: secret,
28
+ timestamp: request.headers["X-CF-Email-Timestamp"],
29
+ signature: request.headers["X-CF-Email-Signature"],
30
+ version: request.headers["X-CF-Email-Signature-Version"],
31
+ envelope: request.headers["X-CF-Email-Envelope"])
32
+ unless preflight == :ok
33
+ payload[:result] = preflight
34
+ next head(preflight == :stale ? :request_timeout : :unauthorized)
35
+ end
36
+ if request.content_length.to_i > max_email_bytes || raw_body.bytesize > max_email_bytes
37
+ payload[:result] = :too_large
38
+ next head(:payload_too_large)
39
+ end
40
+ payload[:bytes] = raw_body.bytesize
23
41
  case Cloudflare::Email::Verification.verify(
24
42
  secret: secret,
25
43
  body: raw_body,
26
44
  timestamp: request.headers["X-CF-Email-Timestamp"],
27
45
  signature: request.headers["X-CF-Email-Signature"],
46
+ version: request.headers["X-CF-Email-Signature-Version"],
47
+ envelope: request.headers["X-CF-Email-Envelope"],
28
48
  )
29
49
  when :stale
30
50
  payload[:result] = :stale
@@ -33,9 +53,19 @@ module Cloudflare
33
53
  payload[:result] = :bad_signature
34
54
  head :unauthorized
35
55
  when :ok
36
- inbound = ActionMailbox::InboundEmail.create_and_extract_message_id!(raw_body)
37
- payload[:result] = :ok
38
- payload[:message_id] = inbound.message_id
56
+ inbound = if defined?(Cloudflare::Email::Mailboxes) && Cloudflare::Email::Mailboxes.enabled?
57
+ recipient = Cloudflare::Email::Envelope.decode(request.headers["X-CF-Email-Envelope"]).fetch("to")
58
+ begin
59
+ Cloudflare::Email::Mailboxes.receive(recipient: recipient) { persist_inbound }
60
+ rescue Cloudflare::Email::Mailboxes::Unavailable
61
+ payload[:result] = :unavailable_mailbox
62
+ next head(:unprocessable_entity)
63
+ end
64
+ else
65
+ persist_inbound
66
+ end
67
+ payload[:result] = inbound ? :ok : :duplicate
68
+ payload[:message_id] = inbound&.message_id
39
69
  head :ok
40
70
  end
41
71
  end
@@ -43,6 +73,24 @@ module Cloudflare
43
73
 
44
74
  private
45
75
 
76
+ def persist_inbound
77
+ envelope = Cloudflare::Email::Envelope.decode(request.headers["X-CF-Email-Envelope"])
78
+
79
+ # Commit trusted routing metadata before ActionMailbox's after_create_commit
80
+ # enqueues routing. Identical MIME for To/Cc/Bcc recipients is independent.
81
+ checksum = OpenSSL::Digest::SHA256.hexdigest("v2\0#{envelope.fetch('to')}\0".b + raw_body.b)
82
+ ActionMailbox::InboundEmail.transaction do
83
+ inbound = ActionMailbox::InboundEmail.create_and_extract_message_id!(raw_body, message_checksum: checksum)
84
+ if inbound
85
+ blob = inbound.raw_email.blob
86
+ blob.update!(metadata: blob.metadata.merge(
87
+ Cloudflare::Email::Envelope::METADATA_KEY => envelope.merge("version" => 2),
88
+ ))
89
+ end
90
+ inbound
91
+ end
92
+ end
93
+
46
94
  # Override ActionMailbox::BaseController's default name inference so
47
95
  # `config.action_mailbox.ingress = :cloudflare` gates this controller.
48
96
  def ingress_name
@@ -52,7 +100,15 @@ module Cloudflare
52
100
  def raw_body
53
101
  @raw_body ||= begin
54
102
  request.body.rewind if request.body.respond_to?(:rewind)
55
- request.body.read
103
+ request.body.read(max_email_bytes + 1).to_s
104
+ end
105
+ end
106
+
107
+ def max_email_bytes
108
+ @max_email_bytes ||= begin
109
+ value = Integer(ENV.fetch("MAX_EMAIL_BYTES", DEFAULT_MAX_EMAIL_BYTES.to_s), 10)
110
+ raise ArgumentError, "MAX_EMAIL_BYTES must be positive" unless value.positive?
111
+ value
56
112
  end
57
113
  end
58
114
 
@@ -0,0 +1,135 @@
1
+ # SQLite mailboxes with activerecord-tenanted
2
+
3
+ Use this setup when each organization has its own SQLite database. The shared database holds receiving domains and delivery-event routing. Mailboxes, messages, and the outbox live in the organization's database.
4
+
5
+ This setup is entirely opt-in. Skip this guide for a single database: installing
6
+ the gem or running the mailbox generator does not enable database tenancy.
7
+
8
+ `activerecord-tenanted` is an optional application dependency. The gem's integration test uses version 0.8 on Rails 8.1; you do not need it for a single database or another tenant adapter.
9
+
10
+ ## 1. Configure your databases
11
+
12
+ Add `gem "activerecord-tenanted", "~> 0.8.0"` to your application's Gemfile and run `bundle install`.
13
+
14
+ For example, in `config/database.yml`:
15
+
16
+ ```yaml
17
+ production:
18
+ primary:
19
+ adapter: sqlite3
20
+ database: storage/directory.sqlite3
21
+ migrations_paths: db/migrate
22
+ tenant:
23
+ adapter: sqlite3
24
+ tenanted: true
25
+ database: storage/tenants/%{tenant}/db.sqlite3
26
+ migrations_paths: db/tenant_migrate
27
+ ```
28
+
29
+ Use equivalent entries for development and test. Keep tenant keys stable, such as an organization's internal identifier. Customer-supplied hostnames are not tenant authorization.
30
+
31
+ Generate the mailbox migrations in the correct directories:
32
+
33
+ ```sh
34
+ bin/rails generate cloudflare:email:mailboxes \
35
+ --directory-migrations-path=db/migrate \
36
+ --tenant-migrations-path=db/tenant_migrate
37
+ ```
38
+
39
+ Put Action Mailbox and Active Storage migrations in the tenant migration directory too. Apply the shared migrations and the tenant migrations using your application's database provisioning workflow. Keep every existing tenant's schema current before starting workers against a new release.
40
+
41
+ ## 2. Load stable connection classes before the mailbox models
42
+
43
+ The optional gem models inherit from your tenant base. That base must keep the same Ruby class identity across Rails development reloads. Define it in a file you explicitly require, outside reloadable model directories. If your app already has these bases, use those same classes; do not create competing versions.
44
+
45
+ For example, `lib/email_database_records.rb`:
46
+
47
+ ```ruby
48
+ class TenantRecord < ActiveRecord::Base
49
+ self.abstract_class = true
50
+ tenanted "tenant"
51
+ end
52
+
53
+ class DirectoryRecord < ActiveRecord::Base
54
+ self.abstract_class = true
55
+ connects_to database: { writing: :primary }
56
+ end
57
+ ```
58
+
59
+ Keep this file outside any paths your application configures for reloadable autoloading. Require it during application initialization after `activerecord-tenanted` has installed its Active Record support, and before the generated mailbox initializer loads the optional models. Add this inside your application class in `config/application.rb`:
60
+
61
+ ```ruby
62
+ config.active_record_tenanted.connection_class = "TenantRecord"
63
+ config.active_record_tenanted.tenanted_rails_records = true
64
+
65
+ config.active_record_tenanted.tenant_resolver = ->(request) do
66
+ if request.path == "/rails/action_mailbox/cloudflare/inbound_emails"
67
+ nil
68
+ else
69
+ request.subdomain # Replace with your existing authorized tenant resolver.
70
+ end
71
+ end
72
+
73
+ initializer "my_app.cloudflare_email_tenancy",
74
+ after: "active_record_tenanted.active_record_base",
75
+ before: :load_config_initializers do
76
+ require Rails.root.join("lib/email_database_records").to_s
77
+ require "cloudflare/email/tenancy"
78
+ require "cloudflare/email/mailboxes/configuration"
79
+
80
+ Cloudflare::Email::Tenancy.configure(
81
+ base_class: TenantRecord,
82
+ current: -> { TenantRecord.current_tenant },
83
+ switch: ->(key, &block) {
84
+ unless TenantRecord.tenant_exist?(key)
85
+ raise Cloudflare::Email::Mailboxes::Unavailable,
86
+ "tenant is not provisioned"
87
+ end
88
+ TenantRecord.with_tenant(key, &block)
89
+ }
90
+ )
91
+ Cloudflare::Email::Mailboxes.configure(directory_base: DirectoryRecord)
92
+ end
93
+ ```
94
+
95
+ Leave the generated `cloudflare_email_mailboxes.rb` initializer to require `cloudflare/email/mailboxes`. Remove any duplicate tenancy configuration from other initializers.
96
+
97
+ The ingress exception in the resolver matters: `activerecord-tenanted` normally locks a request to the tenant selected from its hostname. Email ingress must select its tenant from the verified recipient and shared domain directory, after the gem verifies the complete message signature. Returning `nil` lets that signed-recipient lookup select the correct database.
98
+
99
+ `tenanted_rails_records = true` puts Action Mailbox and Active Storage on the configured tenant connection. Include their tables in tenant migrations and configure private storage. The gem restores its tenant context for its supported email and attachment jobs. For your own jobs that serialize tenant model arguments, prepend `Cloudflare::Email::TenantJobContext` and enqueue them inside `Cloudflare::Email::Tenancy.with(key)`; ordinary host tenant context alone does not establish the gem's context.
100
+
101
+ ## 3. Provision organizations separately from mailboxes
102
+
103
+ Create tenant databases in your trusted organization-provisioning workflow:
104
+
105
+ ```ruby
106
+ TenantRecord.create_tenant("organization-123")
107
+ ```
108
+
109
+ The switch adapter above refuses missing tenants. Receiving mail or replaying a delivery event will never create a new tenant database implicitly.
110
+
111
+ After registering and activating the organization's receiving domain, use the mailbox API within its context:
112
+
113
+ ```ruby
114
+ Cloudflare::Email::Mailboxes.for_tenant("organization-123") do |inboxes|
115
+ mailbox = inboxes.create(
116
+ name: "Support",
117
+ address: "support@customer.example.com",
118
+ owner_ref: "team:42"
119
+ )
120
+ # Provision and activate the address before receiving or sending mail.
121
+ end
122
+ ```
123
+
124
+ See the [mailbox guide](mailboxes.md) for domain activation, address provisioning, sending, and event replay. Your application still authorizes which organization, mailbox, and message each user can access.
125
+
126
+ ## What is verified
127
+
128
+ The optional CI job boots a real Rails application with `activerecord-tenanted`, runs the gem's tenant migrations in two SQLite databases, and checks identical numeric IDs remain isolated. It also verifies context restoration, refusal to create unknown tenant databases, and GlobalID's missing/wrong-tenant rejection. Separate integration tests exercise mailbox ingress, sending, jobs, and delivery-event replay.
129
+
130
+ Run the library-specific smoke test locally with:
131
+
132
+ ```sh
133
+ BUNDLE_GEMFILE=gemfiles/tenanted.gemfile bundle install
134
+ BUNDLE_GEMFILE=gemfiles/tenanted.gemfile bundle exec ruby test/support/activerecord_tenanted.rb
135
+ ```
@@ -0,0 +1,95 @@
1
+ # Gem and inbox responsibilities
2
+
3
+ The Cloudflare gem supplies email transport and reusable delivery infrastructure.
4
+ The inbox application supplies a product built on that infrastructure. Plain Ruby
5
+ users should not need Rails or a database. Rails users should be able to opt into
6
+ durable infrastructure without copying the reference inbox's models and services.
7
+
8
+ ## Implemented in this extraction
9
+
10
+ | Gem capability | Application responsibility |
11
+ | --- | --- |
12
+ | Optional mailbox/address directory and tenant connection adapter | Authorize domain ownership, provision tenant databases and grant user access |
13
+ | Mailbox memberships, read/archive state and retained raw mail | Inbox UI, retention schedule and explicit purge policy |
14
+ | Shared event intake, tenant correlation and tenant-aware jobs | Durable queue workers, scheduled recovery and schema rollout to every tenant |
15
+ | Structured/raw sending and ActionMailer transport | Compose UI, recipients and send authorization |
16
+ | `Response#accepted?`, plus individual recipient outcomes | Handle partial acceptance without resending accepted recipients |
17
+ | Authenticated v2 ingress and recipient-scoped deduplication | Map trusted recipient to an authorized mailbox |
18
+ | `MessageId.normalize` and returned provider IDs | Store conversation membership and scope reply lookups |
19
+ | Queue decoding, validation and ACK | Queue/subscription setup and recurring execution |
20
+ | Optional ActiveRecord event receipts, deduplication and indexed replay | Configure queue polling and receipt retention |
21
+ | Immutable outbox snapshots, send claims, per-recipient outcomes | Authorize sending and choose a stable operation key |
22
+ | `DeliveryEvents` account/message/recipient correlation and ordering | Transactional callback to update product records |
23
+ | Send/replay jobs, recovery tasks and audited reconciliation | Authorize operators and provide evidence of provider outcome |
24
+
25
+ The ActiveRecord adapter is explicit opt-in. Its generator installs a receipt table
26
+ and initializer; the default plain Ruby client does not load ActiveRecord. Its
27
+ handler applies application writes on the same database connection as the receipt.
28
+ It does not wrap HTTP calls or other external effects in a database transaction.
29
+
30
+ Version 0.2 is a preproduction break: v1 ingress and signed Message-ID tokens are
31
+ removed. Cloudflare's documented queue encodings and recipient-response forms are
32
+ provider contracts, so accepting them is not obsolete application compatibility.
33
+ Historical verification reports describe their dated runs, not the current API.
34
+
35
+ ## Implemented: durable outbound delivery
36
+
37
+ The inbox now uses the gem's ledger tables and orchestration. Its model subclasses
38
+ add application message associations and display snapshots; they do not maintain
39
+ a second sending ledger. See the [outbox guide](outbox.md) for installation and API.
40
+
41
+ 1. Claim an application-supplied operation key scoped to an account using a unique
42
+ index before sending; repeated or concurrent calls cannot silently resend.
43
+ 2. Persist immutable rendered MIME and envelope recipients before the network
44
+ request. Track each recipient separately, including partial acceptance.
45
+ 3. Separate preparation failures from ambiguous outcomes. A timeout, process death
46
+ or persistence failure after acceptance keeps the operation blocked. An elapsed
47
+ timeout alone never proves the message was not sent.
48
+ 4. Persist validated provider IDs and recipient acceptance outcomes separately
49
+ from later lifecycle events. Replay correlates account, message and recipient;
50
+ conflicting IDs remain uncertain. The network and database are not atomic.
51
+ 5. Expose audited reconciliation with caller-supplied actor/reason and evidence.
52
+ The application authorizes the operator; the gem enforces legal transitions.
53
+ 6. Provide jobs/tasks and notifications that preserve the same claim on retries.
54
+ Include migration/import support for the reference inbox's existing attempts.
55
+
56
+ The suite exercises concurrent senders, actual process termination, acceptance
57
+ followed by persistence failure, partial recipients, event-before-response,
58
+ out-of-order events, callbacks that roll back, and recovery without a duplicate
59
+ send. PostgreSQL and SQLite run separate concurrency checks. Inbox HTTP and browser
60
+ workflows use the shared implementation, with controlled local provider responses.
61
+
62
+ Historic attempts did not preserve rendered MIME. Their import retains available
63
+ bodies, IDs, outcomes and operator evidence and marks reconstructed snapshots.
64
+ Unknown historical outcomes remain blocked. No migration claims to recreate the
65
+ exact bytes of a previously sent message.
66
+
67
+ ## Operational completion
68
+
69
+ The reference inbox supplies a functional mailbox product over these APIs. Before
70
+ production use, configure sending DNS and receiving routes, queue subscriptions
71
+ and dead-letter handling, durable Rails jobs, receipt/MIME retention and backups,
72
+ and monitoring for prepared/sending/unknown/partial operations. Exercise live
73
+ external mailbox delivery and recovery with the deployed revision. Installation
74
+ does not automatically provision domains or deploy infrastructure.
75
+
76
+ Current limitations are deliberate and visible: no provider exactly-once API,
77
+ no automatic resend of uncertain or rejected operations, no multi-provider-ID
78
+ operation (conflicting IDs require review), and no durable inbound Worker buffer.
79
+ A prepared operation can be dispatched using its existing identity after a job
80
+ enqueue failure; sending/unknown operations require evidence-based reconciliation.
81
+ Notifications indicate method execution and may run inside an outer transaction;
82
+ the durable database record is authoritative.
83
+
84
+ ## Keep in the inbox product
85
+
86
+ Users, tenancy, mailbox ownership, conversations, folders, search and UI remain in
87
+ the inbox. AI context, review flags, approval thresholds, auto-send policy and
88
+ operator permissions also stay there. Suppression/unsubscribe requirements depend
89
+ on message purpose and application policy; expose provider outcomes, but do not
90
+ silently impose a marketing subscription model on transactional mail.
91
+
92
+ The optional [mailbox module](mailboxes.md) now packages mailbox persistence and
93
+ address management over the existing transport/receipt APIs. It includes a
94
+ [tenant adapter](activerecord-tenanted.md), without adding a tenant library to
95
+ the plain Ruby client. A future inbox UI or starter should consume these APIs.
@@ -0,0 +1,177 @@
1
+ # Delivery events
2
+
3
+ Cloudflare Email Sending can publish outbound lifecycle events to Queues. This is separate from inbound Email Routing and the ActionMailbox ingress.
4
+
5
+ ## Provision once
6
+
7
+ 1. Create a dedicated Cloudflare Queue for your application's sending events.
8
+ 2. In that queue's **Subscriptions** tab, subscribe to Email Sending events for your verified sending domain. Subscriptions are domain-specific; add another for each sending domain you need.
9
+ 3. Select delivered, deferred, bounced, failed, rejected, and complained events.
10
+ 4. Enable **HTTP pull** as the queue consumer. Do not also attach a Worker push consumer.
11
+ 5. Configure queue retries and a dead-letter queue so failed or unsupported events can be investigated instead of disappearing after retry exhaustion.
12
+ 6. Create an account-scoped **Queues Read + Write** token (dashboard Queues Edit). Writing is necessary to acknowledge messages.
13
+
14
+ These are explicit Cloudflare setup steps, not actions performed by installing this gem. See [subscription management](https://developers.cloudflare.com/queues/event-subscriptions/manage-event-subscriptions/) and [HTTP pull consumers](https://developers.cloudflare.com/queues/configuration/pull-consumers/).
15
+
16
+ ## Rails setup
17
+
18
+ Add credentials:
19
+
20
+ ```yaml
21
+ cloudflare:
22
+ account_id: your-account-id
23
+ queues_token: your-queues-read-write-token
24
+ event_queue_id: your-queue-id
25
+ ```
26
+
27
+ Environment alternatives are `CLOUDFLARE_ACCOUNT_ID`, `CLOUDFLARE_QUEUES_TOKEN`, and `CLOUDFLARE_EVENT_QUEUE_ID`. Queue consumption uses its own token; it does not use the management-token fallback.
28
+
29
+ Configure a callable application handler:
30
+
31
+ ```ruby
32
+ # config/initializers/cloudflare_delivery_events.rb
33
+ Rails.application.configure do
34
+ config.x.cloudflare_email.event_domains = ["mail.example.com"]
35
+ config.x.cloudflare_email.event_handler = ->(event) {
36
+ DeliveryEventProcessor.call(event)
37
+ }
38
+ end
39
+ ```
40
+
41
+ `DeliveryEventProcessor` supplies application correlation and policy. For Rails,
42
+ use the optional durable receipt adapter below to avoid implementing receipt
43
+ storage, deduplication and replay yourself. Custom handlers must persist durably
44
+ before returning; do not merely enqueue a non-durable job.
45
+
46
+ ### Durable Rails receipts
47
+
48
+ If you also use the [durable outbox](outbox.md), prefer `DeliveryEvents.record`
49
+ and `DeliveryEvents.replay`: they provide account/message/recipient correlation
50
+ and state ordering automatically. The lower-level example below supports an
51
+ application's own delivery schema.
52
+
53
+ ```sh
54
+ bin/rails generate cloudflare:email:tracking
55
+ bin/rails db:migrate
56
+ ```
57
+
58
+ The generator installs a receipt table and an initializer that explicitly loads
59
+ `cloudflare/email/active_record/event_inbox`. ActiveRecord is not loaded by the
60
+ plain Ruby client. Configure the queue handler to commit receipts:
61
+
62
+ ```ruby
63
+ Rails.application.config.x.cloudflare_email.event_handler = ->(event) {
64
+ Cloudflare::Email::ActiveRecord::EventInbox.record(event)
65
+ }
66
+ ```
67
+
68
+ Run replay from your recurring job and after saving a send's provider ID:
69
+
70
+ ```ruby
71
+ Cloudflare::Email::ActiveRecord::EventInbox.replay(account_id: account_id) do |event|
72
+ # Your own model; always scope by account, message AND recipient.
73
+ delivery = RecipientDelivery.find_by(
74
+ account_id: event.account_id,
75
+ provider_message_id: Cloudflare::Email::MessageId.normalize(event.message_id),
76
+ recipient: event.recipient,
77
+ )
78
+ next :unmatched unless delivery
79
+
80
+ delivery.with_lock do
81
+ if event.supersedes?(occurred_at: delivery.occurred_at, terminal: delivery.terminal?)
82
+ delivery.update!(status: event.status, occurred_at: event.occurred_at,
83
+ terminal: event.terminal?)
84
+ end
85
+ end
86
+ :applied
87
+ end
88
+ ```
89
+
90
+ `record` stores the raw event and enforces uniqueness by account/event ID. An ID
91
+ repeated with different content raises. Recording inside an existing database
92
+ transaction raises, because returning before that transaction commits would allow
93
+ premature acknowledgment. An acknowledged receipt can still be pending or unmatched;
94
+ monitor and schedule replay separately from queue polling.
95
+
96
+ `apply(receipt) { |event| ... }` and `replay` lock each receipt and skip applied
97
+ records. Handlers must return `:applied` or `:unmatched`; exceptions and other return
98
+ values roll back the receipt transition and writes on the same database connection.
99
+ `:unmatched` is a successful transaction outcome, so avoid business writes before
100
+ returning it. External effects and writes to other databases cannot be rolled back.
101
+ Concurrent database conflicts raise; retry the job. Do not send email inside this
102
+ handler; use durable outbound orchestration.
103
+
104
+ Replay accepts optional `account_id:`, normalized provider `message_id:`, and
105
+ `batch_size:` filters. The batch size controls database fetch size, not the total
106
+ number processed. `DeliveryEvent#supersedes?` accepts a Time or ISO8601 previous
107
+ timestamp: only a newer known status replaces current state, equal timestamps keep
108
+ existing state, and a nonterminal event cannot replace terminal state. A later
109
+ terminal complaint can replace delivery. Invalid timestamps raise. Match records
110
+ and lock recipient state before using this ordering helper.
111
+
112
+ Receipt retention, recurring jobs, and monitoring are application decisions. See
113
+ the [gem/inbox boundary](architecture.md) for outbound-ledger scope. Existing
114
+ receipt tables from the earlier preproduction adapter must normalize stored
115
+ `message_id` values with `MessageId.normalize` before using filtered replay;
116
+ raw payloads remain unchanged. The reference inbox's outbox migration does this.
117
+
118
+ Poll a single batch:
119
+
120
+ ```sh
121
+ bin/rails cloudflare:email:consume_events
122
+ BATCH_SIZE=20 bin/rails cloudflare:email:consume_events
123
+ ```
124
+
125
+ Run the task from your existing scheduler, or invoke `EventConsumer#poll` from a recurring job. It short-polls once and exits; an empty queue is a successful result. No handler means an error before any queue request, not automatic discard.
126
+
127
+ For more control, use the API directly:
128
+
129
+ ```ruby
130
+ consumer = Cloudflare::Email::EventConsumer.new(
131
+ account_id: Cloudflare::Email::Credentials.account_id,
132
+ api_token: Cloudflare::Email::Credentials.fetch(:queues_token),
133
+ queue_id: Cloudflare::Email::Credentials.fetch(:event_queue_id),
134
+ domains: ["mail.example.com"],
135
+ )
136
+
137
+ count = consumer.poll(batch_size: 5, visibility_timeout_ms: 300_000) do |event|
138
+ DeliveryEventProcessor.call(event)
139
+ end
140
+ ```
141
+
142
+ The same API works in plain Ruby with explicit credentials.
143
+
144
+ ## Event data
145
+
146
+ | Method | Meaning |
147
+ |---|---|
148
+ | `event_id` | Durable application deduplication key |
149
+ | `message_id` | Cloudflare message ID for correlation with the send response |
150
+ | `status` | delivered, deferred, bounced, failed, rejected, complained |
151
+ | `known?` | Whether status is one of those six current types |
152
+ | `terminal?` | Cloudflare's terminal flag; not a guarantee no later complaint arrives |
153
+ | `sender`, `recipient` | Addresses from the event |
154
+ | `account_id`, `domain` | Event source, checked against consumer configuration |
155
+ | `occurred_at` | Original event timestamp string |
156
+ | `delivery`, `bounce`, `rejection`, `failure`, `complaint` | Detail hashes |
157
+ | `raw`, `payload`, `type` | Original provider data |
158
+
159
+ Record outcomes per message **and recipient**. A delivery event means the recipient's server accepted the message, not that the person read it. Use bounce/complaint events to maintain your application's recipient eligibility and unsubscribe state. Cloudflare's own suppression behavior remains controlled by Cloudflare; the consumer does not automatically edit provider suppression lists or your database.
160
+
161
+ Events can be duplicated or arrive out of order. Do not blindly overwrite a newer terminal state with an older deferred event. Decide how your application handles a complaint after delivery. Provider IDs may be absent from older send responses; retain event data even when an application record cannot yet be found.
162
+
163
+ ## Processing guarantees
164
+
165
+ For each message, the consumer decodes the body, validates the event, invokes your handler, and then acknowledges its lease. Email Sending subscriptions can return plain JSON strings with `CF-Content-Type: json`; the consumer also accepts base64-encoded JSON for that content type. `bytes` bodies contain base64-encoded JSON, while `text` bodies contain JSON directly. Invalid bodies and events remain unacknowledged.
166
+
167
+ - A handler that returns normally counts as successful, even if its return value is `false` or `nil`. **Raise** to prevent acknowledgement.
168
+ - A parse error, mismatched account/domain, handler exception, or acknowledgement failure stops the batch. Previously acknowledged messages remain acknowledged; the current and remaining leases become available after the visibility timeout.
169
+ - If the handler commits and acknowledgement fails, the event can be processed again. This is why the handler needs durable idempotency.
170
+ - Automatic HTTP retries are disabled for queue calls. Your scheduler can retry the next poll.
171
+ - The default batch size is 5 (max 100); the default visibility window is five minutes. Choose a window longer than the **whole batch's** worst-case processing time, up to Cloudflare's 12-hour maximum.
172
+ - Unknown future status names in schema version 1 are passed through with `known? == false`. Decide whether to store or raise. Unsupported schema versions are rejected and left unacknowledged.
173
+ - Configure dead-letter retention/alerts. Repeated failure eventually exhausts the queue's retry policy.
174
+
175
+ The `cloudflare_email.delivery_event` notification wraps handler execution with event ID, message ID, and status. It does not mean acknowledgement succeeded. Monitor task/job failures as well.
176
+
177
+ Source schemas: [Email Sending event subscriptions](https://developers.cloudflare.com/email-service/platform/event-subscriptions/), [pull API](https://developers.cloudflare.com/api/resources/queues/subresources/messages/methods/pull/), [ack API](https://developers.cloudflare.com/api/resources/queues/subresources/messages/methods/ack/).