cloudflare-email 0.2.0 → 0.4.0

data/templates/worker/docs/domain-setup.md

@@ -0,0 +1,198 @@
+# Set up once, create mailboxes in Rails
+
+Use the same Worker for either address shape:
+
+| Pattern | Example | What changes when a customer joins |
+| --- | --- | --- |
+| One receiving domain | `acme@in.example.com` | Create a mailbox/address in Rails |
+| Organization subdomains | `invoices@acme.in.example.com` | Register the exact organization domain and its mailboxes in Rails, within your verified receiving namespace |
+
+Neither pattern needs a recipient allowlist in the Worker. Rails owns accepted
+addresses and organization membership. Database multi-tenancy remains optional.
+The [Rails hello-world template](https://github.com/cole-robertson/cloudflare-email-rails-starter)
+demonstrates the first pattern with a normal SQLite database and generated login.
+
+## What is verified, and what Cloudflare documents
+
+**Rebulk's existing deployment uses the second pattern:**
+`<site>@<organization>.rebulk.com`, wildcard MX, a catch-all Worker rule, and Rails
+organization/mailbox lookup. New organizations do not need a Worker enrollment
+list or a routine per-organization Cloudflare approval. Its September 2026
+rehearsal checked exact-domain public MX answers and archived SMTP-envelope
+delivery evidence, alongside the global catch-all rule. The apex keeps its
+separate mail provider. Sources (Rebulk repository access required):
+
+- [Worker architecture and organization setup](https://github.com/Rebulk/rebulk-system/blob/1cc26e0076ba0e9157e89f5879604df8c94430bc/cloudflare/README.md).
+- [Wildcard MX observations and limits of zone-level API evidence](https://github.com/Rebulk/rebulk-system/blob/1cc26e0076ba0e9157e89f5879604df8c94430bc/docs/cloudflare-mailbox-onboarding-rehearsal-2026-09-12.md).
+
+This is deployment evidence, not a Cloudflare-wide service guarantee. Cloudflare's
+[subdomain onboarding guide](https://developers.cloudflare.com/email-service/configuration/subdomains/)
+describes adding subdomains explicitly and currently lists a 30-domain combined
+Routing/Sending limit. It does not document arbitrary wildcard Email Routing as
+a way to bypass that limit. Cloudflare separately documents
+[wildcard DNS and exact-record precedence](https://developers.cloudflare.com/dns/manage-dns-records/reference/wildcard-dns-records/).
+Working wildcard DNS alone does not establish Email Routing acceptance. If your
+account does not accept unlisted subdomains, use documented explicit onboarding
+within its limits, or the single-domain pattern. Do not assume that an existing
+Rebulk setup proves a fresh account's behavior.
+
+A fresh [public DNS observation](https://github.com/cole-robertson/cloudflare-email/blob/main/docs/verification/2026-09-13-wildcard-dns.json)
+also records Cloudflare MX answers for `test.rebulk.com` and two random subdomains
+without creating DNS records or adding those labels in Cloudflare. That report
+is public and deliberately marks `delivery_verified: false`; no email was sent
+by this DNS-only check.
+
+## 1. Fill in your setup worksheet
+
+Use a dedicated receiving namespace when the apex already serves Workspace or
+Microsoft 365. Example values:
+
+| Setting | Your example value |
+| --- | --- |
+| Cloudflare zone | `example.com` |
+| Receiving base | `in.example.com` |
+| Organization domain | `acme.in.example.com` |
+| Worker | `cloudflare-email-ingress` |
+| Rails ingress URL | `https://app.example.com/rails/action_mailbox/cloudflare/inbound_emails` |
+| Shared ingress secret | Generate with `openssl rand -hex 32`; save privately in Rails and the Worker |
+
+Keep staging and production Workers, buckets, queues, secrets, and receiving
+namespaces separate. The standalone deploy-button template takes one environment
+per copy; the regular CLI template uses `--env production` or another explicit env.
+
+## 2. Deploy the shared infrastructure
+
+[![Deploy to Cloudflare](https://deploy.workers.cloudflare.com/button)](https://deploy.workers.cloudflare.com/?url=https://github.com/cole-robertson/cloudflare-email/tree/main/templates/deploy-to-cloudflare)
+
+Deploy Rails with gem 0.3+ first. The button copies the standalone Worker and
+prompts for `RAILS_INGRESS_URL` and `INGRESS_SECRET`. Its configuration includes
+private R2 storage, a Queue producer/consumer, once-per-minute recovery, and logs.
+Cloudflare documents [automatic R2/Queue provisioning and secret prompts](https://developers.cloudflare.com/workers/platform/deploy-buttons/).
+The button does not create email DNS, claim a domain, or register Rails mailboxes.
+
+## 3. Connect DNS and the Worker once
+
+For **one domain**, follow Cloudflare's documented
+[Email Routing onboarding](https://developers.cloudflare.com/email-service/get-started/route-emails/)
+and configure its [catch-all action](https://developers.cloudflare.com/email-service/configuration/email-routing-addresses/#catch-all-rule)
+as **Send to a Worker**, selecting your deployed Worker. This avoids one provider
+rule per local part. Review the actual zone/domain scope and any explicit rules
+that take precedence before saving.
+
+For a **Rebulk-style dynamic namespace**, first establish receiving on your account
+and retain the exact MX targets/priorities assigned by Cloudflare. Then, when
+reproducing and verifying wildcard receiving, the DNS template is:
+
+| DNS type | Name in zone `example.com` | Value |
+| --- | --- | --- |
+| MX | `*.in` | Each Cloudflare-assigned receiving MX target, with its assigned priority |
+
+Create one record per assigned target. Do not invent MX targets, point MX at a
+Worker URL, or replace the apex's existing MX records. Complete the service's
+other required DNS records through its onboarding instructions; do not blindly
+duplicate SPF records. Inspect the account's catch-all **Send to a Worker** rule
+and verify that it actually handles your intended namespace. A Cloudflare Worker
+HTTP route such as `*.example.com/*` is unrelated to email routing.
+
+An explicit DNS name can stop wildcard inheritance **even if its record is TXT
+or A rather than MX**. For example, an existing `acme.in.example.com` record may
+require its own receiving MX records. Test exact customer names as well as fresh
+ones. A wildcard also does not supply MX for the receiving base itself.
+
+## 4. Check without modifying infrastructure
+
+From either current Worker template, run:
+
+```sh
+npm run check:subdomains -- --base in.example.com --labels acme,globex
+```
+
+This requires only Node and DNS access, not a Cloudflare API key. It resolves MX
+for your chosen labels plus two fresh random labels. DNS lookups have bounded
+timeouts. JSON output distinguishes observed Cloudflare MX, other/missing MX, and
+lookup uncertainty. Exit zero means only that all queried names resolved to
+Cloudflare MX; `delivery_verified` remains false. Compare returned targets with
+your account's assigned targets. Existing explicit records can explain differences.
+The command ships in the repository templates; gem 0.3.0's already-published
+template predates it, so refresh your template copy to use it.
+
+For provider rule inspection, the gem also offers:
+
+```sh
+bin/rails cloudflare:email:check_route \
+  ADDRESS=invoices@acme.in.example.com \
+  WORKER_NAME=cloudflare-email-ingress \
+  ACCOUNT_ID=your-cloudflare-account-id
+```
+
+See [diagnostic credentials and limits](https://github.com/cole-robertson/cloudflare-email/blob/main/docs/routing-diagnostics.md).
+That diagnostic inspects exact-domain configured records; it does not resolve
+wildcard DNS inheritance. A working inherited-MX setup can therefore report
+missing exact records. Keep public DNS, provider rule inspection, and actual
+delivery evidence separate; do not force every organization through Cloudflare
+onboarding merely to make that diagnostic green.
+
+## 5. Prove the dynamic path before adopting it
+
+Use two new organization labels that have not been added individually in the
+Cloudflare dashboard:
+
+1. Check their public MX answers with the command above.
+2. Through trusted provisioning code, register both **exact** domains in Rails,
+   each against the correct stable organization key. Create a test mailbox for
+   each. Record your independently checked configuration evidence and activate
+   those test addresses so the gem can receive the verification messages.
+3. Send real email to both addresses from an external mailbox. Confirm the
+   Worker receives each, Rails retains the original message, and each appears
+   only in the correct organization's inbox. Record the message IDs and result.
+4. Repeat with a newly generated label without changing Cloudflare. If delivery
+   fails before the Worker, resolve the Cloudflare acceptance/onboarding issue;
+   changing Rails cannot fix a provider SMTP rejection.
+5. Rehearse Rails being unavailable, then verify retained R2 mail drains after
+   recovery without duplicate app records. Follow the
+   [durable recovery guide](https://github.com/cole-robertson/cloudflare-email/blob/main/templates/worker/docs/durable-inbound.md).
+
+These are deliberate test addresses; no real-email test is sent by the DNS checker.
+Successful tests establish evidence for your receiving namespace, not a guarantee
+that every future DNS change or provider policy will preserve it.
+
+## 6. Create organizations in Rails
+
+Once your namespace policy is verified, use the same gem APIs on organization
+creation. For example, from trusted app code after authorizing the organization:
+
+```ruby
+registry = Cloudflare::Email::Mailboxes
+organization_key = "organization-123" # Trusted stable app identity, not request input.
+domain = registry.register_domain(
+  domain: "acme.in.example.com",
+  tenant_key: organization_key,
+  account_id: Cloudflare::Email::Credentials.account_id
+)
+registry.activate_domain!(domain.id, evidence: verified_namespace_evidence)
+registry.for_tenant(organization_key) do |inboxes|
+  mailbox = inboxes.create(name: "Invoices", address: "invoices@acme.in.example.com",
+    owner_ref: "Organization:123")
+  inboxes.activate_address!(inboxes.addresses(mailbox.id).first.id,
+    evidence: verified_namespace_evidence)
+end
+```
+
+`verified_namespace_evidence` is your stored operator evidence from the setup and
+delivery checks, not a hard-coded claim of success. Make provisioning idempotent
+in your app, validate/reserve slugs, and resolve organization identity from trusted
+records. The gem stores exact domain registrations; it does not accept a `*`
+domain registration or silently authorize an arbitrary recipient from a web form.
+The management engine must obtain allowed domains from your host adapter. The
+hello-world starter intentionally stays single-domain; this recipe is the next
+step for an organization-aware app.
+
+A **Cloudflare routing catch-all** gets mail to the Worker. The gem's optional
+**mailbox catch-all** decides whether unknown local parts enter a mailbox. They
+are separate. Without the latter, unknown/suspended recipients return non-2xx and
+remain in durable pending storage until resolved. Monitor that backlog.
+
+Receiving on dynamic domains does not authorize sending from them. Cloudflare
+[Email Sending onboarding](https://developers.cloudflare.com/email-service/configuration/subdomains/#add-a-subdomain-to-email-sending)
+is separate; keep a verified shared sending domain unless you have explicitly
+onboarded the customer sending domain.

data/templates/worker/docs/durable-inbound.md

@@ -0,0 +1,149 @@
+# Durable inbound delivery
+
+When Rails is down, keep the email at Cloudflare and deliver it when Rails returns.
+The bundled Worker uses R2 storage, Queue delivery and scheduled recovery by
+default. No enable flag, Rails model, database, or tenancy change is needed.
+Missing infrastructure fails visibly; it never silently downgrades to direct delivery.
+
+Upgrade the Rails gem before deploying this Worker. New messages without a parseable
+Message-ID use a deterministic fallback for deduplication across Rails hosts.
+Older records created before this change use ActionMailbox's hostname-dependent
+fallback; replaying those historical messages after upgrading can create a new
+record. Reconcile any existing archive backlog before replaying it automatically.
+
+## Set it up
+
+Create a private R2 bucket and a Queue for each environment:
+
+```sh
+npx wrangler r2 bucket create cloudflare-email-inbound-production
+npx wrangler queues create cloudflare-email-inbound-production
+```
+
+The bundled `wrangler.toml` already configures these resources for each environment.
+Customize their names if needed. Configure **all three** pieces: the `INBOUND_EMAIL_STORE` R2
+binding, `INBOUND_EMAIL_QUEUE` producer and consumer, and the once-per-minute
+scheduled trigger. Keep `RAILS_INGRESS_URL` and `INGRESS_SECRET` configured as
+before. Deploy with `npm run deploy -- --env production`.
+
+Use separate buckets, queues and secrets for staging and production. Do not add
+an R2 lifecycle rule that expires `cloudflare-email/pending/` objects. Pending
+mail has no automatic expiration and can incur storage costs during a long
+outage. Do not publish the bucket or expose an unauthenticated recovery endpoint.
+Review storage access and retention as you would the Rails email database.
+Provision and deploy the infrastructure with Wrangler first; retain that configuration
+in your infrastructure repository. Subsequent `cloudflare:email:deploy_worker` Rails
+task uploads preserve R2/Queue bindings and check the bucket binding, queue producer
+binding, and minute schedule before changing code or secrets. This check does not
+validate the bucket lifecycle or queue consumer: verify those in your infrastructure
+configuration and live drill. The Ruby task does not provision resources.
+
+## Direct fallback and upgrades
+
+Set `INBOUND_DELIVERY_MODE = "direct"` in the environment's Wrangler vars only when
+you deliberately want a single HTTP handoff. With the Ruby task, pass
+`INBOUND_DELIVERY_MODE=direct bin/rails cloudflare:email:deploy_worker`.
+Direct mode permanently rejects on HTTP/network failure and cannot buffer an outage.
+Keep the existing R2/Queue bindings and cron during fallback: queue and scheduled
+recovery continue draining previously retained mail. To return to durable delivery,
+remove the Wrangler override or pass `INBOUND_DELIVERY_MODE=durable` to the Ruby task.
+Unknown values fail rather than choosing a transport implicitly.
+
+The old `DURABLE_INBOUND_ENABLED` rollout flag has been removed. Existing direct
+installations must provision the resources before upgrading the Worker, or explicitly
+select `INBOUND_DELIVERY_MODE=direct` before deploying. Upgrading the Ruby gem alone
+does not redeploy a Worker or change live routing.
+
+## What happens to a message
+
+1. The Worker validates the envelope and actual MIME size. Invalid or oversized
+   mail is permanently rejected, as in direct mode.
+2. It writes one R2 object containing the exact raw bytes, SMTP envelope, original
+   receive time, and optional provider metadata. It waits for the write before
+   returning from the Email Worker handler.
+3. It queues a small `{ version: 1, key }` pointer. If enqueueing fails, the stored
+   object remains discoverable by the scheduled sweep.
+4. The consumer signs the original bytes and metadata with a fresh timestamp and
+   POSTs to Rails. Only HTTP 2xx permits deleting the pending object; the queue
+   message is acknowledged only after that deletion completes.
+5. Timeout, connection failure, redirect, 401, 413 and all other non-2xx responses
+   retain the object and retry. Fix credentials or size configuration for errors
+   that cannot resolve themselves. There is no automatic discard on client errors.
+
+If Rails saved a message but its response was lost, replay sends the same message
+identity. Rails ingress deduplicates using the original bytes and envelope (and
+provider metadata for v3); changing the signing timestamp does not change that
+identity. A failed R2 deletion may also cause replay. A queue pointer to an
+already deleted object is a successful no-op and never resends mail. Do not reuse
+pending keys, alter saved context, or replace the ingress with an endpoint that
+acknowledges before durable ingestion. Retain Rails deduplication records for at
+least your maximum recovery horizon; manual restore/replay after Rails incineration
+can produce another message. Application processing must tolerate duplicates too.
+
+The scheduled handler scans five pending objects per minute and attempts their
+handoff directly. It persists an R2 pagination cursor and advances past failed
+objects, wrapping back to the beginning at the end. Poison messages therefore do
+not permanently block later pages. This sweep handles both enqueue failure and
+exhausted Queue retries without a separate dead-letter queue. During recovery,
+Queue consumers provide the primary throughput; scheduled-only recovery takes
+at least `ceil(pending_count / 5)` minutes per full pass. Overlapping cron runs
+may repeat a page; delivery remains idempotent. Scale and exercise this recovery
+rate against your expected backlog before depending on it.
+
+The template buffers MIME to sign it, and the durable path also builds a framed
+storage payload. Test messages near your configured size limit with realistic
+concurrent traffic against the Worker's memory limit before production rollout.
+Lower `MAX_EMAIL_BYTES` in both Worker and Rails if your workload requires it;
+lowering it below already retained message sizes leaves those messages pending
+until the limit is restored or they are recovered through an authorized process.
+
+## Operate and verify
+
+Enable Worker logs and alerts for the structured
+`component: "cloudflare_email_inbound"` events. They include a random storage key
+and fixed reason, never raw messages, addresses, secrets or HTTP response bodies.
+Alert on `storage_failed`, `enqueue_failed_retained`, repeated `queue_retry_retained`
+or `sweep_retry_retained`, Worker exceptions, and growth/age of objects under the
+pending prefix. `receivedAt` is inside the private stored frame; R2 object upload
+time is also available to infrastructure monitoring. Queue backlog alone is not
+sufficient: exhausted pointers may disappear while retained mail remains in R2.
+This template emits logs; configure your monitoring service to page the operator.
+
+In staging, stop Rails, send mail, confirm pending R2 objects, restore Rails, then
+verify one inbound message and removal of its pending object. Also verify recovery
+with the queue consumer disabled or retries exhausted, a lost HTTP response after
+Rails saved the message, invalid ingress credentials, and a failed R2 write.
+The repository includes unit coverage and a local Worker/Rails integration harness.
+Before changing ingress URLs, remember retained messages use the current configured
+URL and secret. Drain existing retained mail before repointing a bucket to another
+application or disabling the scheduled recovery trigger.
+
+For a custom Worker, import `retainEmail(message, env, { metadata })`,
+`consumeRetainedEmails(batch, env)` and `sweepRetainedEmails(env)` from `src/index.js`.
+Wire the latter two to your queue and scheduled handlers as in the default export.
+For a secondary raw archive, pass `archive: async ({ raw, from, to, key }) => ...`
+to `retainEmail`. It runs after the primary R2 commit with the original MIME bytes;
+`key` is the pending object identity. Its errors or default 10-second timeout are
+logged as `archive_failed_retained` and do not prevent queue delivery. A timed-out
+callback is not canceled. Set `archiveTimeoutMs` to an integer from 1 to 120,000
+milliseconds to fit your host's runtime budget. Treat bytes as read-only and never parse the private
+pending frame to build an archive. The primary R2 write is always mandatory.
+Trusted provider metadata is captured once at receipt, never recomputed on replay.
+The existing `relayEmail` archive callback is best effort and does not enable this
+durable protocol automatically.
+
+## Guarantee boundary
+
+After a successful R2 write, Rails outages and finite Queue retention do not
+discard the stored payload. This is at-least-once handoff, with Rails deduplication;
+it is not an absolute guarantee against all loss. R2 write failures, upstream
+SMTP acceptance behavior, account suspension, deleted infrastructure, misconfigured
+lifecycle rules and manual intervention remain outside that promise. A failed
+initial durable write throws and emits an error; it must not be mistaken for
+successful storage. Monitor that failure separately.
+
+Cloudflare documents [`setReject`](https://developers.cloudflare.com/email-service/api/route-emails/email-handler/)
+as a permanent SMTP rejection. Throwing an Email Worker error is not treated here
+as a documented durable SMTP retry contract. Cloudflare Queues has a
+[128 KB message limit and finite retention/retries](https://developers.cloudflare.com/queues/platform/limits/),
+which is why the queue stores only pointers and R2 retains the message independently.

data/templates/worker/package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "cloudflare-email-ingress",
-  "version": "0.2.0",
+  "version": "0.3.0",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "cloudflare-email-ingress",
-      "version": "0.2.0",
+      "version": "0.3.0",
       "devDependencies": {
         "vitest": "^5.0.0",
         "wrangler": "^4.131.0"

data/templates/worker/package.json

@@ -1,6 +1,6 @@
 {
   "name": "cloudflare-email-ingress",
-  "version": "0.2.0",
+  "version": "0.3.0",
   "private": true,
   "type": "module",
   "engines": { "node": "^22.12.0 || ^24.0.0 || >=26.0.0" },
@@ -9,6 +9,7 @@
     "deploy": "node scripts/wrangler.mjs deploy",
     "dev": "node scripts/wrangler.mjs dev",
     "check": "wrangler deploy --env development --dry-run",
+    "check:subdomains": "node scripts/check-subdomains.mjs",
     "test": "vitest run"
   },
   "devDependencies": {

data/templates/worker/scripts/check-subdomains.mjs

@@ -0,0 +1,54 @@
+// Read-only public DNS check. No Cloudflare credentials or configuration writes.
+import { Resolver } from "node:dns/promises";
+import { randomBytes } from "node:crypto";
+import { parseArgs } from "node:util";
+import { fileURLToPath } from "node:url";
+
+const labelPattern = /^[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$/;
+
+export async function checkSubdomains({ base, labels = [] }, lookup, randomLabel = () => `cf-probe-${randomBytes(8).toString("hex")}`) {
+  base = String(base || "").trim().toLowerCase();
+  if (base.length > 220 || base.split(".").length < 2 || !base.split(".").every(label => labelPattern.test(label))) {
+    throw new Error("Provide a DNS base such as in.example.com, without @, a scheme, or *.");
+  }
+  if (labels.length > 10 || !labels.every(label => labelPattern.test(label) && `${label}.${base}`.length <= 253)) {
+    throw new Error("Provide at most 10 comma-separated single-label subdomains, such as acme,globex.");
+  }
+  const probes = [randomLabel(), randomLabel()];
+  const names = [...new Set([...labels, ...probes])];
+  const checks = await Promise.all(names.map(async label => {
+    const domain = `${label}.${base}`;
+    try {
+      const mx = (await lookup(domain)).map(record => ({ priority: record.priority, exchange: record.exchange.toLowerCase().replace(/\.$/, "") }));
+      const cloudflare = mx.length > 0 && mx.every(record => record.exchange.endsWith(".mx.cloudflare.net"));
+      return { domain, fresh_probe: probes.includes(label), status: cloudflare ? "cloudflare_mx_observed" : "other_or_missing_mx", mx };
+    } catch (error) {
+      return { domain, fresh_probe: probes.includes(label), status: "unknown", reason: error.code || "DNS_LOOKUP_FAILED" };
+    }
+  }));
+  return {
+    checked_at: new Date().toISOString(),
+    base, dns_observation: checks.every(check => check.status === "cloudflare_mx_observed") ? "cloudflare_mx_observed_for_all_names" : "needs_attention",
+    delivery_verified: false, checks,
+    next_step: "Inspect the catch-all Worker rule, register two exact domains/mailboxes in Rails, and send real test emails. MX answers do not verify account ownership, Email Routing acceptance, Worker delivery, or Rails persistence."
+  };
+}
+
+if (process.argv[1] === fileURLToPath(import.meta.url)) {
+  try {
+    const { values } = parseArgs({ options: { base: { type: "string" }, labels: { type: "string" }, help: { type: "boolean" } } });
+    if (values.help) {
+      console.log("npm run check:subdomains -- --base in.example.com [--labels acme,globex]\nRead-only DNS observations, including two fresh labels. This does not verify email delivery.");
+    } else {
+      const resolver = new Resolver({ timeout: 2000, tries: 1 });
+      const labels = values.labels ? values.labels.split(",").map(label => label.trim().toLowerCase()) : [];
+      const report = await checkSubdomains({ base: values.base, labels }, domain => resolver.resolveMx(domain));
+      console.log(JSON.stringify(report, null, 2));
+      if (report.dns_observation === "needs_attention") process.exitCode = 1;
+    }
+  } catch (error) {
+    console.error(error.message);
+    console.error("Usage: npm run check:subdomains -- --base in.example.com [--labels acme,globex]");
+    process.exitCode = 1;
+  }
+}