@serve.zone/coremail 1.2.0 → 32.0.0

package/readme.md

@@ -11,6 +11,12 @@ needs to provision and reconcile bindings without proxying message traffic.
 pnpm add @serve.zone/coremail
 ```
 
+CoreMail installs on Linux only. Its SMTP submission listener is built on
+`@push.rocks/smartmta`, which declares `os: linux` and ships the prebuilt Rust
+`mailer-bin` for `linux-x64` and `linux-arm64` — CoreMail's own runtime target.
+Installing on macOS or Windows fails at the dependency, by design; develop
+against a Linux container or host.
+
 The package exports the service lifecycle and its strict configuration reader:
 
 ```typescript
@@ -58,6 +64,10 @@ CoreMail enforces these boundaries:
 - Active and draining bindings derive their exact permitted operation set from
   the shared contract; disabled bindings cannot authenticate.
 - Every reconnect requires a new Argon2id-authenticated handshake.
+- No CoreMail session negotiates a contract major: interfaces 32.0.0 puts the
+  protocol offer on the registration carriers and gives the CoreMail contracts
+  none, so the control, workload and gateway sessions are first-contact
+  exchanges judged by their exact contract alone.
 - TypedSocket tags are routing metadata, never authentication.
 - Message bytes never travel as JSON, base64 RPC payloads, or VirtualStreams.
   One-time grants authorize bounded HTTP `PUT` or `GET` transfers.
@@ -75,11 +85,19 @@ CoreMail enforces these boundaries:
 One strict hostname surface is exposed:
 
 - `GET /live` reports process liveness.
-- `GET /ready` reports database, exact object-storage, desired-state, and
-  authenticated gateway readiness.
+- `GET /ready` reports database, exact object-storage, desired-state,
+  authenticated gateway, and SMTP submission listener readiness.
 - `GET|PUT /transfers/:grantId` consumes one-time transfer capabilities.
 - `GET /socket` upgrades to the authenticated TypedSocket RPC transport.
 
+When the desired state enables it, an SMTP submission listener (MSA) runs on
+its own port. It offers `STARTTLS` and accepts `AUTH` only on an encrypted
+session — the username is a `bindingId` and the password any accepted
+credential secret of that binding. Accepted messages go straight to the durable
+outbound pipeline as exact bytes; the embedded SMTP stack stores, queues and
+forwards nothing. A listener whose certificate or private key material does not
+resolve stays down and is reported unavailable.
+
 HTTP TypedRequest and built-in routes are disabled. WebSocket messages are
 limited to 64 KiB, unauthenticated connections have five seconds to complete
 their first authentication request, and HTTP connection/header/request
@@ -88,19 +106,27 @@ deadlines are enforced by SmartServe.
 ## Configuration
 
 CoreMail requires Coreflow and the deployment secret foundation to provide
-runtime configuration. For Interfaces v23 secret material, the workload must be
-launched through `workloadinit` protocol v1. The platform integration must write
-the verified map and secret files; `workloadinit` injects their exact values into
-the process environment immediately before executing CoreMail. CoreMail does not
+runtime configuration. Secret material declared by `@serve.zone/interfaces` is
+delivered by launching the workload through `workloadinit`. The platform
+integration must write the verified map and secret files; `workloadinit` injects
+their exact values into the process environment immediately before executing
+CoreMail. CoreMail does not
 fetch, decrypt, or persist runtime secret material itself. Until the dcrouter
 and Coreflow integrations are released, a platform CoreMail deployment is not
 ready. Malformed startup configuration terminates
 startup. Missing desired-state secret references keep the affected gateway or
 cursor operation unavailable.
 
+The SMTP submission listener is configured entirely through desired state
+(`smtp`): `enabled`, `port`, `hostname`, and `tls.certificatePemSecretKey` /
+`tls.privateKeyPemSecretKey`, which name runtime secret keys holding PEM text —
+never the PEM values themselves. An omitted or disabled `smtp` section keeps the
+API-only surface; an enabled one whose secret keys do not resolve leaves the
+listener down. Changing any of these values restarts the listener in place.
+
 ```text
-/opt/serve.zone/runtime-assets/workloadinit/v1/workloadinit run \
-  --map /run/serve.zone/workloadinit-map-v1.json -- node cli.js
+/opt/serve.zone/runtime-assets/workloadinit/workloadinit run \
+  --map /run/serve.zone/workloadinit-map.json -- node cli.js
 ```
 
 | Variable | Purpose |
@@ -140,6 +166,15 @@ header, publishes that MIME exactly once, and queues a lease-fenced worker.
 The downstream transport identity is the CoreMail submission ID and remains
 stable across ambiguous responses and retries.
 
+Every outbound submission records how it entered CoreMail. API submissions are
+composed from declared parts as described above. An SMTP submission is accepted
+as exact RFC822 bytes: it has no part descriptors, its bytes become its
+immutable MIME unchanged, and it enters the same lease-fenced worker in the same
+state a finalized API submission does. It is subject to the same binding
+authority, sender authorization, recipient validation, serialized-MIME byte
+budget, and minute and day quota. Records written before this discriminator
+existed are read as API submissions.
+
 Inbound SMTP recipient resolution returns short-lived opaque handles. The
 gateway uploads one immutable MIME object, after which CoreMail resumes a
 SmartData saga that consumes handles and creates one delivery per binding.
@@ -154,6 +189,25 @@ one-time grant, and may acknowledge only after the exact fetch completes.
 Cursor signatures rotate through current and bounded retiring desired-state
 keys without exposing runtime secret material.
 
+Every gateway-facing handler answers with the same privacy-safe typed error
+envelope the workload handlers use, so the transport can tell a retryable
+`QUOTA_EXCEEDED` or `STATE_CONFLICT` from a permanent refusal without ever
+receiving a cause string.
+
+Inbound deliveries carry the `Message-ID` and `Subject` read from a bounded
+prefix of the stored message. The gateway message descriptor cannot carry
+either, and a header scan that fails never costs a delivery — the fields simply
+stay absent.
+
+Per-binding counters are kept for each UTC day: `submittedApi` and
+`submittedSmtp` when a submission is published to the worker, `delivered`,
+`deferred`, `failed` and `deadLettered` on outbound transitions, and `received`,
+`acknowledgedProcessed` and `acknowledgedDiscarded` on inbound ones. Each
+increment commits in the same transaction as the durable transition it counts,
+so a replayed transition — a repeated finalization, a re-applied gateway status,
+a replayed handoff completion or acknowledgement — counts nothing. Counter rows
+are retained for 90 days.
+
 Desired-state activation uses immutable snapshots and a strictly increasing
 `configEpoch` compare-and-set pointer. Replayed or stale configurations cannot
 replace newer authority.
@@ -182,6 +236,10 @@ pnpm run build:docker
 pnpm run release:docker
 ```
 
+It is built from `Dockerfile_##version##`, so every build publishes the release
+version tag and nothing else. `coremail:latest` is never moved again — it keeps
+the last 1.x build it held — and consumers pin the version tag.
+
 ## License and Legal Information
 
 This repository contains open-source code licensed under the MIT License. A

package/ts/00_commitinfo_data.ts

@@ -2,7 +2,7 @@
  * autocreated commitinfo by @push.rocks/commitinfo
  */
 export const commitinfo = {
-  name: '@serve.zone/coremail',
-  version: '1.2.0',
-  description: 'Authenticated mail persistence and delivery orchestration for serve.zone workloads.'
+  name: "@serve.zone/coremail",
+  version: "32.0.0",
+  description: "Authenticated mail persistence and delivery orchestration for serve.zone workloads."
 }

package/ts/classes.auth.ts

@@ -321,6 +321,83 @@ export class CoreMailAuth {
     }
   }
 
+  /**
+   * Verify an SMTP submission credential.
+   *
+   * SMTP AUTH carries no credential id, so every currently accepted verifier
+   * of the binding is tried. Verification, the argon2id code and the
+   * per-identity limiter are the socket path's: the identity key is the same
+   * `workload:<bindingId>`, so a brute force cannot escape the limit by
+   * switching surface.
+   *
+   * Returns undefined for every refusal — an unknown binding, a binding that
+   * is not active or not outbound-capable, and a wrong secret are deliberately
+   * indistinguishable to the caller.
+   */
+  public async authenticateSmtpSubmission(
+    bindingIdArg: string,
+    secretArg: string,
+  ): Promise<{
+    binding: plugins.serveZoneInterfaces.data.ICoreMailBindingDesiredState;
+    configEpoch: number;
+    credentialId: string;
+    credentialVersion: number;
+  } | undefined> {
+    let failureCode = 'INVALID_INPUT';
+    try {
+      const desiredState = await this.getActiveDesiredState();
+      const bindingId = requireCoreMailIdentifier(bindingIdArg, 'bindingId');
+      const secret = this.requireSecretShape(secretArg);
+      failureCode = 'UNKNOWN_BINDING';
+      const binding = desiredState?.bindings.find((bindingArg) =>
+        bindingArg.bindingId === bindingId
+        && bindingArg.state === 'active'
+        && bindingArg.capabilities.includes('outbound')
+      );
+      if (!desiredState || !binding) {
+        throw new CoreMailAuthenticationError();
+      }
+      failureCode = 'UNKNOWN_CREDENTIAL';
+      const accepted = binding.credentials.filter((credentialArg) =>
+        findAcceptedVerifier(
+          binding.credentials,
+          credentialArg.credentialId,
+          credentialArg.version,
+          this.now(),
+        ) !== undefined
+      );
+      if (accepted.length === 0) {
+        throw new CoreMailAuthenticationError();
+      }
+      failureCode = 'VERIFIER_REJECTED';
+      for (const verifier of accepted) {
+        try {
+          await this.verify(verifier, secret, `workload:${bindingId}`);
+        } catch (error) {
+          if (error instanceof CoreMailAuthLimitError) throw error;
+          continue;
+        }
+        return {
+          binding,
+          configEpoch: desiredState.configEpoch,
+          credentialId: verifier.credentialId,
+          credentialVersion: verifier.version,
+        };
+      }
+      throw new CoreMailAuthenticationError();
+    } catch (error) {
+      await logCoreMailFailure(
+        'auth',
+        error instanceof CoreMailAuthLimitError
+          ? 'LIMITER_EXHAUSTED'
+          : failureCode,
+        undefined,
+        { surface: 'smtp' },
+      ).catch(() => undefined);
+      return undefined;
+    }
+  }
+
   public async authenticateWorkload(
     peerArg: ICoreMailPeer,
     requestArg:

package/ts/classes.coremail.ts

@@ -14,6 +14,7 @@ import {
   type ICoreMailModels,
 } from './classes.models.js';
 import { CoreMailServer } from './classes.server.js';
+import { CoreMailSmtpSubmissionServer } from './classes.smtpsubmission.js';
 import { CoreMailStorage } from './classes.storage.js';
 import { CoreMailSubmissionService } from './classes.submissions.js';
 import { CoreMailTransferService } from './classes.transfer.js';
@@ -39,6 +40,7 @@ export class CoreMail {
   public readonly inbound: CoreMailInboundService;
   public readonly maintenance: CoreMailMaintenanceService;
   public readonly gateway: CoreMailGateway;
+  public readonly smtpSubmission: CoreMailSmtpSubmissionServer;
   public readonly server: CoreMailServer;
   public readonly outboundWorker: CoreMailOutboundWorker;
 
@@ -48,6 +50,7 @@ export class CoreMail {
   private databaseStarted = false;
   private storageStarted = false;
   private gatewayStarted = false;
+  private smtpSubmissionStarted = false;
   private serverStarted = false;
   private workerStarted = false;
   private maintenanceStarted = false;
@@ -85,21 +88,30 @@ export class CoreMail {
       this.database,
       this.models,
       this.transfers,
+      this.storage,
       configArg.replica.serviceId,
       this.desiredState.getActiveDesiredState,
       configArg.resolveRuntimeSecret,
     );
     this.gateway = new CoreMailGateway(
       configArg,
+      this.database,
       this.models,
       this.transfers,
       this.inbound,
       this.desiredState.getActiveDesiredState,
     );
+    this.smtpSubmission = new CoreMailSmtpSubmissionServer(
+      configArg,
+      this.auth,
+      this.submissions,
+      this.desiredState.getActiveDesiredState,
+    );
     this.maintenance = new CoreMailMaintenanceService(this.models, this.storage);
     this.server = server = new CoreMailServer(
       configArg,
       this.database,
+      this.models,
       this.storage,
       this.transfers,
       this.desiredState,
@@ -107,6 +119,7 @@ export class CoreMail {
       this.submissions,
       this.inbound,
       this.gateway,
+      this.smtpSubmission,
     );
     this.outboundWorker = new CoreMailOutboundWorker(
       this.models,
@@ -163,6 +176,8 @@ export class CoreMail {
     await this.desiredState.start();
     this.gateway.start();
     this.gatewayStarted = true;
+    this.smtpSubmission.start();
+    this.smtpSubmissionStarted = true;
     await this.server.start();
     this.serverStarted = true;
     this.maintenance.start();
@@ -226,6 +241,11 @@ export class CoreMail {
       () => this.maintenance.stop(),
       () => { this.maintenanceStarted = false; },
     );
+    await stopOne(
+      this.smtpSubmissionStarted,
+      () => this.smtpSubmission.stop(),
+      () => { this.smtpSubmissionStarted = false; },
+    );
     await stopOne(
       this.gatewayStarted,
       () => this.gateway.stop(),

package/ts/classes.gateway.ts

@@ -6,6 +6,13 @@ import type {
   ICoreMailConfig,
   ITransferAuthority,
 } from './interfaces.js';
+import { addPrivacySafeTypedHandler } from './coremail.errors.js';
+import { withCoreMailTransaction } from './coremail.quota.js';
+import {
+  createCoreMailStatDescriptor,
+  ensureCoreMailStatCounter,
+  incrementCoreMailStatCounter,
+} from './coremail.stats.js';
 import { logCoreMailFailure } from './coremail.log.js';
 import type { ICoreMailSubmissionRecord } from './coremail.persistence.js';
 import {
@@ -42,6 +49,24 @@ export const stableStringify = (valueArg: unknown): string => {
   return JSON.stringify(valueArg) ?? 'null';
 };
 
+/**
+ * The body CoreMail sends to open its dcrouter gateway session.
+ *
+ * A first-contact exchange: `@serve.zone/interfaces` 32.0.0 gives the CoreMail
+ * carriers no `protocol` member, so CoreMail names no major to the gateway and
+ * negotiates nothing. The body lives here, beside the caller, so the session
+ * CoreMail opens can be judged by a specification without dialling a socket.
+ */
+export const createCoreMailGatewayAuthenticationRequest = (
+  gatewayArg: TGatewayDesired,
+  credentialSecretArg: string,
+): plugins.serveZoneInterfaces.requests.coremail
+  .IReq_CoreMailGatewayAuthenticate['request'] => ({
+  credentialId: gatewayArg.credentialId,
+  credentialVersion: gatewayArg.credentialVersion,
+  credentialSecret: credentialSecretArg,
+});
+
 const normalizeTransferOrigin = (valueArg: unknown): string => {
   if (typeof valueArg !== 'string') {
     throw new Error('CoreMail gateway returned an invalid transfer origin.');
@@ -111,6 +136,7 @@ export class CoreMailGateway {
 
   public constructor(
     private readonly config: ICoreMailConfig,
+    private readonly database: plugins.smartdata.SmartdataDb,
     private readonly models: ICoreMailModels,
     private readonly transfers: CoreMailTransferService,
     private readonly inbound: CoreMailInboundService,
@@ -182,31 +208,7 @@ export class CoreMailGateway {
   ): Promise<void> {
     await this.disconnect();
     if (this.stopController.signal.aborted) return;
-    const router = new plugins.typedrequest.TypedRouter();
-    router.addTypedHandler(new plugins.typedrequest.TypedHandler<
-      plugins.serveZoneInterfaces.requests.coremail.IReq_CoreMailGatewayResolveRecipients
-    >('coreMailGatewayResolveRecipients', async (requestArg) => {
-      this.requireAuthenticated(signatureArg);
-      return await this.inbound.resolveRecipients(requestArg);
-    }));
-    router.addTypedHandler(new plugins.typedrequest.TypedHandler<
-      plugins.serveZoneInterfaces.requests.coremail.IReq_CoreMailGatewayPrepareInboundHandoff
-    >('coreMailGatewayPrepareInboundHandoff', async (requestArg) => {
-      this.requireAuthenticated(signatureArg);
-      return await this.inbound.prepareInboundHandoff(requestArg);
-    }));
-    router.addTypedHandler(new plugins.typedrequest.TypedHandler<
-      plugins.serveZoneInterfaces.requests.coremail.IReq_CoreMailGatewayCompleteInboundHandoff
-    >('coreMailGatewayCompleteInboundHandoff', async (requestArg) => {
-      this.requireAuthenticated(signatureArg);
-      return await this.inbound.completeInboundHandoff(requestArg);
-    }));
-    router.addTypedHandler(new plugins.typedrequest.TypedHandler<
-      plugins.serveZoneInterfaces.requests.coremail.IReq_CoreMailGatewayCompleteOutboundHandoff
-    >('coreMailGatewayCompleteOutboundHandoff', async (requestArg) => {
-      this.requireAuthenticated(signatureArg);
-      return await this.completeOutboundHandoff(requestArg);
-    }));
+    const router = this.createGatewayRouter(signatureArg);
     const socket = await plugins.typedsocket.TypedSocket.createClient(
       router,
       gatewayArg.endpointUrl,
@@ -235,6 +237,45 @@ export class CoreMailGateway {
     this.queueAuthentication(gatewayArg, signatureArg);
   }
 
+  /**
+   * The gateway-facing router.
+   *
+   * Every handler goes through the same privacy-safe envelope the workload
+   * router uses, so dcrouter observes the typed `ICoreMailErrorData` code —
+   * `QUOTA_EXCEEDED` versus `STATE_CONFLICT` decides whether it defers a
+   * message or rejects it — instead of an opaque error string.
+   */
+  public createGatewayRouter(
+    signatureArg: string,
+  ): plugins.typedrequest.TypedRouter {
+    const router = new plugins.typedrequest.TypedRouter();
+    addPrivacySafeTypedHandler<
+      plugins.serveZoneInterfaces.requests.coremail.IReq_CoreMailGatewayResolveRecipients
+    >(router, 'gateway', 'coreMailGatewayResolveRecipients', async (requestArg) => {
+      this.requireAuthenticated(signatureArg);
+      return await this.inbound.resolveRecipients(requestArg);
+    });
+    addPrivacySafeTypedHandler<
+      plugins.serveZoneInterfaces.requests.coremail.IReq_CoreMailGatewayPrepareInboundHandoff
+    >(router, 'gateway', 'coreMailGatewayPrepareInboundHandoff', async (requestArg) => {
+      this.requireAuthenticated(signatureArg);
+      return await this.inbound.prepareInboundHandoff(requestArg);
+    });
+    addPrivacySafeTypedHandler<
+      plugins.serveZoneInterfaces.requests.coremail.IReq_CoreMailGatewayCompleteInboundHandoff
+    >(router, 'gateway', 'coreMailGatewayCompleteInboundHandoff', async (requestArg) => {
+      this.requireAuthenticated(signatureArg);
+      return await this.inbound.completeInboundHandoff(requestArg);
+    });
+    addPrivacySafeTypedHandler<
+      plugins.serveZoneInterfaces.requests.coremail.IReq_CoreMailGatewayCompleteOutboundHandoff
+    >(router, 'gateway', 'coreMailGatewayCompleteOutboundHandoff', async (requestArg) => {
+      this.requireAuthenticated(signatureArg);
+      return await this.completeOutboundHandoff(requestArg);
+    });
+    return router;
+  }
+
   private queueAuthentication(
     gatewayArg: TGatewayDesired,
     signatureArg: string,
@@ -259,11 +300,9 @@ export class CoreMailGateway {
         >('coreMailGatewayAuthenticate', undefined, {
           timeoutMs: 5_000,
           abortSignal: this.stopController.signal,
-        }).fire({
-          credentialId: gatewayArg.credentialId,
-          credentialVersion: gatewayArg.credentialVersion,
-          credentialSecret,
-        });
+        }).fire(
+          createCoreMailGatewayAuthenticationRequest(gatewayArg, credentialSecret),
+        );
         if (
           response.authenticated !== true
           || response.credentialId !== gatewayArg.credentialId
@@ -520,49 +559,81 @@ export class CoreMailGateway {
       ) {
         throw new Error('CoreMail gateway status chronology conflicts.');
       }
-      const result = await this.models.Submission.exact.transition({
-        current: submission,
-        change: (modelArg) => {
-          modelArg.transportMessageId = statusArg.transportMessageId;
-          modelArg.acceptedAt = acceptedAt;
-          modelArg.state = statusArg.state;
-          modelArg.attempts = Math.max(
-            submission.attempts,
-            statusArg.attempts,
-          );
-          modelArg.gatewayStatusUpdatedAt = statusArg.updatedAt;
-          if (nextAttemptAt === undefined) {
-            delete modelArg.nextAttemptAt;
-          } else {
-            modelArg.nextAttemptAt = nextAttemptAt;
-          }
-          if (workerRetryAt === undefined) {
-            delete modelArg.workerRetryAt;
-          } else {
-            modelArg.workerRetryAt = workerRetryAt;
-          }
-          if (statusArg.state === 'delivered') {
-            modelArg.deliveredAt = statusArg.deliveredAt;
-          } else {
-            delete modelArg.deliveredAt;
-          }
-          if (statusArg.terminalAt === undefined) {
-            delete modelArg.terminalAt;
-          } else {
-            modelArg.terminalAt = statusArg.terminalAt;
-          }
-          if (statusArg.error === undefined) {
-            delete modelArg.error;
-          } else {
-            modelArg.error = statusArg.error;
+      // A terminal state is reached at most once (the loop returns early once
+      // the record is terminal) and a deferral counts only when the gateway
+      // status is strictly newer than the one already recorded, so replaying
+      // one status never counts twice.
+      const statKind = terminal
+        ? statusArg.state as 'delivered' | 'failed' | 'deadLettered'
+        : statusArg.state === 'deferred'
+            && (
+              submission.gatewayStatusUpdatedAt === undefined
+              || statusArg.updatedAt > submission.gatewayStatusUpdatedAt
+            )
+          ? 'deferred' as const
+          : undefined;
+      const statDescriptor = createCoreMailStatDescriptor(submission, updatedAt);
+      if (statKind) {
+        await ensureCoreMailStatCounter(this.models, statDescriptor);
+      }
+      const result = await withCoreMailTransaction(
+        this.database,
+        async (transactionArg) => {
+          const transitioned = await this.models.Submission.exact.transition({
+            current: submission,
+            change: (modelArg) => {
+              modelArg.transportMessageId = statusArg.transportMessageId;
+              modelArg.acceptedAt = acceptedAt;
+              modelArg.state = statusArg.state;
+              modelArg.attempts = Math.max(
+                submission.attempts,
+                statusArg.attempts,
+              );
+              modelArg.gatewayStatusUpdatedAt = statusArg.updatedAt;
+              if (nextAttemptAt === undefined) {
+                delete modelArg.nextAttemptAt;
+              } else {
+                modelArg.nextAttemptAt = nextAttemptAt;
+              }
+              if (workerRetryAt === undefined) {
+                delete modelArg.workerRetryAt;
+              } else {
+                modelArg.workerRetryAt = workerRetryAt;
+              }
+              if (statusArg.state === 'delivered') {
+                modelArg.deliveredAt = statusArg.deliveredAt;
+              } else {
+                delete modelArg.deliveredAt;
+              }
+              if (statusArg.terminalAt === undefined) {
+                delete modelArg.terminalAt;
+              } else {
+                modelArg.terminalAt = statusArg.terminalAt;
+              }
+              if (statusArg.error === undefined) {
+                delete modelArg.error;
+              } else {
+                modelArg.error = statusArg.error;
+              }
+              modelArg.updatedAt = updatedAt;
+              modelArg.revision += 1;
+              delete modelArg.leaseOwnerTaskId;
+              delete modelArg.leaseToken;
+              delete modelArg.leaseExpiresAt;
+            },
+              }, { session: transactionArg });
+          if (transitioned.status === 'transitioned' && statKind) {
+            await incrementCoreMailStatCounter(
+              this.models,
+              statDescriptor,
+              statKind,
+              updatedAt,
+              transactionArg,
+            );
           }
-          modelArg.updatedAt = updatedAt;
-          modelArg.revision += 1;
-          delete modelArg.leaseOwnerTaskId;
-          delete modelArg.leaseToken;
-          delete modelArg.leaseExpiresAt;
+          return transitioned;
         },
-      });
+      );
       if (result.status === 'transitioned') {
         return result.document;
       }