@push.rocks/smartmta 6.5.2 → 8.0.0

package/readme.md

@@ -210,10 +210,10 @@ const emailServer = new UnifiedEmailServer(dcRouterRef, {
     users: [{ username: 'outbound', password: 'secret' }],
   },
 
-  // TLS certificates
+  // TLS material is provided in memory; SmartMTA does not read certificate files.
   tls: {
-    certPath: '/etc/ssl/mail.crt',
-    keyPath: '/etc/ssl/mail.key',
+    certPem: await certificateStore.getCertificatePem('mail.example.com'),
+    keyPem: await certificateStore.getPrivateKeyPem('mail.example.com'),
   },
 
   outbound: {
@@ -224,9 +224,11 @@ const emailServer = new UnifiedEmailServer(dcRouterRef, {
     connectionProxyProvider: async ({ host, port, ehloDomain }) => null,
   },
 
+  // Durable message bodies belong in SmartBucket (or another managed blob store).
+  // Use { storageMode: 'memory' } only for deliberately non-durable runtimes.
   queue: {
-    storageType: 'disk',
-    persistentPath: '/var/lib/smartmta/email-queue',
+    storageMode: 'managed',
+    storageManager: smartBucketBlobStorage,
   },
 
   maxMessageSize: 25 * 1024 * 1024, // 25 MB
@@ -239,7 +241,28 @@ await emailServer.start();
 
 > 🔒 **Note:** `start()` will throw if the Rust binary is not compiled. Run `pnpm build` first.
 >
+> SmartMTA 8 requires Node.js 22 or newer.
+>
 > `hostname` is the public SMTP identity for greetings and outbound delivery by default. It is not a bind address.
+>
+> `dcRouterRef.storageManager` implements `IStorageManager` for small textual state such as DKIM keys, routes, bounce metadata, and reputation cache; use SmartData for that adapter. Queue bodies and attachment blobs use `IBlobStorageManager`, normally backed by SmartBucket. SmartMTA has no local-filesystem persistence fallback.
+
+### Durable Delivery Checkpoints
+
+Managed queues store complete queue items as SmartBucket blobs. Schema V2 adds per-recipient delivery states and a durable bounce outbox:
+
+- A recipient becomes `delivered` only after the remote SMTP server's final `250` response.
+- A permanent `5xx RCPT TO` response atomically stores both the recipient's terminal state and its pending bounce operation in one queue-item blob mutation.
+- A temporary `4xx` response remains unresolved, so only that recipient is attempted against the next MX host or a later queue retry. Delivered and permanently rejected recipients are never resent.
+- Bounce operation IDs are deterministic hashes of the queue item, normalized recipient, and operation kind. Pending work is drained immediately at delivery-system startup and periodically afterward, including work on terminal queue items.
+- Observer and event-listener failures are logged and isolated after the durable state transition; they cannot reclassify delivery state or statistics.
+
+V1 queue blobs remain readable. They hydrate with an empty bounce outbox and are upgraded to V2 on their next mutation; SmartMTA never invents retroactive bounce work. V2 blobs are intentionally not readable by V7, so rolling back after a V2 write is unsupported.
+
+The managed-prefix ownership guard is process-local only. It coordinates queues only when they share the exact same `IBlobStorageManager` object instance and normalized storage prefix. Different storage-wrapper instances and different JavaScript processes are not coordinated; the embedder must enforce one delivery owner or provide an external distributed lease. This guard does not provide distributed ownership. Because queue writes are forward-only to schema V2, V7 rollback remains unsupported after the first V2 mutation.
+
+Bounce delivery is at-least-once. `BounceManager` uses the operation ID as a durable processed marker and deduplicates concurrent/restarted calls. Its in-process completed-operation LRU is only a hot-duplicate optimization, bounded to 10,000 entries with a 24-hour TTL; cache misses always re-read the durable marker. A process crash after an external side effect but before the queue completion checkpoint can cause the operation to be presented again. Custom `bounceHandler.processSmtpFailure` implementations receive `options.operationId` and must deduplicate it in their own durable store; SmartMTA does not claim exactly-once execution for custom handlers.
+
 
 ### 🛂 Inbound Recipient Policy And Relay Control
 
@@ -378,7 +401,9 @@ DKIM key management is handled by `DKIMCreator`, which generates, stores, and ro
 ```typescript
 import { DKIMCreator } from '@push.rocks/smartmta';
 
-const dkimCreator = new DKIMCreator('/path/to/keys', storageManager);
+// Small structured key state is supplied by the embedder through managed storage.
+// Use a SmartData-backed IStorageManager; SmartMTA never writes keys to local files.
+const dkimCreator = new DKIMCreator(storageManager);
 
 // Auto-generate keys if they don't exist
 await dkimCreator.handleDKIMKeysForDomain('example.com');
@@ -397,11 +422,12 @@ if (needsRotation) {
 ```
 
 When `UnifiedEmailServer.start()` is called:
-- DKIM keys are generated or loaded for every configured domain
-- Signing is applied to all outbound mail via the Rust security bridge
-- Key rotation is checked automatically based on your `rotationInterval` config
+- In the default `automatic` mode, DKIM keys are generated or loaded for every configured domain and rotation follows `rotationInterval`.
+- With `dkimKeyProvisioning: 'caller-managed'`, SmartMTA only loads the requested selector from managed storage; it never generates or rotates keys.
+- A requested missing or invalid key fails closed before SMTP delivery. SmartMTA never falls back to unsigned delivery.
+- Signing is applied exactly once, by the Rust SMTP client at final outbound transmission.
 
-Embedders can check `smartMtaCapabilities.selectorCorrectDkimSigning === true` as a compatibility probe. This guarantees selector-aligned DKIM key generation, lookup, and signing for both direct sends and queued delivery.
+Embedders can check `smartMtaCapabilities.selectorCorrectDkimSigning`, `callerManagedDkimKeys`, `structuredSmtpTransactions`, and `typedDeliveryFailures` as compatibility probes.
 
 ### 🛡️ Email Authentication (SPF, DKIM, DMARC)
 
@@ -545,13 +571,16 @@ Automatic bounce detection (via Rust), classification, and suppression tracking:
 import { Core } from '@push.rocks/smartmta';
 const { BounceManager } = Core;
 
-const bounceManager = new BounceManager();
+const bounceManager = new BounceManager({ storageManager: smartDataStorage });
 
 // Process an SMTP failure
 const bounce = await bounceManager.processSmtpFailure(
   'recipient@example.com',
   '550 5.1.1 User unknown',
-  { originalEmailId: 'msg-123' }
+  {
+    operationId: 'queue-item-and-recipient-derived-operation-id',
+    originalEmailId: 'msg-123',
+  }
 );
 // -> { bounceType: 'invalid_recipient', bounceCategory: 'hard', ... }
 
@@ -597,7 +626,7 @@ const email = await templates.createEmail('welcome', {
 
 ### 🌍 DNS Management
 
-When `UnifiedEmailServer.start()` is called, it automatically ensures MX, SPF, DKIM, and DMARC records are in place for all configured domains:
+DNS behavior is explicit per domain and DKIM ownership mode:
 
 ```typescript
 const emailServer = new UnifiedEmailServer(dcRouterRef, {
@@ -611,12 +640,15 @@ const emailServer = new UnifiedEmailServer(dcRouterRef, {
   // ... other config
 });
 
-// DNS records are set up automatically on start:
-//   - MX records pointing to your mail server
-//   - SPF TXT records authorizing your server IP
-//   - DKIM TXT records with public keys from DKIMCreator
-//   - DMARC TXT records with your policy
+// In automatic key-provisioning mode, internal DNS domains register the
+// configured MX/SPF/DMARC handlers and the generated selector-specific DKIM record.
+// External DNS domains are validated and report the required public changes.
 await emailServer.start();
+
+// With dkimKeyProvisioning: 'caller-managed', startup only validates the exact
+// configured selector in managed storage and reads public DNS for validation.
+// It performs no key generation/rotation, local DNS handler registration,
+// DNS record write/delete, or provisioning fallback.
 ```
 
 ### 🪵 Logging

package/readme.plan.md

@@ -2,6 +2,12 @@
 
 ## Completed Phases
 
+### Phase 6: Durable Per-Recipient Delivery And Bounce Outbox
+- Queue schema V2 stores explicit recipient dispositions and deterministic bounce operations in managed blob storage
+- Final 250 acceptance, permanent 5xx terminal state, and unresolved 4xx retry semantics are checkpointed per recipient
+- Built-in bounce processing is idempotent by operation ID; custom handlers receive the ID for at-least-once deduplication
+- Queue and delivery observers are isolated from committed state, statistics, and downstream listeners
+
 ### Phase 3: Rust Primary Backend (DKIM/SPF/DMARC/IP Reputation)
 - Rust is the mandatory security backend — no TS fallbacks
 - All DKIM signing/verification, SPF, DMARC, IP reputation through Rust bridge

package/ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@push.rocks/smartmta',
-  version: '6.5.2',
+  version: '8.0.0',
   description: 'A high-performance, enterprise-grade Mail Transfer Agent (MTA) built from scratch in TypeScript with Rust acceleration.'
 }

package/ts/functions.errors.ts

@@ -0,0 +1,8 @@
+/** Normalize values caught from promise and callback boundaries. */
+export function getErrorMessage(error: unknown): string {
+  return error instanceof Error ? error.message : String(error);
+}
+
+export function getErrorStack(error: unknown): string | undefined {
+  return error instanceof Error ? error.stack : undefined;
+}

package/ts/index.ts

@@ -5,4 +5,7 @@ export { baseLogger, logger } from './logger.js';
 
 export const smartMtaCapabilities = Object.freeze({
   selectorCorrectDkimSigning: true,
+  callerManagedDkimKeys: true,
+  structuredSmtpTransactions: true,
+  typedDeliveryFailures: true,
 } as const);