kempo-smtp 0.0.0-stage → 0.1.0

package/README.md

@@ -1,3 +1,58 @@
-# Temporary Holding Version
+# kempo-smtp
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Sends the emails kempo logs through any SMTP server, using [nodemailer](https://nodemailer.com): the mailbox that comes with your domain (Hostinger), Google Workspace, Amazon SES, Mailgun, Postmark, Brevo, or a mail server of your own. Kempo does not send email itself: it saves every email in its log (**Admin > Emails**) and hands it to an email extension through the `email:send` hook. This is that extension for SMTP.
+
+## Setup
+
+1. Install it from Admin > Extensions.
+2. On **Admin > Settings**, open the **SMTP** tab and fill in the server (see below), then **Save settings**. The password is stored encrypted and never shown again.
+3. On the **System** tab, set **Email from** to the mailbox address, such as `Your Shop <orders@yourdomain.com>`, if you want a display name. SMTP servers only let you send as the mailbox you sign in with (or its aliases). While Email from is not set (still `noreply@example.com`), kempo-smtp sends as the mailbox you sign in with, so a plain setup works without it.
+4. On **Admin > Emails**, press **Send a test email** to check it.
+
+`SETTINGS_ENCRYPTION_KEY` must be set on the server (kempo makes one for you the first time it starts), because the password is stored encrypted.
+
+### Hostinger (the email that comes with a domain)
+
+Create the mailbox in Hostinger's panel (Emails), then use its **SMTP** settings from the mailbox's configuration page. They are normally:
+
+| Setting | Value |
+|---|---|
+| Host | `smtp.hostinger.com` |
+| Port | `465` with **Secure from the start** on (or `587` with it off) |
+| User | the full mailbox address, `orders@yourdomain.com` |
+| Password | the mailbox password |
+
+The domain's mail (MX) records must point at Hostinger's mail servers, and the SPF and DKIM records Hostinger shows for the mailbox must be in the domain's DNS: if the domain's DNS is at Hostinger this is automatic, otherwise add them where the DNS is. Your website's own A record (to a VPS) does not affect mail. Check the sending limits of your plan: a free mailbox allows only so many emails an hour.
+
+### Other providers
+
+| Provider | Host | Port | User |
+|---|---|---|---|
+| Google Workspace | `smtp.gmail.com` | 465 | the mailbox, with an app password |
+| Amazon SES | `email-smtp.<region>.amazonaws.com` | 465 | the SES SMTP user |
+| Mailgun | `smtp.mailgun.org` | 465 | `postmaster@yourdomain` |
+| A Postfix on the same server | `localhost` | 25 | none (you manage deliverability yourself) |
+
+## What it does
+
+For each email kempo asks to send, the handler connects to the server, signs in, sends the message with the HTML and a plain-text version made from it, and answers kempo with `{ sent: true, provider: 'smtp', messageId }` or `{ sent: false, error }`. Kempo records the result in the log, tries a failed email again (the **Email max retries** setting, 1 by default), and keeps the log for the retention settings.
+
+Failures are reported in plain words: the server was not found, nothing is listening at host:port, the username or password was refused, the server refused the address (with its own reason), or the connection could not be made (with a hint to check whether it is meant to be secure). Connecting times out after 20 seconds, so a wrong host cannot hold up a request.
+
+**Secure from the start** is for port 465, where the connection is encrypted before anything is said. With it off (port 587) the connection starts plain and upgrades to an encrypted one when the server offers it (nodemailer's default). It does not insist on that, so a server that offers no encryption at all, such as a Postfix on the same machine, still works; for anything across the internet use 465 or a server that offers STARTTLS.
+
+With no host saved the handler answers that it did not send, so if another email extension is installed (kempo-resend), it gets the email instead. The first one to send it ends the chain.
+
+## Settings
+
+| Setting | Default | Meaning |
+|---|---|---|
+| `smtp_host` | — | The address of the SMTP server |
+| `smtp_port` | `465` | The port |
+| `smtp_secure` | `true` | Secure from the start (465); off for 587 |
+| `smtp_user` | — | The username, usually the mailbox address. Empty for a server that needs none |
+| `smtp_password` | — | Secret. The password |
+
+## Tests
+
+`npm test` starts a real SMTP server of its own on a free port and sends to it: a delivered message (headers, HTML and text parts, sender and recipient), a wrong password, a refused address, a server that is not there, a server that needs no sign-in, and the plain-text conversion. Nothing leaves the machine.

package/admin/settings-tab.global.html

@@ -0,0 +1 @@
+<content id="kempo-smtp-settings-tab" name="SMTP settings tab" owner="kempo-smtp" enabled="true" locked="true" location="admin-settings-tabs" priority="20"><k-settings-tab name="smtp" label="SMTP" position="410"><p class="tc-muted mb">Sends the site's email through an SMTP server, such as the mailbox that comes with your domain. The address emails are sent from is the Email from setting on the System tab, and it must be that mailbox's address (or one of its aliases). After saving, send a test email from <a href="/admin/emails">Emails</a>.</p><k-settings-panel owner="kempo-smtp"></k-settings-panel></k-settings-tab></content>

package/hooks/send.js

@@ -0,0 +1,18 @@
+import { getSetting } from 'kempo/server/sdk.js';
+import { sendWithSmtp } from '../server/utils/smtp.js';
+
+/*
+  The `email:send` handler: sends the email through the SMTP server in the settings and says how it went (kempo keeps the log). With no
+  host saved it answers that it did not send, so another email extension, if there is one, gets its turn.
+*/
+export default async ({ email }) => {
+  const read = async (name, fallback) => (await getSetting('kempo-smtp', name, fallback))[1];
+  const settings = {
+    host: String(await read('smtp_host', '')).trim(),
+    port: Number(await read('smtp_port', 465)),
+    secure: (await read('smtp_secure', true)) === true,
+    user: String(await read('smtp_user', '')).trim(),
+    password: String(await read('smtp_password', '')),
+  };
+  return sendWithSmtp({ settings, email });
+};

package/install.js

@@ -0,0 +1,8 @@
+/* Nothing to create: the settings come from kempo-config.json. Says what is still to do, and warns when kempo cannot store the password. */
+export default async () => {
+  if(!process.env.SETTINGS_ENCRYPTION_KEY){
+    console.warn('[kempo-smtp] SETTINGS_ENCRYPTION_KEY is not set, so the SMTP password cannot be stored. Start the server once to have kempo make one, then reinstall this extension.');
+    return;
+  }
+  console.log('[kempo-smtp] Installed. Add the SMTP host, port, user and password on the SMTP tab of the Settings page, and set the Email from address to the mailbox address.');
+};

package/kempo-config.json

@@ -0,0 +1,39 @@
+{
+  "docs": "https://github.com/dustinpoissant/kempo-smtp#readme",
+  "git": "https://github.com/dustinpoissant/kempo-smtp",
+  "hooks": {
+    "email:send": "./hooks/send.js"
+  },
+  "settings": [
+    {
+      "name": "smtp_host",
+      "value": "",
+      "type": "string",
+      "description": "The address of your SMTP server, such as smtp.hostinger.com. Your email provider lists it with the mailbox settings"
+    },
+    {
+      "name": "smtp_port",
+      "value": "465",
+      "type": "number",
+      "description": "The SMTP port: 465 for a connection that is secure from the start, 587 for one that upgrades to secure"
+    },
+    {
+      "name": "smtp_secure",
+      "value": "true",
+      "type": "boolean",
+      "description": "Secure from the start (turn on for port 465). Turn off for port 587, which upgrades to a secure connection after connecting"
+    },
+    {
+      "name": "smtp_user",
+      "value": "",
+      "type": "string",
+      "description": "The username to sign in with, which is usually the full mailbox address such as orders@yourdomain.com. Leave empty for a server that needs none"
+    },
+    {
+      "name": "smtp_password",
+      "value": "",
+      "type": "secret",
+      "description": "The password of that mailbox. Stored encrypted and never shown again"
+    }
+  ]
+}

package/package.json

@@ -1,6 +1,38 @@
 {
   "name": "kempo-smtp",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.1.0",
+  "description": "Sends the emails kempo logs through any SMTP server: a Hostinger or Google Workspace mailbox, Amazon SES, Mailgun, or a mail server of your own.",
+  "license": "MIT",
+  "author": "Dustin Poissant",
+  "type": "module",
+  "scripts": {
+    "test": "node --test"
+  },
+  "files": [
+    "admin",
+    "hooks",
+    "server",
+    "install.js",
+    "update.js",
+    "uninstall.js",
+    "kempo-config.json"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/dustinpoissant/kempo-smtp.git"
+  },
+  "dependencies": {
+    "nodemailer": "^7.0.0"
+  },
+  "devDependencies": {
+    "smtp-server": "^3.13.0"
+  },
+  "peerDependencies": {
+    "kempo": ">=4.8.0"
+  },
+  "exports": {
+    "./install": "./install.js",
+    "./update": "./update.js",
+    "./uninstall": "./uninstall.js"
+  }
+}

package/server/utils/smtp.js

@@ -0,0 +1,76 @@
+import nodemailer from 'nodemailer';
+
+/*
+  Sending one email through an SMTP server with nodemailer. The settings and the email are given, and so is `createTransport`, so a test
+  can use a real SMTP server of its own and nothing else is needed. It never throws: it resolves to what an email:send handler answers,
+  `{ sent: true, provider, messageId }` or `{ sent: false, error }` (with `skipped: true` when no host is set up, so kempo knows it did not try).
+
+  `secure: true` connects with TLS straight away (port 465, what most providers call SSL). `secure: false` connects plainly and upgrades with
+  STARTTLS when the server offers it (port 587); the password is not sent before that unless the server cannot do TLS at all.
+*/
+const TIMEOUT_MS = 20 * 1000;
+
+const ENTITIES = { '&amp;': '&', '&lt;': '<', '&gt;': '>', '&quot;': '"', '&#39;': '\'', '&nbsp;': ' ', '&times;': 'x', '&middot;': '-' };
+
+/* A plain-text version of the HTML for mail programs that show text (and spam filters that like to see one). */
+export const htmlToText = html => String(html ?? '')
+  .replace(/<(style|script)[\s\S]*?<\/\1>/gi, '')
+  .replace(/<br\s*\/?>/gi, '\n')
+  .replace(/<\/(p|div|h[1-6]|tr|li|blockquote|table)>/gi, '\n')
+  .replace(/<a\b[^>]*href="([^"]*)"[^>]*>([\s\S]*?)<\/a>/gi, (match, href, text) => (text.replace(/<[^>]*>/g, '').trim() === href ? href : `${text.replace(/<[^>]*>/g, '').trim()} (${href})`))
+  .replace(/<[^>]*>/g, '')
+  .replace(/&(?:amp|lt|gt|quot|#39|nbsp|times|middot);/g, entity => ENTITIES[entity])
+  .replace(/[ \t]+\n/g, '\n')
+  .replace(/\n{3,}/g, '\n\n')
+  .trim();
+
+/* The sentence for a failure, in plain words, from what nodemailer reports. */
+export const explain = (error, { host, port }, from = '') => {
+  const reason = String(error?.response ?? error?.message ?? 'no reason given').replace(/\s+/g, ' ').trim();
+  switch(error?.code){
+    case 'EAUTH': return `The SMTP server did not accept the username or password (${reason})`;
+    case 'ECONNREFUSED': return `Nothing is listening at ${host}:${port}. Check the host and port`;
+    case 'ENOTFOUND': case 'EDNS': return `The host ${host} was not found`;
+    case 'ETIMEDOUT': case 'ESOCKET': case 'ECONNECTION': case 'ECONNRESET': return `Could not connect to ${host}:${port} (${reason}). Check the host, the port and whether the connection is meant to be secure`;
+    case 'EENVELOPE': return /sender|from|5\.7\.|domain/i.test(reason) && from
+      ? `The SMTP server refused the sender address ${from} (${reason}). Set Email from, on the System tab of the Settings page, to the mailbox address or one of its aliases`
+      : `The SMTP server refused the address: ${reason}`;
+    case 'EMESSAGE': return `The SMTP server refused the message: ${reason}`;
+    default: return `SMTP: ${reason}`;
+  }
+};
+
+/* The sender to use: the site's, or when the site has not set one (a placeholder), the mailbox signed in with, which is what an SMTP server accepts. */
+export const senderFor = (email, settings) => (email.fromIsDefault && /^[^\s@<>]+@[^\s@<>]+\.[^\s@<>]+$/.test(settings?.user ?? '') ? settings.user : email.from);
+
+export const validate = settings => {
+  if(!settings?.host) return 'No SMTP server is set up. Add the host on the SMTP tab of the Settings page';
+  if(!Number.isInteger(settings.port) || settings.port < 1 || settings.port > 65535) return 'The SMTP port must be a number from 1 to 65535';
+  return null;
+};
+
+export const sendWithSmtp = async ({ settings, email, createTransport = options => nodemailer.createTransport(options) }) => {
+  const problem = validate(settings);
+  if(problem) return { sent: false, ...(settings?.host ? {} : { skipped: true }), error: problem };
+  const from = senderFor(email, settings);
+
+  const transport = createTransport({
+    host: settings.host,
+    port: settings.port,
+    secure: settings.secure,
+    ...(settings.user ? { auth: { user: settings.user, pass: settings.password } } : {}),
+    connectionTimeout: TIMEOUT_MS,
+    greetingTimeout: TIMEOUT_MS,
+    socketTimeout: TIMEOUT_MS * 2,
+  });
+
+  try {
+    const info = await transport.sendMail({ from, to: email.to, subject: email.subject, html: email.html, text: htmlToText(email.html) });
+    if(Array.isArray(info.accepted) && info.accepted.length === 0) return { sent: false, error: `The SMTP server did not accept ${email.to}${info.rejected?.length ? ` (rejected: ${info.rejected.join(', ')})` : ''}` };
+    return { sent: true, provider: 'smtp', messageId: info.messageId };
+  } catch(error) {
+    return { sent: false, error: explain(error, settings, from) };
+  } finally {
+    transport.close?.();
+  }
+};

package/uninstall.js

@@ -0,0 +1,2 @@
+/* The hook and the setting are removed with the extension, so there is nothing to undo. */
+export default async () => {};

package/update.js

@@ -0,0 +1,6 @@
+import install from './install.js';
+
+/* An update repeats the install, which only says what is left to set up. */
+export default async () => {
+  await install();
+};