haraka 0.0.32 → 3.3.0

package/docs/tutorials/Migrating_from_v1_to_v2.md

@@ -0,0 +1,96 @@
+# Migrating from Haraka v1.x to v2.x
+
+Haraka v2.x contains two significant changes to the v1.x API related to streams.
+
+Streams are an abstraction over a data flow that is provided by Node core and is used throughout node to "pipe" data between two places or more. This makes programming very easy, and is hence why we started using them in Haraka starting with version 2.0.0.
+
+For more information about the Stream API, see http://nodejs.org/api/stream.html
+
+Note that when using bundled Haraka plugins, it's very unlikely you will need to change anything. Though you may want to configure `spool_dir` and `spool_after` in `config/smtp.ini` (v2.x), or in `config/connection.ini` (v3.1 or newer). If you have custom plugins, continue reading.
+
+## Changes To Look For
+
+Firstly, the incoming data in an email (the email body) is now stored in an object which you can treat as a ReadableStream. To find if this is relevant for you, look for instances of `data_lines` in your plugins.
+
+Secondly, if you parse the mail body, attachments are now provided as a stream, rather than custom start/data/end events. To find if this is relevant for you, look for instances of `attachment_hooks` in your plugins.
+
+## Fixing data_lines plugins
+
+Any plugins now working on each line of data will need to change to using a stream. The stream is called `transaction.message_stream`.
+
+These changes may be complicated if you are iterating over each line and doing something with the strings therein. However if you are piping the data to an application or over a network, your code will become significantly simpler (and a lot faster).
+
+In v1.x Haraka populated the `transaction.data_lines` array for each line of data received. If you were writing the data to a socket then you had to handle backpressure manually by checking the return of `write()` and adding `on('drain')` handlers like so:
+
+    var data_marker = 0;
+    var in_data = false;
+    var end_pending = true;
+    var send_data = function () {
+        in_data = true;
+        var wrote_all = true;
+        while (wrote_all && (data_marker < connection.transaction.data_lines.length)) {
+            var line = connection.transaction.data_lines[data_marker];
+            data_marker++;
+            wrote_all = socket.write(Buffer.from(line.replace(/^\./, '..').replace(/\r?\n/g, '\r\n')), 'binary');
+            if (!wrote_all) return;
+        }
+        // we get here if wrote_all still true, and we got to end of data_lines
+        if (end_pending) {
+            end_pending = false;
+            // Finished...
+            socket.send_command('dot');
+        }
+    };
+    socket.on('drain', function () {
+        if (end_pending && in_data) {
+            setImmediate(function () { send_data() });
+        }
+    });
+
+In v2.x this now becomes:
+
+    connection.transaction.message_stream.pipe(socket, {ending_dot: true});
+
+This automatically chunks the data, handles backpressure and will apply any
+necessary format changes. See `docs/Transaction.md` for the full details.
+
+If you need to handle the input data by line, then you will need to create
+your own writable stream and then pipe the message to the stream and then
+extract the lines from the stream of data. See the `dkim` plugin for
+an example.
+
+## Fixing attachment_hooks plugins
+
+For v1.x you passed in functions to `transaction.attachment_hooks()` as
+follows:
+
+    transaction.attachment_hooks(
+        function (ctype, filename, body) {...}, // start
+        function (buf) {...}, // data
+        function () {...} // end
+    );
+
+That has now changed to:
+
+    transaction.attachment_hooks(
+        function (ctype, filename, body, stream) {...}, // start
+    );
+
+This allows you to attach the stream to other streams via `stream.pipe(dest)`.
+
+Sometimes destination streams will apply backpressure on the sending stream,
+for example if you are sending attachments to a remote service. In order
+for this backpressure to apply to the connection itself (so that we don't
+have to buffer up data in memory), we need to provide the connection object
+to the stream:
+
+    var transaction = connection.transaction;
+    transaction.attachment_hooks(
+        function (ctype, filename, body, stream) {
+            stream.connection = connection;
+            ...
+        }
+    );
+
+For a full example of using attachment streams, see the Transaction.md
+documentation file.

package/docs/tutorials/SettingUpOutbound.md

@@ -0,0 +1,62 @@
+# Configuring Haraka For Outbound Email
+
+It is straightforward to run Haraka as an outbound (submission) mail server. Before turning on the server itself, get a few external things in order:
+
+- **DNS PTR record** — make sure it matches the A/AAAA record of the host you are sending from. Receivers that disagree on `HELO`/PTR will treat your mail with suspicion.
+- **SPF, DKIM, and DMARC** — publish records for any domain you send from. Most receivers downgrade or reject mail without them. Haraka signs outbound with [haraka-plugin-dkim](https://github.com/haraka/haraka-plugin-dkim).
+- **Reverse DNS at the IP owner** — if your hosting provider controls the PTR, set the value through their console.
+
+How to provision DNS varies by provider; the records are network-specific so no one-size-fits-all command applies.
+
+## Background
+
+Haraka treats outbound mail as "relaying". When any plugin sets `connection.relaying = true`, the message is queued for outbound delivery once `DATA` ends. The outbound engine then tries each MX in sequence; on permanent failure a DSN is generated and sent to the `MAIL FROM` address. If the DSN itself bounces, Haraka logs the "double bounce" and drops it.
+
+## Setup
+
+Modern submission uses **implicit TLS on port 465** (RFC 8314); port 587 with `STARTTLS` is also still common. Plain port 25 is for server-to-server traffic and should not be used for submission.
+
+Create a new Haraka instance:
+
+```sh
+haraka -i haraka-outbound
+cd haraka-outbound
+```
+
+In `config/smtp.ini`, set the listener:
+
+```ini
+listen=[::0]:465,[::0]:587
+smtps_port=465
+```
+
+Anything in `smtps_port` runs implicit TLS; the other ports advertise `STARTTLS`.
+
+Enable just the TLS and auth plugins. AUTH is only advertised after TLS is established (except for connections from localhost):
+
+```sh
+cat > config/plugins <<'EOF'
+tls
+auth/flat_file
+EOF
+```
+
+Add a user to `config/auth_flat_file.ini`. See [`docs/plugins/auth/flat_file.md`](../plugins/auth/flat_file.md) for the format.
+
+Start Haraka:
+
+```sh
+haraka -c .
+```
+
+In another shell, test with [swaks](https://www.jetmore.org/john/code/swaks/) — substitute your real test address and the credentials you configured:
+
+```sh
+swaks --to youremail@yourdomain.com --from test@example.com \
+    --server localhost --port 587 --tls \
+    --auth-user testuser --auth-password testpassword
+```
+
+For port 465 (implicit TLS), use `--tls-on-connect` instead of `--tls`.
+
+Watch the swaks output for errors and confirm the message arrives. That's all the basic configuration you need; once you're satisfied, turn on DKIM signing for the domains you send from.

package/eslint.config.mjs

@@ -0,0 +1,2 @@
+import haraka from '@haraka/eslint-config'
+export default [...haraka]

package/haraka.js

@@ -0,0 +1,74 @@
+#!/usr/bin/env node
+
+'use strict'
+const path = require('path')
+const makePathJoin = () => path.join(process.env.HARAKA, 'node_modules')
+
+if (!process.env.HARAKA) {
+    console.warn('WARNING: Not running installed Haraka - command line arguments ignored')
+}
+
+// this must be set before "server.js" is loaded
+process.env.HARAKA = process.env.HARAKA || path.resolve('.')
+try {
+    require.paths.push(makePathJoin())
+} catch {
+    process.env.NODE_PATH = process.env.NODE_PATH ? `${process.env.NODE_PATH}:${makePathJoin()}` : makePathJoin()
+    require('module')._initPaths() // Horrible hack
+}
+
+const utils = require('haraka-utils')
+const logger = require('./logger')
+const server = require('./server')
+
+exports.version = utils.getVersion(__dirname)
+
+process.on('uncaughtException', (err) => {
+    if (err.stack) {
+        for (const line of err.stack.split('\n')) logger.crit(line)
+    } else {
+        logger.crit(`Caught exception: ${JSON.stringify(err)}`)
+    }
+    logger.dump_and_exit(1)
+})
+
+let shutting_down = false
+const signals = ['SIGINT']
+
+if (process.pid === 1) signals.push('SIGTERM')
+
+for (const sig of signals) {
+    process.on(sig, () => {
+        if (shutting_down) return process.exit(1)
+        shutting_down = true
+        const [, filename] = process.argv
+        process.title = path.basename(filename, '.js')
+
+        logger.notice(`${sig} received`)
+        logger.dump_and_exit(() => {
+            if (server.cluster?.isMaster) {
+                server.performShutdown()
+            } else if (!server.cluster) {
+                server.performShutdown()
+            }
+        })
+    })
+}
+
+process.on('SIGHUP', () => {
+    logger.notice('Flushing the temp fail queue')
+    server.flushQueue()
+})
+
+process.on('exit', () => {
+    if (shutting_down) return
+    const [, filename] = process.argv
+    process.title = path.basename(filename, '.js')
+
+    logger.notice('Shutting down')
+    logger.dump_logs()
+})
+
+logger.log('NOTICE', `Starting up Haraka version ${exports.version}`)
+
+server.createServer()

package/haraka.sh

@@ -0,0 +1,2 @@
+#!/bin/sh
+exec haraka -c /usr/local/haraka 2>&1

package/http/html/404.html

@@ -0,0 +1,58 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8" />
+    <title>Page Not Found</title>
+    <meta name="viewport" content="width=device-width, initial-scale=1" />
+    <style>
+      * {
+        line-height: 1.2;
+        margin: 0;
+      }
+
+      html {
+        color: #888;
+        display: table;
+        font-family: sans-serif;
+        height: 100%;
+        text-align: center;
+        width: 100%;
+      }
+
+      body {
+        display: table-cell;
+        vertical-align: middle;
+        margin: 2em auto;
+      }
+
+      h1 {
+        color: #555;
+        font-size: 2em;
+        font-weight: 400;
+      }
+
+      p {
+        margin: 0 auto;
+        width: 280px;
+      }
+
+      @media only screen and (max-width: 280px) {
+        body,
+        p {
+          width: 95%;
+        }
+
+        h1 {
+          font-size: 1.5em;
+          margin: 0 0 0.3em;
+        }
+      }
+    </style>
+  </head>
+
+  <body>
+    <h1>Page Not Found</h1>
+    <p>Sorry, but the page you were trying to view does not exist.</p>
+  </body>
+</html>
+<!-- IE needs 512+ bytes: https://docs.microsoft.com/archive/blogs/ieinternals/friendly-http-error-pages -->

package/http/html/index.html

@@ -0,0 +1,47 @@
+<!doctype html>
+<html>
+  <head>
+    <meta charset="utf-8" />
+    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
+    <title></title>
+    <meta name="description" content="" />
+    <meta name="viewport" content="width=device-width, initial-scale=1" />
+
+    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/3.5.1/css/bootstrap.min.css" />
+    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/3.5.1/css/bootstrap-theme.min.css" />
+
+    <script src="https://code.jquery.com/jquery-3.5.1.min.js" async></script>
+    <script src="https://stackpath.bootstrapcdn.com/bootstrap/3.5.1/js/bootstrap.min.js" async></script>
+  </head>
+  <body>
+    <!--[if lt IE 7]>
+      <p class="browsehappy">
+        You are using an <strong>outdated</strong> browser. Please
+        <a href="https://browsehappy.com/">upgrade your browser</a> to improve your experience.
+      </p>
+    <![endif]-->
+
+    <ul id="tabList" class="nav nav-tabs" role="tablist navigation">
+      <li class="active">
+        <a href="#home" role="tab" data-toggle="tab">Home</a>
+      </li>
+    </ul>
+
+    <div id="bodyDivContainer" class="tab-content">
+      <div class="tab-pane fade in active" id="home">
+        <h1>Haraka, a mail server (MTA).</h1>
+
+        <p>
+          You are visiting an installation of
+          <a href="https://haraka.github.io">Haraka</a>.
+        </p>
+
+        <p>
+          Haraka is on the
+          <a href="https://haraka.github.io">web</a> and
+          <a href="https://github.com/haraka/Haraka">GitHub</a>
+        </p>
+      </div>
+    </div>
+  </body>
+</html>

package/http/package.json

@@ -0,0 +1,21 @@
+{
+  "name": "haraka-httpd",
+  "version": "1.0.1",
+  "description": "Haraka HTTP for SMTPd",
+  "main": "server",
+  "keywords": [
+    "http",
+    "watch"
+  ],
+  "authors": [
+    "Matt Simerson"
+  ],
+  "private": true,
+  "license": "MIT",
+  "homepage": "http://haraka.github.io/",
+  "dependencies": {
+    "express": "^4.0"
+  },
+  "devDependencies": {},
+  "scripts": {}
+}

package/line_socket.js

@@ -0,0 +1,24 @@
+'use strict'
+// Back-compat shim. The Socket class lives in haraka-net-utils as LineSocket;
+// the line-processing helper is `add_line_processor` there. The connect()
+// helper stays here because it depends on Haraka's tls_socket.
+
+const { LineSocket, add_line_processor } = require('haraka-net-utils')
+
+const tls_socket = require('./tls_socket')
+
+exports.Socket = LineSocket
+
+// New interface - uses TLS
+exports.connect = (port, host) => {
+    let options = {}
+    if (typeof port === 'object') {
+        options = port
+    } else {
+        options.port = port
+        options.host = host
+    }
+    const sock = tls_socket.connect(options)
+    add_line_processor(sock)
+    return sock
+}