haraka 0.0.33 → 3.3.0

package/docs/Tutorial.md

@@ -0,0 +1,183 @@
+# Writing Haraka Plugins
+
+Part of the joy of using Haraka as your main mail server is having a strong plugin system: you control every aspect of how mail is processed, accepted, and delivered.
+
+This tutorial walks through a small plugin that supports *disposable addresses*: an email like `user-20271231@example.com` is accepted up until 31 December 2027, after which delivery is rejected. Mail that is still within the validity window is rewritten back to `user@example.com` before it is forwarded on.
+
+## What You'll Need
+
+- Node.js (an active LTS release) and npm
+- Haraka
+- A text editor
+- [swaks][swaks] for sending test mail
+
+## Getting Started
+
+Install Haraka and create a project:
+
+```sh
+sudo npm install -g Haraka
+haraka -i /path/to/new_project
+```
+
+Use a directory that does not yet exist. Now scaffold a plugin:
+
+```sh
+haraka -c /path/to/new_project -p rcpt_to.disposable
+```
+
+`haraka -p` reports the files it created:
+
+```
+Plugin rcpt_to.disposable created
+Now edit javascript in:    /path/to/new_project/plugins/rcpt_to.disposable.js
+Add the plugin to config:  /path/to/new_project/config/plugins
+And edit documentation in: /path/to/new_project/docs/plugins/rcpt_to.disposable.md
+```
+
+Edit `config/plugins` so the only enabled lines are:
+
+```
+rcpt_to.disposable
+rcpt_to.in_host_list
+queue/test
+```
+
+The ordering matters — the disposable plugin must run *before* `rcpt_to.in_host_list`, which accepts mail for domains listed in `config/host_list`. `queue/test` writes accepted mail to a `.eml` file in `os.tmpdir()` so you can confirm delivery.
+
+Open `plugins/rcpt_to.disposable.js` and start with:
+
+```js
+exports.hook_rcpt = (next, connection, params) => {
+    const rcpt = params[0]
+    connection.loginfo(`got recipient: ${rcpt}`)
+    next()
+}
+```
+
+Verify it works. In one terminal:
+
+```sh
+echo LOGDEBUG > config/loglevel
+echo myserver.com >> config/host_list
+sudo haraka -c /path/to/new_project
+```
+
+In another:
+
+```sh
+swaks -h example.com -t booya@myserver.com -f sender@example.com -s localhost -p 25
+```
+
+You should see something like this in the Haraka log:
+
+```
+[INFO] [<uuid>] [rcpt_to.disposable] got recipient: <booya@myserver.com>
+```
+
+…and a `.eml` file in your system temp directory containing the message.
+
+## Parsing Out the Date
+
+Detect addresses of the form `user-YYYYMMDD` and parse the date:
+
+```js
+exports.hook_rcpt = (next, connection, params) => {
+    const rcpt = params[0]
+    connection.loginfo(`got recipient: ${rcpt}`)
+
+    const match = /^(.*)-(\d{4})(\d{2})(\d{2})$/.exec(rcpt.user)
+    if (!match) return next()
+
+    // Date constructor uses zero-indexed months (Dec === 11)
+    const expiry = new Date(match[2], match[3] - 1, match[4])
+    connection.loginfo(`expires on: ${expiry.toISOString()}`)
+
+    next()
+}
+```
+
+Restart Haraka and send:
+
+```sh
+swaks -h example.com -t booya-20271231@myserver.com \
+    -f sender@example.com -s localhost -p 25
+```
+
+Logs:
+
+```
+[INFO] [rcpt_to.disposable] got recipient: <booya-20271231@myserver.com>
+[INFO] [rcpt_to.disposable] expires on: 2027-12-31T00:00:00.000Z
+```
+
+## Rejecting Expired Addresses
+
+Compare the parsed date to today and reject if it has already passed:
+
+```js
+exports.hook_rcpt = (next, connection, params) => {
+    const rcpt = params[0]
+
+    const match = /^(.*)-(\d{4})(\d{2})(\d{2})$/.exec(rcpt.user)
+    if (!match) return next()
+
+    const expiry = new Date(match[2], match[3] - 1, match[4])
+    if (expiry < new Date()) {
+        return next(DENY, 'Expired email address')
+    }
+
+    next()
+}
+```
+
+Send mail to an expired address:
+
+```sh
+swaks -h example.com -t booya-20200101@myserver.com \
+    -f sender@example.com -s localhost -p 25
+```
+
+The remote end sees:
+
+```
+<** 550 Expired email address
+```
+
+## Rewriting Live Addresses
+
+When the address is still valid, strip the date tag so the downstream mail store receives plain `user@domain`:
+
+```js
+exports.hook_rcpt = (next, connection, params) => {
+    const rcpt = params[0]
+
+    const match = /^(.*)-(\d{4})(\d{2})(\d{2})$/.exec(rcpt.user)
+    if (!match) return next()
+
+    const expiry = new Date(match[2], match[3] - 1, match[4])
+    if (expiry < new Date()) {
+        return next(DENY, 'Expired email address')
+    }
+
+    rcpt.user = match[1]
+    connection.loginfo(`rewrote recipient to: ${rcpt}`)
+    next()
+}
+```
+
+Send to a live tagged address and watch the log:
+
+```
+[INFO] [rcpt_to.disposable] rewrote recipient to: <booya@myserver.com>
+```
+
+## Further Reading
+
+The Haraka API offers much more — body and header access, ESMTP extension hooks, the outbound delivery hooks, structured results, attachments, and so on. Two good starting points:
+
+- The [Plugins guide](Plugins.md) and the rest of the `docs/` directory.
+- The [Plugin registry](https://github.com/haraka/Haraka/blob/master/PLUGINS.md) for an inventory of real-world plugins.
+- The plugins shipped in the [`plugins/`](../plugins/) directory. Even the most elaborate are under 200 lines; many are under 20.
+
+[swaks]: https://www.jetmore.org/john/code/swaks/

package/docs/deprecated/access.md

@@ -0,0 +1,3 @@
+# access - ACLs
+
+Repackaged as [haraka-plugin-access](https://github.com/haraka/haraka-plugin-access).

package/docs/deprecated/backscatterer.md

@@ -0,0 +1,9 @@
+# backscatterer
+
+This is a very basic pluign that checks the connecting IP against
+ips.backscatterer.org when the envelope-from is null or postmaster@
+as per the instructions at http://www.backscatterer.org/?target=usage
+
+This plugin is used to reject misdirected bounces and autoresponders
+and sender callouts from abusive systems which can happen when a
+local domain is spoofed and used as the envelope-from in a spam run.

package/docs/deprecated/connect.rdns_access.md

@@ -0,0 +1,53 @@
+## DEPRECATION NOTICE
+
+See [haraka-plugin-access](https://github.com/haraka/haraka-plugin-access)
+for upgrade instructions.
+
+# connect.rdns_access
+
+This plugin will evaluate the remote IP address and the remote rDNS hostname
+against a set of white and black lists. The lists are applied in the following
+way:
+
+connect.rdns_access.whitelist (pass)
+connect.rdns_access.whitelist_regex (pass)
+connect.rdns_access.blacklist (block)
+connect.rdns_access.blacklist_regex (block)
+
+## Configuration connect.rdns_access.ini
+
+General configuration file for this plugin.
+
+- connect.rdns_access.general.deny_msg
+
+  Text to send the user on reject (text).
+
+## Configuration connect.rdns_access.whitelist
+
+The whitelist is mostly to counter blacklist entries that match more than
+what one would want. This file should be used for a specific IP address
+or rDNS hostnames, one per line, that should bypass blacklist checks.
+NOTE: We heavily suggest tailoring blacklist entries to be as accurate as
+possible and never using whitelists. Nevertheless, if you need whitelists,
+here they are.
+
+## Configuration connect.rdns_access.whitelist_regex
+
+Does the same thing as the whitelist file, but each line is a regex.
+Each line is also anchored for you, meaning '^' + regex + '$' is added for
+you. If you need to get around this restriction, you may use a '.\*' at
+either the start or the end of your regex. This should help prevent people
+from writing overly permissive rules on accident.
+
+## Configuration connect.rdns_access.blacklist
+
+This file should be used for a specific IP address or rDNS hostnames, one
+per line, that should fail on connect.
+
+## Configuration connect.rdns_access.blacklist_regex
+
+Does the same thing as the blacklist file, but each line is a regex.
+Each line is also anchored for you, meaning '^' + regex + '$' is added for
+you. If you need to get around this restriction, you may use a '.\*' at
+either the start or the end of your regex. This should help prevent people
+from writing overly permissive rules on accident.

package/docs/deprecated/data.headers.md

@@ -0,0 +1,3 @@
+# data.headers
+
+Deprecated by [haraka-plugin-headers](https://github.com/haraka/haraka-plugin-headers/)

package/docs/deprecated/data.nomsgid.md

@@ -0,0 +1,7 @@
+# data.nomsgid
+
+NOTICE: this plugin is deprecated. Use data.headers instead.
+
+Quite simply enabling this plugin blocks all mails lacking a Message-Id
+header. This is an aggressive anti-spam measure, but since most mail systems
+will add a Message-Id header, it tends to block a good chunk of abusive mail.

package/docs/deprecated/data.noreceived.md

@@ -0,0 +1,11 @@
+# data.noreceived
+
+NOTICE: this plugin is deprecated. Use data.headers instead.
+
+This plugin very simply blocks any mail arriving at your system that has no
+`Received` headers.
+
+This is an aggressive anti-spam measure, but works because all real mail
+relays will add a `Received` header according to the RFCs. It may false
+positive on some bulk mail that uses a custom tool to send, but this appears
+to be fairly rare.

package/docs/deprecated/data.rfc5322_header_checks.md

@@ -0,0 +1,11 @@
+# data.rfc5322_header_checks
+
+NOTICE: this plugin is deprecated. Use data.headers instead.
+
+This plugin enforces RFC 5322 Section 3.6 which states that:
+
+All messages MUST have a 'Date' and 'From' header and a message may not contain
+more than one 'Date', 'From', 'Sender', 'Reply-To', 'To', 'Cc', 'Bcc',
+'Message-Id', 'In-Reply-To', 'References' or 'Subject' header.
+
+Any message that does not meet these requirements will be rejected.

package/docs/deprecated/dkim_sign.md

@@ -0,0 +1,97 @@
+# dkim_sign
+
+This plugin implements the [DKIM Core specification](dkimcore.org).
+
+This plugin only _signs_ outbound messages. It does not validate DKIM signatures.
+
+## Getting Started
+
+Generate a DKIM selector and keys for your domain:
+
+```sh
+cd /path/to/haraka/config/dkim
+./dkim_key_gen.sh example.org
+```
+
+Within the config/dkim/${domain} directory will be 4 files:
+
+```sh
+ls config/dkim/example.org/
+dns private public selector
+```
+
+The selector file contains the DNS label where the DKIM public key is published. The `private` and `public` files contain the DKIM keys.
+
+The `dns` file contains a formatted record of the public key suitable for copy/pasting into your domains zone file. It also has suggestions for DKIM, SPF, and DMARC policy records.
+
+The DKIM DNS record will look like this:
+
+    may2013._domainkey TXT "v=DKIM1;p=[public key stripped of whitespace];"
+
+The values in the address have the following meaning:
+
+    hash: h=[ sha1 | sha256 ]
+    test; t=[ s | s:y ]
+    granularity: g=[ ]
+    notes: n=[ ]
+    services: s=[email]
+    keytypes: [ rsa ]
+
+## Key size
+
+The default key size created by `dkim_key_gen.sh` is 2048. That is considered secure as of mid-2014 but after 2020, you should be using 4096.
+
+# What to sign
+
+The DKIM signing key for messages from example.org _should_ be signed with
+a DKIM key for example.org. Failing to do so will result in messages not
+having an _aligned_ DKIM signature. For DMARC enabled domains, this will
+likely result in deliverability problems.
+
+For correct alignment, Haraka signs each message with that domains DKIM key.
+For an alternative, see the legacy Single Domain Configuration below.
+
+# Configuration
+
+This plugin is configured in `dkim_sign.ini`.
+
+- disabled = [ 1 | true | yes ] (OPTIONAL)
+
+  Set this to disable DKIM signing
+
+- headers_to_sign = list, of; headers (REQUIRED)
+
+  Set this to the list of headers that should be signed, separated by commas, colons or semi-colons. Signing prevents tampering with the specified headers.
+  The 'From' header is required by the RFC and will be added if missing.
+
+## Single Domain Configuration
+
+To sign all messages with a single DKIM key, you must set the selector and domain in dkim_sign.ini. You must also save your DKIM private key in the file `dkim.private.key` in the Haraka config directory.
+
+- selector = name
+
+  Set this to the selector name published in DNS under the
+  \_domainkey sub-domain of the domain referenced below.
+
+- domain = name
+
+  Set this to the domain name that will be used to sign messages
+  which don't match a per-domain DKIM key. The DNS TXT entry for:
+
+        <selector>._domainkey.<domain>
+
+Test that your DKIM key is published properly with a DNS request like this:
+
+```sh
+drill TXT $SELECTOR._domainkey.$DOMAIN
+dig TXT $SELECTOR._domainkey.$DOMAIN +short
+```
+
+### Example DNS query
+
+```sh
+export SELECTOR=mar2013
+export DOMAIN=simerson.net
+$ dig TXT $SELECTOR._domainkey.$DOMAIN +short
+"v=DKIM1;p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoyUzGOTSOmakY8BcxXgi0mN/nFegLBPs7aaGQUtjHfa8yUrt9T2j6GSXgdjLuG3R43WjePQv3RHzc+bwwOkdw0XDOXiztn5mhrlaflbVr5PMSTrv64/cpFQKLtgQx8Vgqp7Dh3jw13rLomRTqJFgMrMHdhIibZEa69gtuAfDqoeXo6QDSGk5JuBAeRHEH27FriHulg5ob" "4F4lmh7fMFVsDGkQEF6jaIVYqvRjDyyQed3R3aTJX3fpb3QrtRqvfn/LAf+3kzW58AjsERpsNCSTD2RquxbnyoR/1wdGKb8cUlD/EXvqtvpVnOzHeSeMEqex3kQI8HOGsEehWZlKd+GqwIDAQAB"
+```

package/docs/deprecated/dkim_verify.md

@@ -0,0 +1,28 @@
+# dkim_verify
+
+This plugin will verify DKIM signatures as defined by RFC 6376 and add
+an Authentication-Results header as appropriate.
+
+## Configuration
+
+- allowed_time_skew
+
+  How far can we stretch on time matching, in secs. Useful when clock is skewed.
+
+- sigerror_log_level
+
+## Testing
+
+This plugin also provides a command-line test tool that can be used to
+debug DKIM issues or to check results.
+
+```
+# dkimverify < message
+identity="@gmail.com" domain="gmail.com" result=pass
+```
+
+You can add `--debug` to the option arguments to see a full trace of the processing.
+
+## Notes
+
+This plugin and underlying library does not currently support DKIM body length limits (l=).

package/docs/deprecated/dnsbl.md

@@ -0,0 +1,80 @@
+# dnsbl
+
+This plugin looks up the connecting IP address in an IP blocklist. Mails
+found to be in the blocklist are rejected.
+
+## Configuration
+
+This plugins uses the following files:
+
+dnsbl.zones - Contains a list of zones to query, one per line.
+
+dnsbl.ini - INI format with options described below:
+
+- zones
+
+  A comma or semi-colon list of zones to query. It will be merged with
+  any lists in dnsbl.zones.
+
+- periodic_checks
+
+  If enabled, this will check all the zones every n minutes.
+  The minimum value that will be accepted here is 5. Any value less
+  than 5 will cause the checks to be run at start-up only.
+
+  The checks confirm that the list is responding and that it is not
+  listing the world. If any errors are detected, then the zone is
+  disabled and will be re-checked on the next test. If a zone
+  subsequently starts working correctly then it will be re-enabled.
+
+- enable_stats
+
+  To use this feature you must have installed the 'redis' module and
+  have a redis server running.
+
+  When enabled, this will record several list statistics to redis.
+
+  It will track the total number of queries (TOTAL) and the average
+  response time (AVG_RT) and the return type (e.g. LISTED or ERROR)
+  to a redis hash where the key is 'dns-list-stat:zone' and the hash
+  field is the response type.
+
+  It will also track the positive response overlap between the lists
+  in another redis hash where the key is 'dns-list-overlap:zone' and
+  the hash field is the other list names.
+
+  Example:
+    <pre><code>redis 127.0.0.1:6379> hgetall dns-list-stat:zen.spamhaus.org
+    1) "TOTAL"
+    2) "23"
+    3) "ENOTFOUND"
+    4) "11"
+    5) "LISTED"
+    6) "12"
+    7) "AVG_RT"
+    8) "45.5"
+    redis 127.0.0.1:6379> hgetall dns-list-overlap:zen.spamhaus.org
+    1) "b.barracudacentral.org"
+    2) "1"
+    3) "bl.spamcop.net"
+    4) "1"
+    5) "TOTAL"
+    6) "1"
+    </code></pre>
+
+- stats_redis_host
+
+  In the form of `host:port` this option allows you to specify a different
+  host on which redis runs.
+
+- reject (default: true)
+
+  Reject connections from IPs that are blacklisted. Setting this to false
+  makes dnsbl informational. reject=false is best used in conjunction with
+  plugins like [karma](/manual/plugins/karma.html) that employ a scoring
+  engine to make choices about message delivery.
+
+- search: (default: first)
+
+  first: consider first DNSBL response conclusive. End processing.
+  all: process all DNSBL results

package/docs/deprecated/dnswl.md

@@ -0,0 +1,73 @@
+# dnswl
+
+This plugin looks up the connecting IP address in an IP whitelist.
+If the host is listed, then the plugin will return OK for all hooks
+up to hook_data.
+
+IMPORTANT! The order of plugins in config/plugins is important when
+this plugin is used. It should be listed _before_ any plugins that
+you wish to skip, but after any plugins that accept recipients.
+
+## Configuration
+
+This plugins uses the following files:
+
+dnswl.zones - Contains a list of zones to query, one per line.
+
+dnswl.ini - INI format with options described below:
+
+- zones
+
+  A comma or semi-colon list of zones to query. It will be merged with
+  any lists in dnswl.zones.
+
+- periodic_checks
+
+  If enabled, this will check all the zones every n minutes.
+  The minimum value that will be accepted here is 5. Any value less
+  than 5 will cause the checks to be run at start-up only.
+
+  The checks confirm that the list is responding and that it is not
+  listing the world. If any errors are detected, then the zone is
+  disabled and will be re-checked on the next test. If a zone
+  subsequently starts working correctly then it will be re-enabled.
+
+- enable_stats
+
+  To use this feature you must have installed the 'redis' module and
+  have a redis server running.
+
+  When enabled, this will record several list statistics to redis.
+
+  It will track the total number of queries (TOTAL) and the average
+  response time (AVG_RT) and the return type (e.g. LISTED or ERROR)
+  to a redis hash where the key is 'dns-list-stat:zone' and the hash
+  field is the response type.
+
+  It will also track the positive response overlap between the lists
+  in another redis hash where the key is 'dns-list-overlap:zone' and
+  the hash field is the other list names.
+
+  Example:
+    <pre><code>redis 127.0.0.1:6379> hgetall dns-list-stat:zen.spamhaus.org
+    1) "TOTAL"
+    2) "23"
+    3) "ENOTFOUND"
+    4) "11"
+    5) "LISTED"
+    6) "12"
+    7) "AVG_RT"
+    8) "45.5"
+    redis 127.0.0.1:6379> hgetall dns-list-overlap:zen.spamhaus.org
+    1) "b.barracudacentral.org"
+    2) "1"
+    3) "bl.spamcop.net"
+    4) "1"
+    5) "TOTAL"
+    6) "1"
+    </code></pre>
+
+- stats_redis_host
+
+  In the form of `host:port` this option allows you to specify a different
+  host on which redis runs.

package/docs/deprecated/lookup_rdns.strict.md

@@ -0,0 +1,67 @@
+# lookup_rdns.strict
+
+This plugin checks the reverse-DNS and compares the resulting addresses
+against forward DNS for a match. If there is no match it sends a
+DENYDISCONNECT, otherwise if it matches it sends an OK. DENYDISCONNECT
+messages are configurable.
+
+## Configuration lookup_rdns.strict.ini
+
+This is the general configuration file for the plugin. In it you can find
+ways to customize user messages, specify timeouts, and some whitelist
+parsing options.
+
+- lookup_rdns.strict.general.nomatch
+
+  Text to send the user if there is no reverse to forward match (text).
+
+- lookup_rdns.strict.general.timeout
+
+  How long we should give this plugin before we time it out (seconds).
+
+- lookup_rdns.strict.general.timeout_msg
+
+  Text to send when plugin reaches timeout (text).
+
+- lookup_rdns.strict.forward.nxdomain
+
+  Text to send the user if there is no forward match (text).
+
+- lookup_rdns.strict.forward.dnserror
+
+  Text to send the user if there is some other error with the forward
+  lookup (text).
+
+- lookup_rdns.strict.reverse.nxdomain
+
+  Text to send the user if there is no reverse match (text).
+
+- lookup_rdns.strict.reverse.dnserror
+
+  Text to send the user if there is some other error with the reverse
+  lookup (text).
+
+## Configuration lookup_rdns.strict.timeout
+
+This is how we specify to Haraka that our plugin should have a certain timeout.
+If you specify 0 here, then the plugin will never timeout while the connection
+is active. This is also required for this plugin, which needs to handle its
+own timeouts. To actually specify the timeout for this plugin, please see
+the general config in lookup_rdns.strict.ini.
+
+## Configuration lookup_rdns.strict.whitelist
+
+No matter how much you believe in checking that DNS and rDNS match, it is not
+required by RFC, and there will always be some legitimate mail server that
+has great trouble getting their DNS in order. For this reason we are
+providing a whitelist.
+
+This file will match exactly what you put on each line.
+
+## Configuration lookup_rdns.strict.whitelist_regex
+
+Does the same thing as the whitelist file, but each line is a regex.
+Each line is also anchored for you, meaning '^' + regex + '$' is added for
+you. If you need to get around this restriction, you may use a '.\*' at
+either the start or the end of your regex. This should help prevent people
+from writing overly permissive rules on accident.

package/docs/deprecated/mail_from.access.md

@@ -0,0 +1,52 @@
+## DEPRECATION NOTICE
+
+See [haraka-plugin-access](https://github.com/haraka/haraka-plugin-access)
+for upgrade instructions.
+
+# mail_from.access
+
+This plugin will evaluate the address against a set of white and black lists.
+The lists are applied in the following way:
+
+mail_from.access.whitelist (pass)
+mail_from.access.whitelist_regex (pass)
+mail_from.access.blacklist (block)
+mail_from.access.blacklist_regex (block)
+
+## Configuration mail_from.access.ini
+
+General configuration file for this plugin.
+
+- mail_from.access.general.deny_msg
+
+  Text to send the user on reject (text).
+
+## Configuration mail_from.access.whitelist
+
+The whitelist is mostly to counter blacklist entries that match more than
+what one would want. This file should be used for a specific address,
+one per line, that should bypass blacklist checks.
+NOTE: We heavily suggest tailoring blacklist entries to be as accurate as
+possible and never using whitelists. Nevertheless, if you need whitelists,
+here they are.
+
+## Configuration mail_from.access.whitelist_regex
+
+Does the same thing as the whitelist file, but each line is a regex.
+Each line is also anchored for you, meaning '^' + regex + '$' is added for
+you. If you need to get around this restriction, you may use a '.\*' at
+either the start or the end of your regex. This should help prevent people
+from writing overly permissive rules on accident.
+
+## Configuration mail_from.access.blacklist
+
+This file should be used for a specific address, one per line, that should
+fail on connect.
+
+## Configuration mail_from.access.blacklist_regex
+
+Does the same thing as the blacklist file, but each line is a regex.
+Each line is also anchored for you, meaning '^' + regex + '$' is added for
+you. If you need to get around this restriction, you may use a '.\*' at
+either the start or the end of your regex. This should help prevent people
+from writing overly permissive rules on accident.