mailauth 5.0.3 → 7.0.0

package/docs/dmarc.md

@@ -6,6 +6,8 @@ This document describes the result object returned by DMARC verification.
 
 DMARC (Domain-based Message Authentication, Reporting, and Conformance) is verified during the authentication process. The DMARC result depends on both SPF and DKIM verification results.
 
+Policy discovery and Organizational Domains follow the DNS Tree Walk of RFC 9989 section 4.10, not the Public Suffix List. The policy is taken from the record of the author domain, or else from the record of its Organizational Domain, or else from a record published with `psd=y` above it. Relaxed alignment compares the Organizational Domains of the author domain and of each authenticated identifier, which can take further walks. A walk makes at most eight queries, lookups are shared within a message, only identifiers at or below the author's Organizational Domain are walked, and at most ten identifiers per message are walked. An identifier past that limit is treated like one whose walk failed: it gives `temperror` unless another identifier aligns, never a definite `fail`.
+
 ```javascript
 const { authenticate } = require('mailauth');
 
@@ -16,35 +18,70 @@ const { dmarc } = await authenticate(message, {
 });
 ```
 
+The author domain is the domain of the addr-spec in the From header (RFC 9989 5.3.1, RFC 5322 3.4.1). An "@" inside a quoted local-part (`"ceo@x"@bank.example`) or an obsolete source route (`<@relay.example:ceo@bank.example>`) does not change it. It is lower-cased, converted to an A-label, and a trailing dot is ignored. Several From addresses are evaluated when they all have the same domain. When there is no domain, or more than one, or a mailbox has no usable domain, or the message has more than one From header field, DMARC validation is not possible: `dmarc()` returns `false` and `authenticate()` returns `dmarc: false`, as for a message without a From address. The lower level `evaluateDmarc()` from `mailauth/lib/dmarc` also returns `reason` (`"no-author-domain"`, `"multiple-author-domains"`, `"invalid-author-domain"` or `"multiple-from-fields"`) and `authorDomains` in that case. RFC 9989 11.5 warns that such messages are often abusive, so a receiver should handle them by local policy instead of treating them as mail without DMARC.
+
+```javascript
+const { dmarc } = require('mailauth/lib/dmarc');
+
+const result = await dmarc({
+    headerFrom: 'user@example.com', // or a list of From addresses
+    spfDomains: ['example.com'], // the MAIL FROM domain, if SPF passed
+    dkimDomains: [{ domain: 'example.com' }], // signing domains of passing DKIM signatures
+    strict: false, // true for the RFC tag-list rules and no header.d, see below
+    resolver // optional, defaults to dns.promises.resolve
+});
+```
+
+### Strict mode
+
+With `strict: true`, which `authenticate(message, { strict: true })` passes on:
+
+- Records are parsed by the tag-list rules of RFC 6376 3.2 that DMARC uses (RFC 9989 4.7). Tag names are case sensitive, so `P=reject` is an unknown tag (only the version tag may be `V`, which the DMARC ABNF allows), and a record with a duplicated tag is invalid and ignored, as if it was not published. By default tag names are case-folded and the last of duplicated tags wins, and the result lists `"tag-case"` or `"duplicate-tag"` in `warnings`.
+- `status.header.d` is not set and `header.d` is left out of the Authentication-Results entry, because it is not a registered property for `dmarc` (RFC 9989 9.1). `policyDomain` still has the domain.
+
 ## Result Object Fields
 
-| Field       | Type     | Presence     | Description                                                |
-| ----------- | -------- | ------------ | ---------------------------------------------------------- |
-| `status`    | `object` | Always       | Verification status object (see below)                     |
-| `domain`    | `string` | Always       | Organizational domain used for DMARC lookup                |
-| `policy`    | `string` | Record found | Effective policy (`"reject"`, `"quarantine"`, or `"none"`) |
-| `p`         | `string` | Record found | Policy from `p=` tag                                       |
-| `sp`        | `string` | Record found | Subdomain policy from `sp=` tag (defaults to `p` value)    |
-| `pct`       | `number` | Record found | Percentage of messages to apply policy (0-100)             |
-| `rr`        | `string` | Record found | Raw DMARC DNS TXT record                                   |
-| `alignment` | `object` | Record found | SPF and DKIM alignment details (see below)                 |
-| `error`     | `string` | On temperror | Error message                                              |
-| `info`      | `string` | Always       | Formatted Authentication-Results header value              |
+| Field          | Type       | Presence     | Description                                                                      |
+| -------------- | ---------- | ------------ | -------------------------------------------------------------------------------- |
+| `status`       | `object`   | Always       | Verification status object (see below)                                           |
+| `domain`       | `string`   | Always       | Organizational Domain of the author domain (Tree Walk)                           |
+| `policy`       | `string`   | Record found | Effective policy (`"reject"`, `"quarantine"`, or `"none"`), see below            |
+| `p`            | `string`   | Record found | Policy from `p=` tag                                                             |
+| `sp`           | `string`   | Record found | Subdomain policy from `sp=` tag (defaults to `p` value)                          |
+| `np`           | `string`   | Record found | Non-existent subdomain policy from `np=` tag, if published                       |
+| `testMode`     | `boolean`  | Record found | `true` when the record has `t=y`                                                 |
+| `rr`           | `string`   | Record found | Raw DMARC DNS TXT record                                                         |
+| `policyDomain` | `string`   | Record found | The DMARC Policy Domain (RFC 9989 3.2.5), where the applied record was published |
+| `alignment`    | `object`   | Record found | SPF and DKIM alignment details (see below)                                       |
+| `error`        | `string`   | On temperror | Error message                                                                    |
+| `warnings`     | `string[]` | Lax parsing  | What the default mode accepted that `strict` would not (see Strict mode)         |
+| `info`         | `string`   | Always       | Formatted Authentication-Results header value                                    |
+
+On a `temperror` that happens after a record was found, `policy`, `p`, `sp`, `rr` and `alignment` are still set. That is the case when a Tree Walk needed for relaxed alignment failed or was over the limit and no other identifier aligned, or when the existence query that chooses between `sp` and `np` failed and nothing aligned.
+
+`policyDomain` is the domain an RFC 9990 aggregate report lists as the domain of the published policy. By default it is also in `status.header.d`. `domain` is the Organizational Domain, which is a different domain when the policy came from the author domain's own record below it, or from a PSD above it.
+
+`policy` is `p` for the author domain's own record. For a record inherited from the Organizational Domain or a PSD it is `np` when the author domain does not exist (NXDOMAIN, RFC 9989 A.4), otherwise `sp`, both falling back to `p`. The existence query is only made when the record has `np`. When it fails and `np` differs from the other policy, a message that passes still passes, `policy` is the weaker of the two and `policy.dmarc` is left out, and a message that does not pass gets `temperror`. With `t=y` the policy is one level below that, so `reject` becomes `quarantine` and `quarantine` becomes `none`, while `p`, `sp` and `np` stay as published.
+
+A record without a valid `p`, or with an invalid `sp` or `np`, is applied as `p=none` when its `rua` lists a syntactically valid URI (RFC 3986, so `mailto:` counts and a bare address does not), and otherwise gets no DMARC processing (result `none`). `pct` is historic in RFC 9989 and is ignored.
 
 ## status Object
 
-| Field     | Type     | Description                   |
-| --------- | -------- | ----------------------------- |
-| `result`  | `string` | DMARC result code (see below) |
-| `header`  | `object` | Header information            |
-| `comment` | `string` | Policy and ARC information    |
+| Field     | Type     | Description                                             |
+| --------- | -------- | ------------------------------------------------------- |
+| `result`  | `string` | DMARC result code (see below)                           |
+| `header`  | `object` | Header information                                      |
+| `policy`  | `object` | `{ dmarc }`, the evaluated policy, on `pass` and `fail` |
+| `comment` | `string` | Policy and ARC information                              |
+
+`policy.dmarc` (RFC 9989 9.1) is the `policy` value, so after `sp`, `np` and `t=y` were applied. It is only set when the result is `pass` or `fail` and the policy is known.
 
 ### status.header Object
 
-| Field  | Type     | Description                         |
-| ------ | -------- | ----------------------------------- |
-| `from` | `string` | Domain of the From header address   |
-| `d`    | `string` | Domain where DMARC record was found |
+| Field  | Type     | Description                                                                                                     |
+| ------ | -------- | --------------------------------------------------------------------------------------------------------------- |
+| `from` | `string` | Domain of the From header address                                                                               |
+| `d`    | `string` | Domain where the applied DMARC record was found (author, Organizational Domain, or PSD). Not set in strict mode |
 
 ## alignment Object
 
@@ -72,12 +109,12 @@ const { dmarc } = await authenticate(message, {
 
 ## Result Values
 
-| Result      | Description                                           |
-| ----------- | ----------------------------------------------------- |
-| `pass`      | Message passed DMARC (SPF or DKIM aligned and passed) |
-| `fail`      | Message failed DMARC (neither SPF nor DKIM aligned)   |
-| `none`      | No DMARC record found                                 |
-| `temperror` | Temporary error during DNS lookup                     |
+| Result      | Description                                                                                                                               |
+| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
+| `pass`      | Message passed DMARC (SPF or DKIM aligned and passed)                                                                                     |
+| `fail`      | Message failed DMARC (neither SPF nor DKIM aligned)                                                                                       |
+| `none`      | No DMARC record found                                                                                                                     |
+| `temperror` | Temporary error during a DNS lookup that the result depends on, or an identifier that could not be checked because of the Tree Walk limit |
 
 ## Policy Values
 
@@ -109,12 +146,17 @@ For example: `"p=REJECT sp=REJECT arc=pass"`
         "header": {
             "from": "example.com",
             "d": "example.com"
+        },
+        "policy": {
+            "dmarc": "reject"
         }
     },
     "domain": "example.com",
+    "policyDomain": "example.com",
     "policy": "reject",
     "p": "reject",
     "sp": "reject",
+    "testMode": false,
     "rr": "v=DMARC1; p=reject; rua=mailto:dmarc@example.com",
     "alignment": {
         "spf": {
@@ -125,7 +167,7 @@ For example: `"p=REJECT sp=REJECT arc=pass"`
             "strict": false
         }
     },
-    "info": "dmarc=pass (p=REJECT) header.from=example.com header.d=example.com"
+    "info": "dmarc=pass (p=REJECT) policy.dmarc=reject header.from=example.com header.d=example.com"
 }
 ```
 
@@ -139,12 +181,17 @@ For example: `"p=REJECT sp=REJECT arc=pass"`
         "header": {
             "from": "example.com",
             "d": "example.com"
+        },
+        "policy": {
+            "dmarc": "reject"
         }
     },
     "domain": "example.com",
+    "policyDomain": "example.com",
     "policy": "reject",
     "p": "reject",
     "sp": "reject",
+    "testMode": false,
     "rr": "v=DMARC1; p=reject; rua=mailto:dmarc@example.com",
     "alignment": {
         "spf": {
@@ -154,7 +201,7 @@ For example: `"p=REJECT sp=REJECT arc=pass"`
             "strict": false
         }
     },
-    "info": "dmarc=fail (p=REJECT) header.from=example.com header.d=example.com"
+    "info": "dmarc=fail (p=REJECT) policy.dmarc=reject header.from=example.com header.d=example.com"
 }
 ```
 
@@ -168,7 +215,7 @@ For example: `"p=REJECT sp=REJECT arc=pass"`
             "from": "no-dmarc.example.com"
         }
     },
-    "domain": "example.com",
+    "domain": "no-dmarc.example.com",
     "info": "dmarc=none header.from=no-dmarc.example.com"
 }
 ```
@@ -183,9 +230,13 @@ For example: `"p=REJECT sp=REJECT arc=pass"`
         "header": {
             "from": "sub.example.com",
             "d": "example.com"
+        },
+        "policy": {
+            "dmarc": "quarantine"
         }
     },
     "domain": "example.com",
+    "policyDomain": "example.com",
     "policy": "quarantine",
     "p": "none",
     "sp": "quarantine",
@@ -199,7 +250,7 @@ For example: `"p=REJECT sp=REJECT arc=pass"`
             "strict": false
         }
     },
-    "info": "dmarc=pass (p=NONE sp=QUARANTINE) header.from=sub.example.com header.d=example.com"
+    "info": "dmarc=pass (p=NONE sp=QUARANTINE) policy.dmarc=quarantine header.from=sub.example.com header.d=example.com"
 }
 ```
 

package/docs/mta-sts.md

@@ -10,41 +10,53 @@ MTA-STS allows domain owners to declare that their mail servers support TLS and
 const { getPolicy, validateMx } = require('mailauth/lib/mta-sts');
 ```
 
+## Options
+
+`getPolicy`, `fetchPolicy` and `resolvePolicy` accept an options object as their last argument. `parsePolicy` and `validateMx` accept `{ strict }`.
+
+| Option          | Type       | Default                | Description                                                                                |
+| --------------- | ---------- | ---------------------- | ------------------------------------------------------------------------------------------ |
+| `resolver`      | `function` | `dns.promises.resolve` | Async DNS resolver with the `dns.promises.resolve` signature                               |
+| `strict`        | `boolean`  | `false`                | Apply the RFC 8461 rules exactly, see [Strict Mode](#strict-mode)                          |
+| `timeout`       | `number`   | `60000`                | Overall time limit for the HTTPS policy request in milliseconds (the idle timeout is 15 s) |
+| `maxPolicySize` | `number`   | `65536`                | Maximum policy file size in bytes, larger responses fail with the `policy_too_large` error |
+
 ## getPolicy Result
 
-The `getPolicy` function fetches and returns the MTA-STS policy for a domain.
+The `getPolicy` function fetches and returns the MTA-STS policy for a domain. The domain may also be given as an email address.
 
 ```javascript
-const { policy, status } = await getPolicy('example.com', knownPolicy);
+const { policy, status, warnings } = await getPolicy('example.com', knownPolicy, { strict: false });
 ```
 
 ### Result Object Fields
 
-| Field    | Type     | Description                           |
-| -------- | -------- | ------------------------------------- |
-| `policy` | `object` | The MTA-STS policy object (see below) |
-| `status` | `string` | Policy retrieval status (see below)   |
+| Field      | Type       | Description                                                                                       |
+| ---------- | ---------- | ------------------------------------------------------------------------------------------------- |
+| `policy`   | `object`   | The MTA-STS policy object to apply and to store in your cache (see below)                         |
+| `status`   | `string`   | Policy retrieval status (see below)                                                               |
+| `warnings` | `string[]` | Only present when lax mode accepted something strict mode would reject, see [Warnings](#warnings) |
 
 ### Status Values
 
-| Status      | Description                                                |
-| ----------- | ---------------------------------------------------------- |
-| `found`     | New or updated policy was fetched successfully             |
-| `not_found` | No MTA-STS policy exists for the domain                    |
-| `renewed`   | Existing policy is still valid (ID unchanged, not expired) |
-| `errored`   | Policy discovery failed due to an error                    |
+| Status      | Description                                                                                                                                                                          |
+| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `found`     | A new or updated policy was fetched successfully                                                                                                                                     |
+| `not_found` | No MTA-STS policy exists for the domain, and there is no valid cached policy                                                                                                         |
+| `renewed`   | The cached policy has the same ID as the DNS record and has not expired. It is returned unchanged, without fetching the policy file                                                  |
+| `errored`   | No live policy could be discovered or fetched. `policy.error` holds the reason. `policy` is the valid cached policy if there is one, otherwise a policy with mode `none` (see below) |
 
 ### Policy Object Fields
 
-| Field     | Type            | Presence        | Description                                            |
-| --------- | --------------- | --------------- | ------------------------------------------------------ |
-| `id`      | `string\|false` | Always          | Policy ID from DNS TXT record, or `false` if not found |
-| `version` | `string`        | Policy found    | Always `"STSv1"` for valid policies                    |
-| `mode`    | `string`        | Always          | Policy mode (see below)                                |
-| `mx`      | `string[]`      | Mode not "none" | Array of allowed MX hostnames (may include wildcards)  |
-| `maxAge`  | `number`        | Policy found    | Policy validity period in seconds                      |
-| `expires` | `string`        | Policy found    | ISO 8601 expiration timestamp                          |
-| `error`   | `Error`         | On error        | Error object if discovery failed                       |
+| Field     | Type            | Presence         | Description                                                               |
+| --------- | --------------- | ---------------- | ------------------------------------------------------------------------- |
+| `id`      | `string\|false` | Always           | Policy ID from DNS TXT record, or `false` if not found                    |
+| `version` | `string`        | Policy found     | Always `"STSv1"` for valid policies                                       |
+| `mode`    | `string`        | Always           | Policy mode (see below)                                                   |
+| `mx`      | `string[]`      | Listed in policy | Array of allowed MX hostnames (may include wildcards)                     |
+| `maxAge`  | `number`        | Policy found     | Policy validity period in seconds                                         |
+| `expires` | `string`        | Policy found     | ISO 8601 expiration timestamp                                             |
+| `error`   | `Error`         | Status errored   | Error object explaining why no live policy could be discovered or fetched |
 
 ### Mode Values
 
@@ -54,6 +66,57 @@ const { policy, status } = await getPolicy('example.com', knownPolicy);
 | `enforce` | Strictly enforce TLS and MX restrictions                    |
 | `none`    | No policy in effect                                         |
 
+## Caching
+
+Pass the `policy` value of the previous result as `knownPolicy`, and store the `policy` value of every result, whatever the status. `getPolicy` implements the RFC 8461 3.3 and 5.1 cache rules itself:
+
+```javascript
+const knownPolicy = await cache.get(domain); // undefined if not cached
+const { policy } = await getPolicy(domain, knownPolicy);
+await cache.set(domain, policy); // keep the entry at least until policy.expires
+```
+
+- **Valid cached policy, same ID in DNS:** the cached policy is returned as `renewed` without an HTTPS request. Its `expires` value is not extended, so the policy file is fetched again once it expires.
+- **Valid cached policy, no live policy:** if the TXT lookup fails, the TXT record is missing, unusable or duplicated, the policy host has no address, or the new policy can not be fetched or is invalid, the cached policy is returned with `status: "errored"` and the reason in `policy.error`. A valid cached policy is never replaced by `mode: "none"` because of a discovery failure. An exception is a cached policy that already has mode `none`, which may be replaced by the retry placeholder described below.
+- **Expired cached policy:** the policy file is fetched again. If the fetch fails, the default lax mode keeps returning the expired `enforce` or `testing` policy (with the `expired-cache` warning), while strict mode treats the domain as having no policy.
+- **No usable cached policy and the fetch fails:** the result is a placeholder with the new policy ID, mode `none`, and `expires` set one hour ahead. Storing it makes the next calls return it as `renewed` without HTTPS requests, so the policy host is not retried for an hour. A changed policy ID is fetched right away.
+
+## Strict Mode
+
+With `strict: true` the RFC 8461 rules are applied exactly. The default lax mode accepts the following, and reports each acceptance in the `warnings` array of the `getPolicy` result:
+
+| Rule                                                                                                                                 | Warning         |
+| ------------------------------------------------------------------------------------------------------------------------------------ | --------------- |
+| The TXT record must match the RFC 8461 3.1 syntax: case-sensitive `v=STSv1` at the start, `id` of 1 to 32 letters and digits         | `txt-syntax`    |
+| Only HTTP status 200 is accepted (lax accepts any 2xx)                                                                               | `http-status`   |
+| The policy must be served with the `text/plain` media type                                                                           | `content-type`  |
+| Policy field names and the `mode` value are case-sensitive, `max_age` is 1 to 10 digits, `mx` values are `["*."] Domain`             | `policy-syntax` |
+| The Policy Host certificate must match through a DNS-ID subjectAltName, with no CN fallback and no partial-label wildcards (`mta-*`) | `cert-identity` |
+| An expired cached policy is not applied when the policy can not be fetched                                                           | `expired-cache` |
+
+A strict TXT record syntax error makes the record unusable, so it is handled like a missing record. The other strict failures are fetch errors, so a valid cached policy is kept.
+
+These rules apply in both modes:
+
+- Only records that begin with `v=STSv1;` are counted when checking for multiple TXT records. Two or more remaining records fail with `multi_sts_records`.
+- If a TXT or policy field other than `mx` is repeated, the first value is used.
+- TXT extension fields may contain any value allowed by RFC 8461, including `:`, `(` and quotes.
+- A policy without a `mode` field is invalid.
+- Redirects are not followed, and only 2xx responses (200 in strict mode) are accepted.
+- The policy file size is limited by `maxPolicySize` and the whole request by `timeout`.
+- Internationalized domain names are converted to A-labels for DNS queries, TLS SNI, the Host header and MX matching.
+
+## Warnings
+
+| Warning         | Meaning                                                                       |
+| --------------- | ----------------------------------------------------------------------------- |
+| `txt-syntax`    | The TXT record does not match the RFC 8461 3.1 syntax                         |
+| `http-status`   | The policy was served with a 2xx status other than 200                        |
+| `content-type`  | The policy was not served as `text/plain`                                     |
+| `policy-syntax` | The policy file does not match the RFC 8461 3.2 syntax                        |
+| `cert-identity` | The Policy Host certificate matched only through the CN or a partial wildcard |
+| `expired-cache` | An expired cached policy was returned because the policy could not be fetched |
+
 ## validateMx Result
 
 The `validateMx` function checks if an MX hostname is valid according to the MTA-STS policy.
@@ -62,14 +125,16 @@ The `validateMx` function checks if an MX hostname is valid according to the MTA
 const result = validateMx('alt1.mx.example.com', policy);
 ```
 
+MX names are matched case-insensitively, a trailing dot is ignored, and U-label names are converted to A-labels. A wildcard pattern such as `*.example.com` matches exactly one complete left-most label: `mail.example.com` matches, but `example.com` and `foo.bar.example.com` do not. An `enforce` or `testing` policy without an `mx` list matches nothing. `validateMx` does not look at `policy.expires`; use `getPolicy` to refresh the policy.
+
 ### Result Object Fields
 
-| Field     | Type      | Presence    | Description                                           |
-| --------- | --------- | ----------- | ----------------------------------------------------- |
-| `valid`   | `boolean` | Always      | Whether the MX hostname is allowed                    |
-| `mode`    | `string`  | Always      | Policy mode (`"testing"`, `"enforce"`, or `"none"`)   |
-| `match`   | `string`  | Valid match | The pattern that matched (exact hostname or wildcard) |
-| `testing` | `boolean` | Always      | Whether policy is in testing mode                     |
+| Field     | Type      | Presence    | Description                                                                           |
+| --------- | --------- | ----------- | ------------------------------------------------------------------------------------- |
+| `valid`   | `boolean` | Always      | Whether the MX hostname is allowed                                                    |
+| `mode`    | `string`  | Always      | Policy mode (`"testing"`, `"enforce"`, or `"none"`)                                   |
+| `match`   | `string`  | Valid match | The matching hostname, or for a wildcard the suffix without `*` (e.g. `.example.com`) |
+| `testing` | `boolean` | Always      | Whether policy is in testing mode                                                     |
 
 ## Example Output
 
@@ -101,7 +166,9 @@ const result = validateMx('alt1.mx.example.com', policy);
 }
 ```
 
-### Policy Renewed (Cached)
+### Policy Renewed
+
+The cached policy is returned unchanged.
 
 ```json
 {
@@ -117,7 +184,9 @@ const result = validateMx('alt1.mx.example.com', policy);
 }
 ```
 
-### Policy Error
+### Policy Error Without a Cached Policy
+
+The policy ID is kept, and `expires` is one hour ahead, so the policy host is not retried before that.
 
 ```json
 {
@@ -134,6 +203,26 @@ const result = validateMx('alt1.mx.example.com', policy);
 }
 ```
 
+### Policy Error With a Valid Cached Policy
+
+```json
+{
+    "policy": {
+        "id": "20240115T120000",
+        "version": "STSv1",
+        "mode": "enforce",
+        "mx": ["mx1.example.com", "mx2.example.com"],
+        "maxAge": 86400,
+        "expires": "2024-01-16T12:00:00.000Z",
+        "error": {
+            "message": "No usable MTA-STS TXT record found for example.com",
+            "code": "sts_record_not_found"
+        }
+    },
+    "status": "errored"
+}
+```
+
 ### MX Validation - Valid Match
 
 ```json
@@ -191,25 +280,34 @@ const result = validateMx('alt1.mx.example.com', policy);
 
 Errors that may appear in `policy.error`:
 
-| Code                  | Description                                          |
-| --------------------- | ---------------------------------------------------- |
-| `multi_sts_records`   | Multiple TXT records found for `_mta-sts.{domain}`   |
-| `invalid_sts_version` | Policy file has invalid or missing version field     |
-| `invalid_sts_mode`    | Policy file has invalid mode value                   |
-| `invalid_sts_max_age` | Policy file has invalid max_age value                |
-| `invalid_sts_mx`      | Policy file missing mx field in enforce/testing mode |
-| `HTTP_SOCKET_TIMEOUT` | HTTP request timed out                               |
-| `http_status_{code}`  | HTTP request returned non-2xx status                 |
-| `ENOTFOUND`           | DNS lookup failed (domain not found)                 |
-| `ENODATA`             | DNS lookup returned no data                          |
+| Code                           | Description                                                                                                    |
+| ------------------------------ | -------------------------------------------------------------------------------------------------------------- |
+| `multi_sts_records`            | More than one TXT record beginning with `v=STSv1;` found for `_mta-sts.{domain}`                               |
+| `sts_record_not_found`         | No MTA-STS TXT record (only reported when a valid cached policy is returned)                                   |
+| `invalid_sts_record`           | The TXT record has no usable `id`, or a syntax error in strict mode (only reported with a valid cached policy) |
+| `policy_host_not_found`        | `mta-sts.{domain}` has no A or AAAA record (only reported with a valid cached policy)                          |
+| `invalid_sts_version`          | Policy file has invalid or missing version field                                                               |
+| `invalid_sts_mode`             | Policy file has invalid or missing mode field                                                                  |
+| `invalid_sts_max_age`          | Policy file has invalid max_age value                                                                          |
+| `invalid_sts_mx`               | Policy file missing mx field in enforce/testing mode, or invalid mx value (strict)                             |
+| `invalid_content_type`         | Policy was not served as `text/plain` (strict mode only)                                                       |
+| `policy_too_large`             | Policy file is larger than `maxPolicySize`                                                                     |
+| `HTTP_SOCKET_TIMEOUT`          | The HTTPS connection was idle for too long                                                                     |
+| `HTTP_REQUEST_TIMEOUT`         | The HTTPS request took longer than `timeout`                                                                   |
+| `http_incomplete_response`     | The HTTPS response ended before the complete body was received                                                 |
+| `http_status_{code}`           | HTTP request returned a non-2xx status (any status other than 200 in strict mode)                              |
+| `ERR_TLS_CERT_ALTNAME_INVALID` | The Policy Host certificate is not valid for `mta-sts.{domain}`                                                |
+| `ESERVFAIL`, `ETIMEOUT`, ...   | DNS lookup failed                                                                                              |
 
 ## Usage Example
 
 ```javascript
 const { getPolicy, validateMx } = require('mailauth/lib/mta-sts');
 
-// Fetch policy
-const { policy, status } = await getPolicy('gmail.com');
+// Fetch policy, using and updating the cached policy
+const knownPolicy = await cache.get('gmail.com');
+const { policy, status } = await getPolicy('gmail.com', knownPolicy);
+await cache.set('gmail.com', policy);
 
 console.log(`Policy status: ${status}`);
 console.log(`Policy mode: ${policy.mode}`);

package/docs/spf.md

@@ -17,6 +17,22 @@ const result = await spf({
 });
 ```
 
+## Options
+
+| Option            | Type       | Default           | Description                                                                                                    |
+| ----------------- | ---------- | ----------------- | -------------------------------------------------------------------------------------------------------------- |
+| `sender`          | `string`   | `postmaster@helo` | MAIL FROM address. An empty value means a null reverse-path, so the HELO identity is checked                   |
+| `ip`              | `string`   | Required          | SMTP client IP address. IPv4-mapped IPv6 addresses (`::ffff:192.0.2.1`, `::ffff:c000:201`) are checked as IPv4 |
+| `helo`            | `string`   |                   | EHLO/HELO hostname                                                                                             |
+| `mta`             | `string`   | `os.hostname()`   | Hostname of the MTA performing the check                                                                       |
+| `resolver`        | `function` | `dns.resolve`     | Custom DNS resolver                                                                                            |
+| `maxResolveCount` | `number`   | `10`              | Maximum DNS lookups allowed                                                                                    |
+| `maxVoidCount`    | `number`   | `2`               | Maximum void (empty) DNS lookups allowed                                                                       |
+| `maxElapsedTime`  | `number`   | No limit          | Maximum time in milliseconds for the whole evaluation. If exceeded, the result is `temperror` (RFC 7208 4.6.4) |
+| `strict`          | `boolean`  | `false`           | Follow RFC 7208 exactly instead of the lenient default, see [Strict Mode](#strict-mode)                        |
+
+A missing or invalid `ip` gives a `temperror` result with the comment `missing or invalid client IP address`.
+
 ## Result Object Fields
 
 | Field           | Type     | Presence     | Description                                                 |
@@ -30,6 +46,8 @@ const result = await spf({
 | `info`          | `string` | Always       | Formatted Authentication-Results header value               |
 | `rr`            | `string` | Record found | Raw SPF DNS TXT record                                      |
 | `lookups`       | `object` | Always       | DNS lookup statistics (see below)                           |
+| `explanation`   | `string` | Fail only    | Explanation string from the `exp` modifier (see below)      |
+| `warnings`      | `array`  | Default mode | Leniencies that strict mode would have rejected (see below) |
 
 ## status Object
 
@@ -46,6 +64,26 @@ const result = await spf({
 | `mailfrom` | `string` | The MAIL FROM address  |
 | `helo`     | `string` | The HELO/EHLO hostname |
 
+In strict mode `status.smtp` holds only the identity that was checked (RFC 8601 section 2.7.2): `mailfrom` for a MAIL FROM check, or `helo` for a null reverse-path. The `mailfrom` value is then only the domain, unless the evaluated policy uses the `%{l}` or `%{s}` macro.
+
+## explanation
+
+When the result is `fail` because of a mechanism match and the record has an `exp` modifier, the explanation string is fetched and expanded as described in RFC 7208 section 6.2. Any problem with it (DNS error, no or several TXT records, syntax error, non-ASCII text) means that there is no explanation. The explanation lookup does not count toward the DNS lookup limits. The explanation of an included record is never used, and after a redirect only the explanation of the redirect target is used.
+
+The explanation is text from the domain owner. It is not added to the generated headers. If you show it to an SMTP client, make clear that it comes from a third party, for example by prepending `"<domain> explains: "`.
+
+## warnings
+
+In the default mode some input is accepted that strict mode rejects. When that happens, the result gets a `warnings` array. It is never rendered into the headers. The values are:
+
+| Warning                | Meaning                                                                                                                                                                   |
+| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `syntax-error`         | An evaluated record has a syntax error (for example after the matching term, a `c`, `r` or `t` macro outside of explanation text, or a duplicate `exp`), or leading space |
+| `void-lookup-limit`    | The void lookup limit would have been exceeded if single-stack hosts were counted as void lookups, as RFC 7208 does                                                       |
+| `non-ascii-local-part` | A `%{l}` or `%{s}` term was evaluated with a non-ASCII local-part (RFC 8616 section 4 says such terms do not match)                                                       |
+
+`warnings` is not set for a `permerror` result.
+
 ## lookups Object
 
 | Field        | Type     | Description                                           |
@@ -81,6 +119,33 @@ The `status.comment` field follows this format based on the result:
 | `permerror` | `"{mta}: permanent error in processing during lookup of {sender}: {text}"`              |
 | `temperror` | `"{mta}: error in processing during lookup of {sender}: {text}"`                        |
 
+## Received-SPF Header
+
+The `header` field holds a `Received-SPF` header (RFC 7208 section 9.1) with the `client-ip`, `envelope-from` and `helo` keys. `envelope-from` is omitted for a null reverse-path, and `client-ip` if no valid IP was given. Values that are not a dot-atom, such as IPv6 addresses and email addresses, are quoted.
+
+## Strict Mode
+
+By default the SPF check is lenient where the leniency does not change which domain is authenticated. With `strict: true` it follows RFC 7208 exactly:
+
+- The whole record is validated before evaluation (RFC 7208 section 4.6). A syntax error anywhere in the record gives `permerror`, even after the matching term. By default records are evaluated lazily, so an error after the matching term is not noticed.
+- The `c`, `r` and `t` macros are only allowed in explanation text, a macro number must be nonzero, and transformers must follow the ABNF (`%{d0}`, `%{d2x}` and `%{dr2}` are errors).
+- `exp` may appear only once, and modifier names must follow the `name` rule.
+- `ip4` and `ip6` networks and prefix lengths are checked against the ABNF (`ip4:2001:db8::1`, `ip6:192.0.2.1`, `ip6:2001:db8::/32//64` and `ip4:192.0.2.0/00` are errors).
+- A record must start with exactly `v=spf1` followed by a space or the end of the record. Records with leading whitespace are ignored.
+- A void lookup is counted for the query of the client's address type alone. By default an `a` or `mx` host that has only addresses of the other type is not counted as void.
+- A term that uses `%{l}` or `%{s}` does not match if the local-part is not ASCII (RFC 8616 section 4).
+- A target name that is not a valid domain name after macro expansion makes the `a`, `mx`, `exists` or `ptr` mechanism not match. By default it is a `permerror`. For `include` and `redirect` it is a `permerror` in both modes.
+- Authentication-Results reports only the checked identity, see [status.smtp Object](#statussmtp-object).
+
+These rules apply in both modes:
+
+- A DNS error (other than NXDOMAIN or an empty answer) in any lookup, including the targets of `include` and `redirect`, gives `temperror`.
+- Every `ptr` mechanism and every `%{p}` expansion counts toward the DNS lookup limit. A DNS error in the PTR lookup makes the `ptr` mechanism not match.
+- `%{p}` expands to a validated PTR name or `unknown`.
+- Trailing dots in domain-specs are accepted, names longer than 253 characters are truncated from the left, and uppercase macros are URL escaped.
+- An internationalized MAIL FROM domain is converted to A-labels.
+- Address lookups of MX hosts are not counted as void lookups.
+
 ## Example Output
 
 ### SPF Pass
@@ -99,7 +164,7 @@ The `status.comment` field follows this format based on the result:
             "helo": "mail.example.com"
         }
     },
-    "header": "Received-SPF: pass (mx.receiver.com: domain of user@example.com designates 192.0.2.1 as permitted sender) client-ip=192.0.2.1;",
+    "header": "Received-SPF: pass (mx.receiver.com: domain of user@example.com designates 192.0.2.1 as permitted sender) client-ip=192.0.2.1;\r\n envelope-from=\"user@example.com\"; helo=mail.example.com;",
     "info": "spf=pass (mx.receiver.com: domain of user@example.com designates 192.0.2.1 as permitted sender) smtp.mailfrom=user@example.com smtp.helo=mail.example.com",
     "rr": "v=spf1 ip4:192.0.2.0/24 -all",
     "lookups": {
@@ -127,7 +192,7 @@ The `status.comment` field follows this format based on the result:
             "helo": "attacker.example.net"
         }
     },
-    "header": "Received-SPF: fail (mx.receiver.com: domain of user@example.com does not designate 203.0.113.1 as permitted sender) client-ip=203.0.113.1;",
+    "header": "Received-SPF: fail (mx.receiver.com: domain of user@example.com does not designate 203.0.113.1 as permitted sender) client-ip=203.0.113.1;\r\n envelope-from=\"user@example.com\"; helo=attacker.example.net;",
     "info": "spf=fail (mx.receiver.com: domain of user@example.com does not designate 203.0.113.1 as permitted sender) smtp.mailfrom=user@example.com smtp.helo=attacker.example.net",
     "rr": "v=spf1 ip4:192.0.2.0/24 -all",
     "lookups": {
@@ -155,7 +220,7 @@ The `status.comment` field follows this format based on the result:
             "helo": "mail.no-spf.example.com"
         }
     },
-    "header": "Received-SPF: none (mx.receiver.com: no-spf.example.com does not designate permitted sender hosts) client-ip=192.0.2.1;",
+    "header": "Received-SPF: none (mx.receiver.com: no-spf.example.com does not designate permitted sender hosts) client-ip=192.0.2.1;\r\n envelope-from=\"user@no-spf.example.com\"; helo=mail.no-spf.example.com;",
     "info": "spf=none (mx.receiver.com: no-spf.example.com does not designate permitted sender hosts) smtp.mailfrom=user@no-spf.example.com smtp.helo=mail.no-spf.example.com",
     "lookups": {
         "limit": 10,
@@ -180,7 +245,7 @@ The `status.comment` field follows this format based on the result:
             "mailfrom": "user@complex.example.com"
         }
     },
-    "header": "Received-SPF: permerror (mx.receiver.com: permanent error in processing during lookup of user@complex.example.com: Too many DNS requests) client-ip=192.0.2.1;",
+    "header": "Received-SPF: permerror (mx.receiver.com: permanent error in processing during lookup of user@complex.example.com: Too many DNS requests) client-ip=192.0.2.1;\r\n envelope-from=\"user@complex.example.com\";",
     "info": "spf=permerror (mx.receiver.com: permanent error in processing during lookup of user@complex.example.com: Too many DNS requests) smtp.mailfrom=user@complex.example.com",
     "lookups": {
         "limit": 10,