backend-manager 5.6.1 → 5.6.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +19 -0
- package/CLAUDE.md +1 -1
- package/README.md +2 -2
- package/docs/consent.md +29 -4
- package/docs/email-system.md +2 -2
- package/package.json +1 -1
- package/src/manager/libraries/email/data/disposable-domains.json +6 -0
- package/src/manager/libraries/email/marketing/consent-gate.test.js +249 -0
- package/src/manager/libraries/email/marketing/index.js +67 -2
- package/src/manager/libraries/email/validation-provider-neverbounce.js +28 -9
- package/src/manager/libraries/email/validation.js +1 -1
- package/src/manager/libraries/email/validation.test.js +31 -1
- package/src/manager/routes/marketing/contact/delete.js +58 -0
- package/src/manager/routes/marketing/email-preferences/post.js +32 -3
- package/test/email/validation.js +103 -11
- package/test/routes/marketing/webhook.js +70 -5
package/CHANGELOG.md
CHANGED
|
@@ -14,6 +14,25 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
|
|
14
14
|
- `Fixed` for any bug fixes.
|
|
15
15
|
- `Security` in case of vulnerabilities.
|
|
16
16
|
|
|
17
|
+
# [5.6.3] - 2026-06-11
|
|
18
|
+
|
|
19
|
+
### Fixed
|
|
20
|
+
- **NeverBounce single-check result parsing — every signup mailbox check has failed since v5.5.1.** `validation-provider-neverbounce.js` compared `data.result` against numeric codes (`data.result === 0 || 3 || 4`), but the NeverBounce v4 single/check API returns string textcodes (`"valid"`, `"invalid"`, ...) — so every completed check returned `{ valid: false, status: 'unknown' }`, including deliverable mailboxes, and the signup route silently skipped marketing sync for ALL signups. Production-confirmed via GCF logs: somiibo June 7 = 30 synced / 0 failed; June 9–11 = 0 synced / 93 failed; first failure 25 minutes after the v5.5.1 deploy (which switched signup to `ALL_CHECKS` and exposed the latent v5.5.0 bug). New `parseResult()` handles textcodes (canonical), tolerates numeric codes, and normalizes `catch-all` → `catchall`; allowed results stay valid/catchall/unknown. 11 regression parse cases added to `validation.test.js` (suite now 69 cases); fix live-validated against the real API. ZeroBounce provider audited — already string-based, no change.
|
|
21
|
+
- **Library-level marketing consent gate.** `Marketing.sync()`/`Marketing.add()` (`src/manager/libraries/email/marketing/index.js`) now skip users whose `consent.marketing.status` is the literal string `'revoked'` and return `{ blocked: 'consent', email }` (mirroring the `{ blocked: 'validation', ... }` shape) — BEFORE validation and provider calls. Previously the gate existed only at SOME call sites, so the payments on-write sync (`events/firestore/payments-webhooks/on-write.js`) and the admin PUT re-sync re-added users who had unsubscribed. `sync()` gates after the user doc is resolved (doc or uid); `add()` looks up the user by email first (same `auth.email` query as the webhook processors) — no user doc or lookup failure proceeds (fail open). ONLY `'revoked'` blocks: missing/null consent proceeds, so legacy users without a `consent` field keep syncing. `remove()` stays ungated. Covers all callers including the legacy API-command twins (`add-marketing-contact.js`). Gate logic unit-tested in `src/manager/libraries/email/marketing/consent-gate.test.js` (28 plain-node cases, no emulator).
|
|
22
|
+
- **Anonymous HMAC unsubscribe now removes the contact from Beehiiv too** (`routes/marketing/email-preferences/post.js`). The email-footer one-click unsubscribe only suppressed the SendGrid ASM group, leaving the contact live on Beehiiv — a compliance bug. The route now also calls `mailer.remove(email)` (best-effort, after the ASM call succeeds).
|
|
23
|
+
- **Anonymous HMAC re-subscribe now actually re-adds the contact.** Previously it only lifted the ASM suppression and wrote consent `granted` — the contact was never re-added to providers. `mirrorAnonymousToUserDoc` now returns the matched uid (or null); the route calls `mailer.sync(uid)` for matched users (the mirror writes `granted` first, so the library consent gate passes — no bypass flags) or `mailer.add({ email, source: 'resubscribe' })` for pure newsletter contacts. Best-effort, same testing guard as the ASM call.
|
|
24
|
+
- **Admin `DELETE /marketing/contact` now sticks** (`routes/marketing/contact/delete.js`). The route removed the contact from providers but left `consent.marketing.status = 'granted'`, so any later sync re-added the contact. After the provider removal it now mirrors `consent.marketing.status = 'revoked'` (`revokedAt` with `source: 'admin'`, same write shape as the webhook processors) to the matching user doc — best-effort, silent when the email maps to no user.
|
|
25
|
+
- **Stale docs/comments corrected in the same change set:** `docs/consent.md` now describes the shipped library gate, cross-provider HMAC unsubscribe, re-subscribe re-add, and admin-DELETE revoke mirror (new capture point 5); `validation.js` header no longer claims signup uses "disposable check only" (it runs `ALL_CHECKS` before marketing sync); `marketing/index.js` header no longer attributes the signup sync to the "Auth on-create handler" (it's the `/user/signup` route).
|
|
26
|
+
|
|
27
|
+
# [5.6.2] - 2026-06-11
|
|
28
|
+
|
|
29
|
+
### Fixed
|
|
30
|
+
- **4 stale framework tests aligned with shipped v5.5.4–v5.5.6 validation behavior.** The default `npx mgr test` run failed 4 tests whose expectations predated intentional source changes: `test/email/validation.js` still expected all-numeric (`123456@`) and short letter+number (`a123@`) local parts to be blocked (both patterns were removed in v5.5.6 after NeverBounce confirmed real users — QQ emails, real Gmail accounts — were being blocked; tests renamed `localpart-all-numeric-allowed` / `localpart-letter-plus-numbers-allowed`) and expected the pre-v5.5.5 `DEFAULT_CHECKS`/`ALL_CHECKS` lists (now include `typo`, and `dns` in `ALL_CHECKS`); `test/routes/marketing/webhook.js`'s bounce test sent no `bounce_classification`, which v5.5.4 deliberately skips (renamed `sendgrid-hard-bounce-event-handled`, now sends `'Invalid Address'`).
|
|
31
|
+
|
|
32
|
+
### Added
|
|
33
|
+
- **Suite coverage for the v5.5.5 `typo` + `dns` email validation checks** (previously only covered by the standalone `validation.test.js` script): typo-domain blocking (`gamil.com`, `gmail.con`) + correct-domain pass-through, dns-not-in-default-checks, an offline-safe dns positive (network errors skip, never block), and an extended-gated (`TEST_EXTENDED_MODE`) dns negative for nonexistent domains.
|
|
34
|
+
- **Suite coverage for the v5.5.4 bounce-classification filter**: `dropped` + `'Invalid Address'` revokes, technical bounce (`'Technical Failure'`) skipped, and unclassified bounce skipped — locking in that sender-side bounces never revoke recipient consent.
|
|
35
|
+
|
|
17
36
|
# [5.6.1] - 2026-06-11
|
|
18
37
|
|
|
19
38
|
### Added
|
package/CLAUDE.md
CHANGED
|
@@ -156,7 +156,7 @@ Deep references live in `docs/`. **Whenever you make a behavioral change, update
|
|
|
156
156
|
- [docs/admin-post-route.md](docs/admin-post-route.md) — `POST/PUT /admin/post` blog creation via GitHub (image extraction + resize at ingest + `@post/` rewriting). Also the publish target for the Ghostii article engine (`libraries/content/ghostii.js`).
|
|
157
157
|
- [docs/payment-system.md](docs/payment-system.md) — full payment pipeline: Intent → Webhook → On-Write → Transition; subscription model, statuses, `resolveSubscription()`, transition handlers, processor interface, product config, test processor
|
|
158
158
|
- [docs/marketing-campaigns.md](docs/marketing-campaigns.md) — campaign CRUD routes, recurring campaigns, generator pipeline (newsletter), newsletter-driven blog article (`content.article.enabled`), template-owned schemas, asset hosting, seed campaigns
|
|
159
|
-
- [docs/consent.md](docs/consent.md) — marketing consent capture: canonical `consent.{legal,marketing}` user-doc shape, signup-form capture, account-page toggle, HMAC unsub link, SendGrid+Beehiiv webhook receivers, parent forwarder (`/marketing/webhook/forward`), migration script template
|
|
159
|
+
- [docs/consent.md](docs/consent.md) — marketing consent capture: canonical `consent.{legal,marketing}` user-doc shape, signup-form capture, account-page toggle, HMAC unsub link (cross-provider unsub + re-add on resubscribe), admin contact-DELETE revoke mirror, SendGrid+Beehiiv webhook receivers, parent forwarder (`/marketing/webhook/forward`), library-level consent gate in `email.add()`/`email.sync()` (revoked-only skip), migration script template
|
|
160
160
|
- [docs/mcp.md](docs/mcp.md) — Model Context Protocol server: 19 tools, stdio + HTTP transports, OAuth, Claude Chat/Code configuration
|
|
161
161
|
|
|
162
162
|
### Subsystems & Libraries
|
package/README.md
CHANGED
|
@@ -451,10 +451,10 @@ GDPR/CASL-compliant consent capture and cross-provider unsubscribe sync.
|
|
|
451
451
|
- **Canonical user-doc shape** — `consent.{legal,marketing}.{status, grantedAt, revokedAt}` with full audit metadata (timestamp, source, IP, exact label text)
|
|
452
452
|
- **Server-authoritative timestamps** — client timestamps ignored, defending against clock manipulation
|
|
453
453
|
- **Account-page toggle** — `/account` notifications section lets logged-in users opt in/out, hits both SendGrid + Beehiiv
|
|
454
|
-
- **HMAC unsubscribe links** — email-footer one-click flow continues to work
|
|
454
|
+
- **HMAC unsubscribe links** — email-footer one-click flow continues to work; unsubscribe removes the contact from ALL providers (not just the SendGrid ASM group), re-subscribe re-adds the contact
|
|
455
455
|
- **Provider webhook receivers** — `POST /marketing/webhook?provider=sendgrid|beehiiv&key=X` catches unsubscribe / spam / bounce events from SendGrid and Beehiiv, writes the user doc + syncs to the OTHER provider
|
|
456
456
|
- **Parent forwarder** — single public webhook endpoint (`/marketing/webhook/forward`) on the parent BEM fans out to every brand's child BEM so each one updates its own Firestore
|
|
457
|
-
- **
|
|
457
|
+
- **Library-level consent gate** — `email.add()` and `email.sync()` skip users whose `consent.marketing.status === 'revoked'` (covers every call site: payment syncs, admin re-syncs, newsletter form); admin contact DELETE mirrors `revoked` back to the user doc so removals stick
|
|
458
458
|
|
|
459
459
|
See [docs/consent.md](docs/consent.md) for the full architecture, source enum reference, migration script template, and provider configuration steps.
|
|
460
460
|
|
package/docs/consent.md
CHANGED
|
@@ -9,7 +9,7 @@ This doc covers the **server-side** (BEM) part of the system. The matching front
|
|
|
9
9
|
1. **Capture explicit, affirmative consent** at signup with separate checkboxes for legal terms (required) and marketing communications (optional). Store the exact label text the user agreed to.
|
|
10
10
|
2. **Let users withdraw consent at any time** via the account-page toggle or the email-footer unsubscribe link.
|
|
11
11
|
3. **Stay in sync with provider-side actions** — when a user clicks unsubscribe in a SendGrid or Beehiiv email, the user doc updates AND the OTHER provider is also notified.
|
|
12
|
-
4. **Never re-add an unsubscribed user** — `email.add()` and `email.sync()` short-circuit when `consent.marketing.status === 'revoked'
|
|
12
|
+
4. **Never re-add an unsubscribed user** — `email.add()` and `email.sync()` short-circuit in the library itself when `consent.marketing.status === 'revoked'`, so every call site (payment-event syncs, admin re-syncs, public newsletter form, legacy API commands) is covered. See "Email library consent gate" below.
|
|
13
13
|
|
|
14
14
|
## Canonical user-doc shape
|
|
15
15
|
|
|
@@ -51,7 +51,7 @@ consent: {
|
|
|
51
51
|
|
|
52
52
|
## Capture points
|
|
53
53
|
|
|
54
|
-
There are
|
|
54
|
+
There are five places where consent gets recorded or updated. All five converge on the same canonical shape.
|
|
55
55
|
|
|
56
56
|
### 1. Signup form (Phase B)
|
|
57
57
|
|
|
@@ -110,6 +110,8 @@ Body: { email, asmId, sig, action: 'subscribe' | 'unsubscribe' }
|
|
|
110
110
|
- `sig = HMAC-SHA256(email, UNSUBSCRIBE_HMAC_KEY)` — proves we generated the link.
|
|
111
111
|
- IP-rate-limited (5/day per IP).
|
|
112
112
|
- **Also writes the user doc** if the email maps to a user — `consent.marketing.{status, revokedAt}` with `source: 'sendgrid'` (since HMAC links only appear in SendGrid email footers).
|
|
113
|
+
- **Unsubscribe is cross-provider** — besides the SendGrid ASM suppression, the route calls `mailer.remove(email)` (best-effort) so the contact is removed from Beehiiv too. ASM suppression alone only stops SendGrid sends.
|
|
114
|
+
- **Subscribe actually re-adds the contact** — the user-doc mirror writes `granted` BEFORE the provider calls, then the route calls `mailer.sync(uid)` when the email maps to a user (the fresh grant passes the library consent gate — no bypass flags) or `mailer.add({ email, source: 'resubscribe' })` when it doesn't. Best-effort, same testing guard as the ASM call.
|
|
113
115
|
- Backward-compatible — old in-flight email links continue to work.
|
|
114
116
|
|
|
115
117
|
### 4. Provider webhooks (Phase E)
|
|
@@ -145,6 +147,17 @@ Each processor's `handleEvent` does the same shape of work:
|
|
|
145
147
|
|
|
146
148
|
**Beehiiv publication filter.** Each Beehiiv event includes a `publication_id`. The processor compares this against `beehiivProvider.getPublicationId()`, which reads `Manager.config.marketing.newsletter.publicationId` (populated at brand-onboarding time by OMEGA's `beehiiv/ensure/publication.js`). Mismatch → silent skip. This is how shared-publication events (e.g. devbeans shared by 6 brands) get routed correctly — each brand processes only events matching its own publication. Brands without `publicationId` in config silently skip all Beehiiv webhook events. The same convention applies to SendGrid: `marketing.campaigns.listId` is populated by OMEGA's `sendgrid/ensure/list.js`.
|
|
147
149
|
|
|
150
|
+
### 5. Admin contact removal
|
|
151
|
+
|
|
152
|
+
[src/manager/routes/marketing/contact/delete.js](../src/manager/routes/marketing/contact/delete.js) — admin-only endpoint.
|
|
153
|
+
|
|
154
|
+
```
|
|
155
|
+
DELETE /backend-manager/marketing/contact
|
|
156
|
+
Body: { email }
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
After removing the contact from all providers, the route mirrors `consent.marketing.status = 'revoked'` to the user doc when the email maps to a user — `revokedAt` from server time with `source: 'admin'`, same write shape as the webhook processors (`grantedAt` preserved as audit history). Without this mirror, the next `sync()` (payment event, admin re-sync) would re-add the contact the admin just removed. Best-effort and silent when no user matches.
|
|
160
|
+
|
|
148
161
|
## Parent forwarder (Phase E)
|
|
149
162
|
|
|
150
163
|
[src/manager/routes/marketing/webhook/forward/post.js](../src/manager/routes/marketing/webhook/forward/post.js)
|
|
@@ -187,11 +200,22 @@ The parent BEM has its own brand (e.g. `itw-creative-works`) with its own users.
|
|
|
187
200
|
4. The 6 brands sharing the devbeans publication: `getPublicationId()` matches, they each look up the user, only the brand(s) with the user write the doc and call `mailer.remove`.
|
|
188
201
|
5. The brands with dedicated publications: `getPublicationId()` mismatch, silent skip.
|
|
189
202
|
|
|
190
|
-
## Email library
|
|
203
|
+
## Email library consent gate
|
|
191
204
|
|
|
192
205
|
[src/manager/libraries/email/marketing/index.js](../src/manager/libraries/email/marketing/index.js)
|
|
193
206
|
|
|
194
|
-
`email.add()` and `email.sync()` check the user's `consent.marketing.status` before
|
|
207
|
+
`email.add()` and `email.sync()` check the user's `consent.marketing.status` IN THE LIBRARY, before validation and provider calls. Because the gate lives in the library rather than at call sites, every caller is covered: the signup route, the email-preferences toggle, payment-event syncs ([events/firestore/payments-webhooks/on-write.js](../src/manager/events/firestore/payments-webhooks/on-write.js)), the admin PUT re-sync (`routes/marketing/contact/put.js`), the public newsletter form (`routes/marketing/contact/post.js`), and the legacy API-command twins (`functions/core/actions/api/general/add-marketing-contact.js`).
|
|
208
|
+
|
|
209
|
+
**Semantics:**
|
|
210
|
+
|
|
211
|
+
- **Revoked-only skip.** ONLY the literal string `'revoked'` blocks. Missing consent, `null`, or any other value proceeds — legacy users have no `consent` field and must keep syncing.
|
|
212
|
+
- **`sync()`** — after the user doc is resolved (whether passed as a doc or fetched by uid), a revoked doc logs a warn and returns `{ blocked: 'consent', email }` (mirrors the `{ blocked: 'validation', ... }` shape) BEFORE validation and provider calls.
|
|
213
|
+
- **`add()`** — looks up the user doc by email first (same `auth.email` equality query the webhook processors use). Doc found + revoked → `{ blocked: 'consent', email }`. No doc found → proceed (pure newsletter contact). Lookup failure → proceed (fail open, logged) — a rare duplicate add is recoverable; silently dropping contacts is not.
|
|
214
|
+
- **`remove()` is never gated** — removal is always safe.
|
|
215
|
+
|
|
216
|
+
The signup route ADDITIONALLY gates at its call site (`userRecord.consent.marketing.status === 'granted'` before calling `mailer.sync(uid)`) — that check runs before the route's paid NeverBounce/ZeroBounce mailbox validation, so declined users never trigger a paid check. The library gate is the safety net for everyone else.
|
|
217
|
+
|
|
218
|
+
The gate logic is unit-tested without Firestore in [src/manager/libraries/email/marketing/consent-gate.test.js](../src/manager/libraries/email/marketing/consent-gate.test.js) — run with `node src/manager/libraries/email/marketing/consent-gate.test.js`.
|
|
195
219
|
|
|
196
220
|
## Configuration
|
|
197
221
|
|
|
@@ -304,6 +328,7 @@ After the migration: optionally run a re-opt-in drip campaign to legally recover
|
|
|
304
328
|
- [test/routes/marketing/webhook.js](../test/routes/marketing/webhook.js) — 15+ tests covering SendGrid + Beehiiv processors against the emulator
|
|
305
329
|
- [test/routes/marketing/webhook-forward.js](../test/routes/marketing/webhook-forward.js) — verifies the forwarder route returns 404 on non-parent BEMs
|
|
306
330
|
- [test/helpers/webhook-forward.js](../test/helpers/webhook-forward.js) — 12 unit-style tests with mocked admin + fetch, covering fan-out, URL derivation, failure isolation, self-inclusion, edge cases
|
|
331
|
+
- [src/manager/libraries/email/marketing/consent-gate.test.js](../src/manager/libraries/email/marketing/consent-gate.test.js) — 28 plain-node cases for the library consent gate (revoked-only skip semantics, `{ blocked: 'consent' }` returns from `sync()`/`add()`, by-email lookup query + normalization, fail-open on lookup errors); runs directly with `node`, no emulator
|
|
307
332
|
|
|
308
333
|
**Total: 75+ tests across the consent system.**
|
|
309
334
|
|
package/docs/email-system.md
CHANGED
|
@@ -58,7 +58,7 @@ After `build()`, `send()` delivers via SendGrid, handles scheduled sends (>71h
|
|
|
58
58
|
|
|
59
59
|
### Email Validation Pipeline (`validation.js`)
|
|
60
60
|
|
|
61
|
-
All marketing contact operations (`add`, `sync`) pass through `validate()` before reaching providers. Checks run in order; the first failure short-circuits.
|
|
61
|
+
All marketing contact operations (`add`, `sync`) pass through `validate()` before reaching providers. Checks run in order; the first failure short-circuits. (The library consent gate runs even earlier — a user with `consent.marketing.status === 'revoked'` returns `{ blocked: 'consent', email }` before validation; see [consent.md](consent.md#email-library-consent-gate).)
|
|
62
62
|
|
|
63
63
|
| # | Check | What it catches | Cost | Default |
|
|
64
64
|
|---|---|---|---|---|
|
|
@@ -284,7 +284,7 @@ All email tests live under `test/email/`, mirroring the source at `src/manager/l
|
|
|
284
284
|
|---|---|---|
|
|
285
285
|
| `templates.js` | MJML rendering for all 4 email templates (11 tests) | No |
|
|
286
286
|
| `transactional.js` | Transactional email building (assertions on output shape) | No |
|
|
287
|
-
| `validation.js` | Email format/disposable/corporate/local-part/typo/dns checks (
|
|
287
|
+
| `validation.js` | Email format/disposable/corporate/local-part/typo/dns checks (52 tests) | No |
|
|
288
288
|
| `transactional-send.js` | Single transactional email send via SendGrid | Yes |
|
|
289
289
|
| `campaign-send.js` | Marketing campaign send with title + CTA + discount code | Yes |
|
|
290
290
|
| `feedback-and-plain-send.js` | Feedback + plain template visual test sends | Yes |
|
package/package.json
CHANGED
|
@@ -1256,6 +1256,7 @@
|
|
|
1256
1256
|
"cloudgen.world",
|
|
1257
1257
|
"cloudsign.in",
|
|
1258
1258
|
"cloudwhitespace.cc.cd",
|
|
1259
|
+
"cloudwhitezone.cc.cd",
|
|
1259
1260
|
"clout.wiki",
|
|
1260
1261
|
"clowmail.com",
|
|
1261
1262
|
"clrmail.com",
|
|
@@ -2406,6 +2407,7 @@
|
|
|
2406
2407
|
"gdmail.top",
|
|
2407
2408
|
"gdqoe.net",
|
|
2408
2409
|
"gdxs.cc.cd",
|
|
2410
|
+
"gear3.pro",
|
|
2409
2411
|
"gebrauchtwarencenter.com",
|
|
2410
2412
|
"gedmail.win",
|
|
2411
2413
|
"geekforex.com",
|
|
@@ -2624,6 +2626,7 @@
|
|
|
2624
2626
|
"gpt-mail-free-6419.dynv6.net",
|
|
2625
2627
|
"gpt-mail-free-6603.dynv6.net",
|
|
2626
2628
|
"gpt-mail-free-6664.dynv6.net",
|
|
2629
|
+
"gpt-mail-free-6691.dynv6.net",
|
|
2627
2630
|
"gpt-mail-free-6779.dynv6.net",
|
|
2628
2631
|
"gpt-mail-free-6812.dynv6.net",
|
|
2629
2632
|
"gpt-mail-free-6925.dynv6.net",
|
|
@@ -3412,6 +3415,7 @@
|
|
|
3412
3415
|
"kruay.com",
|
|
3413
3416
|
"krypton.tk",
|
|
3414
3417
|
"krysentra.cfd",
|
|
3418
|
+
"kskblzdjdwkzkbl.top",
|
|
3415
3419
|
"ksmtrck.tk",
|
|
3416
3420
|
"kuhaku.indevs.in",
|
|
3417
3421
|
"kuhrap.com",
|
|
@@ -5818,6 +5822,7 @@
|
|
|
5818
5822
|
"tempalias.com",
|
|
5819
5823
|
"tempblockchain.com",
|
|
5820
5824
|
"tempe-mail.com",
|
|
5825
|
+
"tempebossok.my.id",
|
|
5821
5826
|
"tempemail.biz",
|
|
5822
5827
|
"tempemail.co.za",
|
|
5823
5828
|
"tempemail.com",
|
|
@@ -6656,6 +6661,7 @@
|
|
|
6656
6661
|
"x24.com",
|
|
6657
6662
|
"x80.dpdns.org",
|
|
6658
6663
|
"x80la.shop",
|
|
6664
|
+
"x9ad1.my",
|
|
6659
6665
|
"xagloo.co",
|
|
6660
6666
|
"xagloo.com",
|
|
6661
6667
|
"xandoria.cfd",
|
|
@@ -0,0 +1,249 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Consent gate test — verifies Marketing.add()/sync() skip users whose
|
|
3
|
+
* consent.marketing.status is the literal string 'revoked' (and ONLY then).
|
|
4
|
+
*
|
|
5
|
+
* Plain-node control-flow test (no emulator, no providers, no network):
|
|
6
|
+
* - isMarketingRevoked() — the pure gate semantic (revoked-only skip).
|
|
7
|
+
* - sync() — gate fires after doc resolution, BEFORE validation/providers, so a
|
|
8
|
+
* revoked doc returns { blocked: 'consent', email } with zero I/O. Proceed cases
|
|
9
|
+
* stop at the testing-mode provider guard (assistant.isTesting() → true here).
|
|
10
|
+
* - add() — by-email lookup runs through a minimal in-memory firestore stand-in to
|
|
11
|
+
* exercise add()'s wiring (block on revoked, proceed on no-user, fail open on
|
|
12
|
+
* lookup error). The emulator suites (test/routes/marketing/*) remain the
|
|
13
|
+
* integration surface for the real Firestore paths.
|
|
14
|
+
*
|
|
15
|
+
* Run: node src/manager/libraries/email/marketing/consent-gate.test.js
|
|
16
|
+
*/
|
|
17
|
+
const Marketing = require('./index.js');
|
|
18
|
+
|
|
19
|
+
const { isMarketingRevoked } = Marketing;
|
|
20
|
+
|
|
21
|
+
// Proceed cases must stop at the testing-mode provider guard — never let an ambient
|
|
22
|
+
// TEST_EXTENDED_MODE (or provider API keys) in the shell turn this into a live call.
|
|
23
|
+
delete process.env.TEST_EXTENDED_MODE;
|
|
24
|
+
|
|
25
|
+
// ─── isMarketingRevoked(): ONLY the literal 'revoked' blocks ───
|
|
26
|
+
const GATE_CASES = [
|
|
27
|
+
{ name: 'status revoked', doc: { consent: { marketing: { status: 'revoked' } } }, expect: true },
|
|
28
|
+
{ name: 'status granted', doc: { consent: { marketing: { status: 'granted' } } }, expect: false },
|
|
29
|
+
{ name: 'status null', doc: { consent: { marketing: { status: null } } }, expect: false },
|
|
30
|
+
{ name: 'status missing', doc: { consent: { marketing: {} } }, expect: false },
|
|
31
|
+
{ name: 'marketing missing', doc: { consent: {} }, expect: false },
|
|
32
|
+
{ name: 'consent missing (legacy user)', doc: { auth: { email: 'legacy@gmail.com' } }, expect: false },
|
|
33
|
+
{ name: 'empty doc', doc: {}, expect: false },
|
|
34
|
+
{ name: 'null doc (no user)', doc: null, expect: false },
|
|
35
|
+
{ name: 'undefined doc', doc: undefined, expect: false },
|
|
36
|
+
{ name: 'status REVOKED (case-sensitive literal)', doc: { consent: { marketing: { status: 'REVOKED' } } }, expect: false },
|
|
37
|
+
{ name: 'status pending (future enum value)', doc: { consent: { marketing: { status: 'pending' } } }, expect: false },
|
|
38
|
+
{ name: 'status true (non-string)', doc: { consent: { marketing: { status: true } } }, expect: false },
|
|
39
|
+
];
|
|
40
|
+
|
|
41
|
+
/**
|
|
42
|
+
* Minimal assistant — just what the Marketing constructor + gate paths read.
|
|
43
|
+
* isTesting() → true so proceed cases stop at the provider guard (no network).
|
|
44
|
+
*/
|
|
45
|
+
function buildAssistant(admin) {
|
|
46
|
+
const calls = { logs: [], warns: [], errors: [] };
|
|
47
|
+
|
|
48
|
+
const assistant = {
|
|
49
|
+
Manager: {
|
|
50
|
+
libraries: { admin },
|
|
51
|
+
config: {},
|
|
52
|
+
},
|
|
53
|
+
isTesting: () => true,
|
|
54
|
+
log: (...args) => calls.logs.push(args),
|
|
55
|
+
warn: (...args) => calls.warns.push(args),
|
|
56
|
+
error: (...args) => calls.errors.push(args),
|
|
57
|
+
};
|
|
58
|
+
|
|
59
|
+
return { assistant, calls };
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
/**
|
|
63
|
+
* Minimal in-memory firestore stand-in for add()'s by-email lookup and sync()'s
|
|
64
|
+
* by-uid fetch. `userDoc: null` → empty query result / nonexistent doc.
|
|
65
|
+
* `failLookup: true` → the query get() rejects (exercises the fail-open path).
|
|
66
|
+
*/
|
|
67
|
+
function buildAdmin({ userDoc = null, failLookup = false } = {}) {
|
|
68
|
+
const captured = { queries: [], docPaths: [] };
|
|
69
|
+
|
|
70
|
+
const admin = {
|
|
71
|
+
firestore: () => ({
|
|
72
|
+
collection: (name) => ({
|
|
73
|
+
where: (field, op, value) => {
|
|
74
|
+
captured.queries.push({ collection: name, field, op, value });
|
|
75
|
+
return {
|
|
76
|
+
limit: () => ({
|
|
77
|
+
get: async () => {
|
|
78
|
+
if (failLookup) {
|
|
79
|
+
throw new Error('firestore unavailable');
|
|
80
|
+
}
|
|
81
|
+
return userDoc
|
|
82
|
+
? { empty: false, docs: [{ id: 'test-uid', data: () => userDoc }] }
|
|
83
|
+
: { empty: true, docs: [] };
|
|
84
|
+
},
|
|
85
|
+
}),
|
|
86
|
+
};
|
|
87
|
+
},
|
|
88
|
+
}),
|
|
89
|
+
doc: (path) => {
|
|
90
|
+
captured.docPaths.push(path);
|
|
91
|
+
return {
|
|
92
|
+
get: async () => ({
|
|
93
|
+
exists: !!userDoc,
|
|
94
|
+
data: () => userDoc,
|
|
95
|
+
}),
|
|
96
|
+
};
|
|
97
|
+
},
|
|
98
|
+
}),
|
|
99
|
+
};
|
|
100
|
+
|
|
101
|
+
return { admin, captured };
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
const REVOKED_DOC = {
|
|
105
|
+
auth: { email: 'revoked.user@gmail.com' },
|
|
106
|
+
consent: { marketing: { status: 'revoked' } },
|
|
107
|
+
};
|
|
108
|
+
|
|
109
|
+
const GRANTED_DOC = {
|
|
110
|
+
auth: { email: 'granted.user@gmail.com' },
|
|
111
|
+
consent: { marketing: { status: 'granted' } },
|
|
112
|
+
};
|
|
113
|
+
|
|
114
|
+
const LEGACY_DOC = {
|
|
115
|
+
auth: { email: 'legacy.user@gmail.com' },
|
|
116
|
+
// No consent field at all — pre-consent-system user, must keep syncing
|
|
117
|
+
};
|
|
118
|
+
|
|
119
|
+
async function run() {
|
|
120
|
+
let passed = 0;
|
|
121
|
+
let failed = 0;
|
|
122
|
+
|
|
123
|
+
function check(ok, name, detail) {
|
|
124
|
+
if (ok) {
|
|
125
|
+
passed++;
|
|
126
|
+
return;
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
failed++;
|
|
130
|
+
console.log(` ✗ ${name}`);
|
|
131
|
+
if (detail) {
|
|
132
|
+
console.log(` ${detail}`);
|
|
133
|
+
}
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
// ─── 1. isMarketingRevoked() semantics ───
|
|
137
|
+
for (const { name, doc, expect } of GATE_CASES) {
|
|
138
|
+
const actual = isMarketingRevoked(doc);
|
|
139
|
+
check(actual === expect, `isMarketingRevoked: ${name}`, `Expected ${expect}, got ${actual}`);
|
|
140
|
+
}
|
|
141
|
+
|
|
142
|
+
// ─── 2. sync() blocks a revoked doc (gate fires before validation/providers) ───
|
|
143
|
+
{
|
|
144
|
+
const { assistant, calls } = buildAssistant(null);
|
|
145
|
+
const result = await new Marketing(assistant).sync(REVOKED_DOC);
|
|
146
|
+
|
|
147
|
+
check(result.blocked === 'consent', 'sync(): blocks revoked doc', `Expected blocked='consent', got ${JSON.stringify(result)}`);
|
|
148
|
+
check(result.email === 'revoked.user@gmail.com', 'sync(): blocked result includes email', `Got ${JSON.stringify(result)}`);
|
|
149
|
+
check(calls.warns.length === 1, 'sync(): revoked skip logs a warn', `Got ${calls.warns.length} warns`);
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
// ─── 3. sync() proceeds on missing consent (legacy user) ───
|
|
153
|
+
{
|
|
154
|
+
const { assistant } = buildAssistant(null);
|
|
155
|
+
const result = await new Marketing(assistant).sync(LEGACY_DOC);
|
|
156
|
+
|
|
157
|
+
check(result.blocked === undefined, 'sync(): proceeds on missing consent', `Expected no block, got ${JSON.stringify(result)}`);
|
|
158
|
+
}
|
|
159
|
+
|
|
160
|
+
// ─── 4. sync() proceeds on granted consent ───
|
|
161
|
+
{
|
|
162
|
+
const { assistant } = buildAssistant(null);
|
|
163
|
+
const result = await new Marketing(assistant).sync(GRANTED_DOC);
|
|
164
|
+
|
|
165
|
+
check(result.blocked === undefined, 'sync(): proceeds on granted consent', `Expected no block, got ${JSON.stringify(result)}`);
|
|
166
|
+
}
|
|
167
|
+
|
|
168
|
+
// ─── 5. sync() by uid blocks when the fetched doc is revoked ───
|
|
169
|
+
{
|
|
170
|
+
const { admin, captured } = buildAdmin({ userDoc: REVOKED_DOC });
|
|
171
|
+
const { assistant } = buildAssistant(admin);
|
|
172
|
+
const result = await new Marketing(assistant).sync('revoked-uid');
|
|
173
|
+
|
|
174
|
+
check(result.blocked === 'consent', 'sync(uid): blocks revoked fetched doc', `Expected blocked='consent', got ${JSON.stringify(result)}`);
|
|
175
|
+
check(captured.docPaths[0] === 'users/revoked-uid', 'sync(uid): fetches users/{uid}', `Got ${JSON.stringify(captured.docPaths)}`);
|
|
176
|
+
}
|
|
177
|
+
|
|
178
|
+
// ─── 6. add() blocks when the email maps to a revoked user ───
|
|
179
|
+
{
|
|
180
|
+
const { admin, captured } = buildAdmin({ userDoc: REVOKED_DOC });
|
|
181
|
+
const { assistant, calls } = buildAssistant(admin);
|
|
182
|
+
const result = await new Marketing(assistant).add({ email: 'revoked.user@gmail.com' });
|
|
183
|
+
|
|
184
|
+
check(result.blocked === 'consent', 'add(): blocks revoked user by email', `Expected blocked='consent', got ${JSON.stringify(result)}`);
|
|
185
|
+
check(result.email === 'revoked.user@gmail.com', 'add(): blocked result includes email', `Got ${JSON.stringify(result)}`);
|
|
186
|
+
check(calls.warns.length === 1, 'add(): revoked skip logs a warn', `Got ${calls.warns.length} warns`);
|
|
187
|
+
|
|
188
|
+
const query = captured.queries[0];
|
|
189
|
+
check(
|
|
190
|
+
query
|
|
191
|
+
&& query.collection === 'users'
|
|
192
|
+
&& query.field === 'auth.email'
|
|
193
|
+
&& query.op === '=='
|
|
194
|
+
&& query.value === 'revoked.user@gmail.com',
|
|
195
|
+
'add(): looks up users by auth.email (webhook-processor query)',
|
|
196
|
+
`Got ${JSON.stringify(query)}`
|
|
197
|
+
);
|
|
198
|
+
}
|
|
199
|
+
|
|
200
|
+
// ─── 7. add() normalizes the email for the lookup ───
|
|
201
|
+
{
|
|
202
|
+
const { admin, captured } = buildAdmin({ userDoc: null });
|
|
203
|
+
const { assistant } = buildAssistant(admin);
|
|
204
|
+
await new Marketing(assistant).add({ email: ' Revoked.User@GMAIL.com ' });
|
|
205
|
+
|
|
206
|
+
check(
|
|
207
|
+
captured.queries[0]?.value === 'revoked.user@gmail.com',
|
|
208
|
+
'add(): lookup trims + lowercases the email',
|
|
209
|
+
`Got ${JSON.stringify(captured.queries[0])}`
|
|
210
|
+
);
|
|
211
|
+
}
|
|
212
|
+
|
|
213
|
+
// ─── 8. add() proceeds when no user doc exists (pure newsletter contact) ───
|
|
214
|
+
{
|
|
215
|
+
const { admin } = buildAdmin({ userDoc: null });
|
|
216
|
+
const { assistant } = buildAssistant(admin);
|
|
217
|
+
const result = await new Marketing(assistant).add({ email: 'newsletter.reader@gmail.com' });
|
|
218
|
+
|
|
219
|
+
check(result.blocked === undefined, 'add(): proceeds when no user doc exists', `Expected no block, got ${JSON.stringify(result)}`);
|
|
220
|
+
}
|
|
221
|
+
|
|
222
|
+
// ─── 9. add() proceeds when the matched user has no consent field (legacy) ───
|
|
223
|
+
{
|
|
224
|
+
const { admin } = buildAdmin({ userDoc: LEGACY_DOC });
|
|
225
|
+
const { assistant } = buildAssistant(admin);
|
|
226
|
+
const result = await new Marketing(assistant).add({ email: 'legacy.user@gmail.com' });
|
|
227
|
+
|
|
228
|
+
check(result.blocked === undefined, 'add(): proceeds on legacy user (no consent field)', `Expected no block, got ${JSON.stringify(result)}`);
|
|
229
|
+
}
|
|
230
|
+
|
|
231
|
+
// ─── 10. add() fails open when the lookup errors ───
|
|
232
|
+
{
|
|
233
|
+
const { admin } = buildAdmin({ failLookup: true });
|
|
234
|
+
const { assistant, calls } = buildAssistant(admin);
|
|
235
|
+
const result = await new Marketing(assistant).add({ email: 'someone@gmail.com' });
|
|
236
|
+
|
|
237
|
+
check(result.blocked === undefined, 'add(): fails open on lookup error', `Expected no block, got ${JSON.stringify(result)}`);
|
|
238
|
+
check(calls.errors.length === 1, 'add(): lookup error is logged', `Got ${calls.errors.length} errors`);
|
|
239
|
+
}
|
|
240
|
+
|
|
241
|
+
console.log('');
|
|
242
|
+
console.log(`${passed} passed, ${failed} failed`);
|
|
243
|
+
|
|
244
|
+
if (failed > 0) {
|
|
245
|
+
process.exit(1);
|
|
246
|
+
}
|
|
247
|
+
}
|
|
248
|
+
|
|
249
|
+
run();
|
|
@@ -20,10 +20,15 @@
|
|
|
20
20
|
*
|
|
21
21
|
* Used by:
|
|
22
22
|
* - routes/marketing/contact (add)
|
|
23
|
-
* -
|
|
23
|
+
* - routes/user/signup (sync on signup — call site also gates on consent to skip the paid mailbox check)
|
|
24
24
|
* - Payment transition handlers (sync on subscription change)
|
|
25
25
|
* - Auth on-delete handler (remove contact)
|
|
26
26
|
* - Campaign cron jobs (send campaigns)
|
|
27
|
+
*
|
|
28
|
+
* Consent gate: add() and sync() skip users whose consent.marketing.status is the
|
|
29
|
+
* literal string 'revoked' and return { blocked: 'consent', email }. Missing consent,
|
|
30
|
+
* null, or any other value proceeds — legacy users have no consent field and must
|
|
31
|
+
* keep syncing. remove() is always safe and stays ungated.
|
|
27
32
|
*/
|
|
28
33
|
const _ = require('lodash');
|
|
29
34
|
|
|
@@ -52,9 +57,52 @@ function Marketing(assistant) {
|
|
|
52
57
|
}
|
|
53
58
|
|
|
54
59
|
// ============================================================
|
|
55
|
-
// Contact management (add / sync / remove)
|
|
60
|
+
// Contact management (add / sync / remove)
|
|
56
61
|
// ============================================================
|
|
57
62
|
|
|
63
|
+
/**
|
|
64
|
+
* Consent gate — true ONLY when consent.marketing.status is the literal string 'revoked'.
|
|
65
|
+
* Missing consent, null, or any other value proceeds — legacy users have no consent field
|
|
66
|
+
* and must keep syncing.
|
|
67
|
+
*
|
|
68
|
+
* @param {object|null|undefined} userDoc - User doc data (or null/undefined when no user exists)
|
|
69
|
+
* @returns {boolean}
|
|
70
|
+
*/
|
|
71
|
+
function isMarketingRevoked(userDoc) {
|
|
72
|
+
return userDoc?.consent?.marketing?.status === 'revoked';
|
|
73
|
+
}
|
|
74
|
+
|
|
75
|
+
/**
|
|
76
|
+
* Look up a user doc by email (same query the marketing webhook processors use).
|
|
77
|
+
* Returns the doc data, or null when no user matches OR the lookup fails (fail open —
|
|
78
|
+
* a rare duplicate add is recoverable; silently dropping contacts is not).
|
|
79
|
+
*
|
|
80
|
+
* @param {object} admin - firebase-admin instance
|
|
81
|
+
* @param {object} assistant - Assistant (for logging)
|
|
82
|
+
* @param {string} email - Email address
|
|
83
|
+
* @returns {Promise<object|null>}
|
|
84
|
+
*/
|
|
85
|
+
async function findUserByEmail(admin, assistant, email) {
|
|
86
|
+
const snapshot = await admin.firestore().collection('users')
|
|
87
|
+
.where('auth.email', '==', email.trim().toLowerCase())
|
|
88
|
+
.limit(1)
|
|
89
|
+
.get()
|
|
90
|
+
.catch((e) => {
|
|
91
|
+
assistant.error('Marketing: User lookup by email failed (proceeding without consent gate):', e);
|
|
92
|
+
return null;
|
|
93
|
+
});
|
|
94
|
+
|
|
95
|
+
if (!snapshot || snapshot.empty) {
|
|
96
|
+
return null;
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
return snapshot.docs[0].data();
|
|
100
|
+
}
|
|
101
|
+
|
|
102
|
+
// Exposed as statics for plain-node unit tests (consent-gate.test.js)
|
|
103
|
+
Marketing.isMarketingRevoked = isMarketingRevoked;
|
|
104
|
+
Marketing.findUserByEmail = findUserByEmail;
|
|
105
|
+
|
|
58
106
|
Marketing.prototype.add = async function (options) {
|
|
59
107
|
const self = this;
|
|
60
108
|
const assistant = self.assistant;
|
|
@@ -65,6 +113,16 @@ Marketing.prototype.add = async function (options) {
|
|
|
65
113
|
return {};
|
|
66
114
|
}
|
|
67
115
|
|
|
116
|
+
// Consent gate — if this email maps to a user who revoked marketing consent, never
|
|
117
|
+
// re-add them. No user doc → proceed (pure newsletter contact). Lookup failure →
|
|
118
|
+
// proceed (fail open, logged in findUserByEmail).
|
|
119
|
+
const userDoc = await findUserByEmail(self.admin, assistant, email);
|
|
120
|
+
|
|
121
|
+
if (isMarketingRevoked(userDoc)) {
|
|
122
|
+
assistant.warn(`Marketing.add(): Consent revoked, skipping: ${email}`);
|
|
123
|
+
return { blocked: 'consent', email };
|
|
124
|
+
}
|
|
125
|
+
|
|
68
126
|
const validation = await validate(email);
|
|
69
127
|
if (!validation.valid) {
|
|
70
128
|
assistant.warn(`Marketing.add(): Validation failed, skipping: ${email}`, validation.checks);
|
|
@@ -140,6 +198,13 @@ Marketing.prototype.sync = async function (userDocOrUid) {
|
|
|
140
198
|
return {};
|
|
141
199
|
}
|
|
142
200
|
|
|
201
|
+
// Consent gate — never re-add a user who revoked marketing consent (payment-event
|
|
202
|
+
// syncs, admin re-syncs, and batch syncs all funnel through here).
|
|
203
|
+
if (isMarketingRevoked(userDoc)) {
|
|
204
|
+
assistant.warn(`Marketing.sync(): Consent revoked, skipping: ${email}`);
|
|
205
|
+
return { blocked: 'consent', email };
|
|
206
|
+
}
|
|
207
|
+
|
|
143
208
|
const validation = await validate(email);
|
|
144
209
|
if (!validation.valid) {
|
|
145
210
|
assistant.warn(`Marketing.sync(): Validation failed, skipping: ${email}`, validation.checks);
|
|
@@ -1,13 +1,36 @@
|
|
|
1
1
|
const fetch = require('wonderful-fetch');
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
// NeverBounce numeric → textcode map. The v4 single/check API returns `result`
|
|
4
|
+
// as a textcode STRING ('valid', 'invalid', ...); numeric codes only appear in
|
|
5
|
+
// other response modes — tolerate both so a representation change can't silently
|
|
6
|
+
// fail every check again (that exact bug skipped marketing sync for all signups
|
|
7
|
+
// between BEM 5.5.1 and 5.6.1).
|
|
8
|
+
const NUMERIC_RESULT_MAP = {
|
|
4
9
|
0: 'valid',
|
|
5
10
|
1: 'invalid',
|
|
6
11
|
2: 'disposable',
|
|
7
|
-
3: '
|
|
12
|
+
3: 'catchall',
|
|
8
13
|
4: 'unknown',
|
|
9
14
|
};
|
|
10
15
|
|
|
16
|
+
// valid, catchall, unknown are allowed; invalid, disposable are blocked
|
|
17
|
+
const ALLOWED_RESULTS = new Set(['valid', 'catchall', 'unknown']);
|
|
18
|
+
|
|
19
|
+
/**
|
|
20
|
+
* Normalize a NeverBounce single-check `result` into { valid, status }.
|
|
21
|
+
* Accepts textcode strings (canonical) or numeric codes; 'catch-all' → 'catchall'.
|
|
22
|
+
*
|
|
23
|
+
* @param {string|number} result - NeverBounce `result` field
|
|
24
|
+
* @returns {{ valid: boolean, status: string }}
|
|
25
|
+
*/
|
|
26
|
+
function parseResult(result) {
|
|
27
|
+
const status = typeof result === 'number'
|
|
28
|
+
? (NUMERIC_RESULT_MAP[result] || 'unknown')
|
|
29
|
+
: String(result).toLowerCase().replace('-', '');
|
|
30
|
+
|
|
31
|
+
return { valid: ALLOWED_RESULTS.has(status), status };
|
|
32
|
+
}
|
|
33
|
+
|
|
11
34
|
/**
|
|
12
35
|
* @param {string} email
|
|
13
36
|
* @returns {Promise<{ valid: boolean, status?: string, subStatus?: string, error?: string, provider: string }>}
|
|
@@ -29,14 +52,10 @@ async function verify(email) {
|
|
|
29
52
|
return { valid: true, error: 'Unexpected response format', provider: 'neverbounce' };
|
|
30
53
|
}
|
|
31
54
|
|
|
32
|
-
const status =
|
|
33
|
-
// 0=valid, 3=catch-all, 4=unknown are allowed; 1=invalid, 2=disposable are blocked
|
|
34
|
-
const nbValid = data.result === 0
|
|
35
|
-
|| data.result === 3
|
|
36
|
-
|| data.result === 4;
|
|
55
|
+
const { valid, status } = parseResult(data.result);
|
|
37
56
|
|
|
38
57
|
return {
|
|
39
|
-
valid
|
|
58
|
+
valid,
|
|
40
59
|
status,
|
|
41
60
|
subStatus: data.flags?.length ? data.flags.join(',') : null,
|
|
42
61
|
provider: 'neverbounce',
|
|
@@ -47,4 +66,4 @@ async function verify(email) {
|
|
|
47
66
|
}
|
|
48
67
|
}
|
|
49
68
|
|
|
50
|
-
module.exports = { verify };
|
|
69
|
+
module.exports = { verify, parseResult };
|
|
@@ -18,7 +18,7 @@
|
|
|
18
18
|
* Used by:
|
|
19
19
|
* - routes/marketing/contact/post.js
|
|
20
20
|
* - functions/core/actions/api/general/add-marketing-contact.js
|
|
21
|
-
* - routes/user/signup/post.js (
|
|
21
|
+
* - routes/user/signup/post.js (ALL_CHECKS before marketing sync; isDisposable() for affiliate fraud prevention)
|
|
22
22
|
* - libraries/email/marketing/index.js (safety net before Beehiiv/SendGrid add/sync)
|
|
23
23
|
*/
|
|
24
24
|
const path = require('path');
|
|
@@ -1,9 +1,11 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* Email validation test — verifies all free checks (format, disposable, corporate, localPart, typo, dns)
|
|
3
|
+
* plus NeverBounce result parsing (no API calls).
|
|
3
4
|
*
|
|
4
5
|
* Run: node src/manager/libraries/email/validation.test.js
|
|
5
6
|
*/
|
|
6
7
|
const { validate } = require('./validation.js');
|
|
8
|
+
const { parseResult } = require('./validation-provider-neverbounce.js');
|
|
7
9
|
|
|
8
10
|
const FREE_CHECKS = ['format', 'disposable', 'corporate', 'localPart', 'typo', 'dns'];
|
|
9
11
|
|
|
@@ -85,10 +87,38 @@ const CASES = [
|
|
|
85
87
|
{ email: 'someone@example.com', expect: 'fail', check: 'dns' },
|
|
86
88
|
];
|
|
87
89
|
|
|
90
|
+
// NeverBounce single-check `result` parsing — the API returns STRING textcodes.
|
|
91
|
+
// Regression: BEM 5.5.1–5.6.1 compared against numbers, failing every mailbox
|
|
92
|
+
// check and silently skipping marketing sync for all signups.
|
|
93
|
+
const NB_PARSE_CASES = [
|
|
94
|
+
{ result: 'valid', expectValid: true, expectStatus: 'valid' },
|
|
95
|
+
{ result: 'catchall', expectValid: true, expectStatus: 'catchall' },
|
|
96
|
+
{ result: 'catch-all', expectValid: true, expectStatus: 'catchall' },
|
|
97
|
+
{ result: 'unknown', expectValid: true, expectStatus: 'unknown' },
|
|
98
|
+
{ result: 'invalid', expectValid: false, expectStatus: 'invalid' },
|
|
99
|
+
{ result: 'disposable', expectValid: false, expectStatus: 'disposable' },
|
|
100
|
+
{ result: 0, expectValid: true, expectStatus: 'valid' },
|
|
101
|
+
{ result: 1, expectValid: false, expectStatus: 'invalid' },
|
|
102
|
+
{ result: 2, expectValid: false, expectStatus: 'disposable' },
|
|
103
|
+
{ result: 3, expectValid: true, expectStatus: 'catchall' },
|
|
104
|
+
{ result: 4, expectValid: true, expectStatus: 'unknown' },
|
|
105
|
+
];
|
|
106
|
+
|
|
88
107
|
async function run() {
|
|
89
108
|
let passed = 0;
|
|
90
109
|
let failed = 0;
|
|
91
110
|
|
|
111
|
+
for (const { result, expectValid, expectStatus } of NB_PARSE_CASES) {
|
|
112
|
+
const parsed = parseResult(result);
|
|
113
|
+
if (parsed.valid === expectValid && parsed.status === expectStatus) {
|
|
114
|
+
passed++;
|
|
115
|
+
} else {
|
|
116
|
+
failed++;
|
|
117
|
+
console.log(` ✗ parseResult(${JSON.stringify(result)})`);
|
|
118
|
+
console.log(` Expected: valid=${expectValid} status=${expectStatus}, Got: valid=${parsed.valid} status=${parsed.status}`);
|
|
119
|
+
}
|
|
120
|
+
}
|
|
121
|
+
|
|
92
122
|
for (const { email, expect: expected, check: expectedCheck } of CASES) {
|
|
93
123
|
const result = await validate(email, { checks: FREE_CHECKS });
|
|
94
124
|
const actualPass = result.valid;
|
|
@@ -115,7 +145,7 @@ async function run() {
|
|
|
115
145
|
}
|
|
116
146
|
|
|
117
147
|
console.log('');
|
|
118
|
-
console.log(`${passed} passed, ${failed} failed out of ${CASES.length} cases`);
|
|
148
|
+
console.log(`${passed} passed, ${failed} failed out of ${CASES.length + NB_PARSE_CASES.length} cases`);
|
|
119
149
|
|
|
120
150
|
if (failed > 0) {
|
|
121
151
|
process.exit(1);
|
|
@@ -1,6 +1,10 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* DELETE /marketing/contact - Remove marketing contact
|
|
3
3
|
* Admin-only endpoint to unsubscribe from newsletter
|
|
4
|
+
*
|
|
5
|
+
* Also mirrors consent.marketing.status = 'revoked' (source: 'admin') to the user doc
|
|
6
|
+
* when the email maps to a user — otherwise the next sync (payment event, admin re-sync)
|
|
7
|
+
* would re-add the contact the admin just removed.
|
|
4
8
|
*/
|
|
5
9
|
|
|
6
10
|
module.exports = async ({ assistant, Manager, settings, analytics }) => {
|
|
@@ -28,6 +32,11 @@ module.exports = async ({ assistant, Manager, settings, analytics }) => {
|
|
|
28
32
|
const mailer = Manager.Email(assistant);
|
|
29
33
|
const providerResults = await mailer.remove(email);
|
|
30
34
|
|
|
35
|
+
// Mirror the removal to the user doc's consent so future syncs hit the email
|
|
36
|
+
// library's consent gate instead of silently re-adding the contact
|
|
37
|
+
// (best-effort, silent when the email maps to no user)
|
|
38
|
+
await mirrorRevokedConsent({ assistant, Manager, email });
|
|
39
|
+
|
|
31
40
|
// Log result
|
|
32
41
|
assistant.log('marketing/contact delete result:', {
|
|
33
42
|
email,
|
|
@@ -42,3 +51,52 @@ module.exports = async ({ assistant, Manager, settings, analytics }) => {
|
|
|
42
51
|
providers: providerResults,
|
|
43
52
|
});
|
|
44
53
|
};
|
|
54
|
+
|
|
55
|
+
/**
|
|
56
|
+
* Write consent.marketing.status = 'revoked' (source: 'admin') to the user doc that
|
|
57
|
+
* matches the removed email. Same lookup + write shape as the marketing webhook
|
|
58
|
+
* processors' revoke write. Silent when no user matches.
|
|
59
|
+
*/
|
|
60
|
+
async function mirrorRevokedConsent({ assistant, Manager, email }) {
|
|
61
|
+
const { admin } = Manager.libraries;
|
|
62
|
+
|
|
63
|
+
const snapshot = await admin.firestore().collection('users')
|
|
64
|
+
.where('auth.email', '==', email)
|
|
65
|
+
.limit(1)
|
|
66
|
+
.get()
|
|
67
|
+
.catch((e) => {
|
|
68
|
+
assistant.error('marketing/contact delete: Failed to look up user by email:', e);
|
|
69
|
+
return null;
|
|
70
|
+
});
|
|
71
|
+
|
|
72
|
+
if (!snapshot || snapshot.empty) {
|
|
73
|
+
return; // Silent — email may not map to a current user
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
const uid = snapshot.docs[0].id;
|
|
77
|
+
const timestamp = assistant.meta.startTime.timestamp;
|
|
78
|
+
const timestampUNIX = assistant.meta.startTime.timestampUNIX;
|
|
79
|
+
|
|
80
|
+
// Write consent.marketing.status = 'revoked' (preserve grantedAt — informational audit trail)
|
|
81
|
+
await admin.firestore().doc(`users/${uid}`).set({
|
|
82
|
+
consent: {
|
|
83
|
+
marketing: {
|
|
84
|
+
status: 'revoked',
|
|
85
|
+
revokedAt: {
|
|
86
|
+
timestamp,
|
|
87
|
+
timestampUNIX,
|
|
88
|
+
source: 'admin',
|
|
89
|
+
ip: null,
|
|
90
|
+
text: null,
|
|
91
|
+
},
|
|
92
|
+
},
|
|
93
|
+
},
|
|
94
|
+
metadata: Manager.Metadata().set({ tag: 'marketing/contact:delete' }),
|
|
95
|
+
}, { merge: true })
|
|
96
|
+
.then(() => {
|
|
97
|
+
assistant.log(`marketing/contact delete: Mirrored revoked consent to user ${uid} (${email})`);
|
|
98
|
+
})
|
|
99
|
+
.catch((e) => {
|
|
100
|
+
assistant.error(`marketing/contact delete: Failed to mirror revoked consent to ${uid}:`, e);
|
|
101
|
+
});
|
|
102
|
+
}
|
|
@@ -14,6 +14,9 @@
|
|
|
14
14
|
* - HMAC validates the link was generated by us.
|
|
15
15
|
* - Hits SendGrid ASM directly (legacy behavior — preserves existing one-click unsub).
|
|
16
16
|
* - Also writes canonical consent.marketing to user doc when email maps to a user.
|
|
17
|
+
* - Then cross-syncs via the email library (best-effort): unsubscribe removes the
|
|
18
|
+
* contact from ALL providers (Beehiiv too — compliance); subscribe re-adds the
|
|
19
|
+
* contact (sync when the email maps to a user, add when it doesn't).
|
|
17
20
|
*/
|
|
18
21
|
const fetch = require('wonderful-fetch');
|
|
19
22
|
const crypto = require('crypto');
|
|
@@ -148,8 +151,10 @@ async function handleAnonymous({ assistant, Manager, settings, analytics }) {
|
|
|
148
151
|
usage.increment('email-preferences');
|
|
149
152
|
await usage.update();
|
|
150
153
|
|
|
151
|
-
// Mirror to the user doc if this email maps to a user (best-effort, silent on miss)
|
|
152
|
-
|
|
154
|
+
// Mirror to the user doc if this email maps to a user (best-effort, silent on miss).
|
|
155
|
+
// Runs BEFORE the provider calls below — on subscribe it writes consent granted first,
|
|
156
|
+
// so the email library's consent gate passes when we re-sync.
|
|
157
|
+
const uid = await mirrorAnonymousToUserDoc({ assistant, Manager, email, action });
|
|
153
158
|
|
|
154
159
|
// Call SendGrid ASM (legacy behavior)
|
|
155
160
|
const shouldCallExternalAPIs = !assistant.isTesting() || process.env.TEST_EXTENDED_MODE;
|
|
@@ -184,6 +189,26 @@ async function handleAnonymous({ assistant, Manager, settings, analytics }) {
|
|
|
184
189
|
return assistant.respond('Failed to process your request', { code: 500 });
|
|
185
190
|
}
|
|
186
191
|
|
|
192
|
+
// Cross-provider sync via the email library (best-effort — ASM suppression above
|
|
193
|
+
// already succeeded). Unsubscribe must remove the contact from Beehiiv too
|
|
194
|
+
// (compliance); subscribe must actually re-add the contact: sync the matched user
|
|
195
|
+
// (the mirror above already wrote consent granted, so the library consent gate
|
|
196
|
+
// passes) or add a bare newsletter contact when no user matched.
|
|
197
|
+
const mailer = Manager.Email(assistant);
|
|
198
|
+
|
|
199
|
+
try {
|
|
200
|
+
if (action === 'unsubscribe') {
|
|
201
|
+
await mailer.remove(email);
|
|
202
|
+
} else if (uid) {
|
|
203
|
+
await mailer.sync(uid);
|
|
204
|
+
} else {
|
|
205
|
+
await mailer.add({ email, source: 'resubscribe' });
|
|
206
|
+
}
|
|
207
|
+
} catch (e) {
|
|
208
|
+
assistant.error(`email-preferences (anon) provider sync failed:`, e);
|
|
209
|
+
// Doc + ASM are already updated — provider sync is best-effort. Don't fail the request.
|
|
210
|
+
}
|
|
211
|
+
|
|
187
212
|
assistant.log('email-preferences (anon) result:', { email, asmId, action });
|
|
188
213
|
analytics.event('marketing/email-preferences', { action, mode: 'anonymous' });
|
|
189
214
|
|
|
@@ -194,6 +219,8 @@ async function handleAnonymous({ assistant, Manager, settings, analytics }) {
|
|
|
194
219
|
* Anonymous HMAC unsub also writes to the user doc (if found) so consent stays in sync.
|
|
195
220
|
* Silent if the email doesn't map to a user. Source is recorded as 'sendgrid' since the
|
|
196
221
|
* HMAC link only fires from SendGrid email footers.
|
|
222
|
+
*
|
|
223
|
+
* @returns {Promise<string|null>} The matched user's uid, or null when no user matches.
|
|
197
224
|
*/
|
|
198
225
|
async function mirrorAnonymousToUserDoc({ assistant, Manager, email, action }) {
|
|
199
226
|
const { admin } = Manager.libraries;
|
|
@@ -208,7 +235,7 @@ async function mirrorAnonymousToUserDoc({ assistant, Manager, email, action }) {
|
|
|
208
235
|
});
|
|
209
236
|
|
|
210
237
|
if (!snapshot || snapshot.empty) {
|
|
211
|
-
return; // Silent — email may not map to a current user
|
|
238
|
+
return null; // Silent — email may not map to a current user
|
|
212
239
|
}
|
|
213
240
|
|
|
214
241
|
const userDoc = snapshot.docs[0];
|
|
@@ -233,4 +260,6 @@ async function mirrorAnonymousToUserDoc({ assistant, Manager, email, action }) {
|
|
|
233
260
|
.catch((e) => {
|
|
234
261
|
assistant.error(`email-preferences (anon): Failed to mirror to user doc ${uid}:`, e);
|
|
235
262
|
});
|
|
263
|
+
|
|
264
|
+
return uid;
|
|
236
265
|
}
|
package/test/email/validation.js
CHANGED
|
@@ -1,8 +1,9 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* Test: Email validation library (libraries/email/validation.js)
|
|
3
|
-
* Unit tests for format, local part, disposable domain, and mailbox checks
|
|
3
|
+
* Unit tests for format, local part, disposable domain, corporate domain, typo domain, DNS, and mailbox checks
|
|
4
4
|
*
|
|
5
|
-
* Format, local part, and
|
|
5
|
+
* Format, local part, disposable, corporate, and typo tests always run (free, sync, offline-safe).
|
|
6
|
+
* DNS negative tests require TEST_EXTENDED_MODE (live DNS resolution).
|
|
6
7
|
* Mailbox verification tests require TEST_EXTENDED_MODE + NEVERBOUNCE_API_KEY or ZEROBOUNCE_API_KEY.
|
|
7
8
|
*/
|
|
8
9
|
const { validate, isDisposable, isCorporate, DEFAULT_CHECKS, ALL_CHECKS } = require('../../src/manager/libraries/email/validation.js');
|
|
@@ -115,15 +116,17 @@ module.exports = {
|
|
|
115
116
|
},
|
|
116
117
|
|
|
117
118
|
{
|
|
118
|
-
name: 'localpart-all-numeric-
|
|
119
|
+
name: 'localpart-all-numeric-allowed',
|
|
119
120
|
timeout: 5000,
|
|
120
121
|
|
|
121
122
|
async run({ assert }) {
|
|
123
|
+
// All-numeric local parts are legitimate (QQ emails like 1549482839@qq.com,
|
|
124
|
+
// student IDs) — the ^\d+$ pattern was removed in v5.5.6 after NeverBounce
|
|
125
|
+
// confirmed real users were being blocked.
|
|
122
126
|
const result = await validate('123456@gmail.com');
|
|
123
127
|
|
|
124
|
-
assert.equal(result.valid,
|
|
125
|
-
assert.propertyEquals(result, 'checks.localPart.
|
|
126
|
-
assert.propertyEquals(result, 'checks.localPart.reason', 'Matches junk pattern', 'Should match junk pattern');
|
|
128
|
+
assert.equal(result.valid, true, 'All-numeric local part should be allowed');
|
|
129
|
+
assert.propertyEquals(result, 'checks.localPart.valid', true, 'localPart check should pass');
|
|
127
130
|
},
|
|
128
131
|
},
|
|
129
132
|
|
|
@@ -164,14 +167,17 @@ module.exports = {
|
|
|
164
167
|
},
|
|
165
168
|
|
|
166
169
|
{
|
|
167
|
-
name: 'localpart-letter-plus-numbers-
|
|
170
|
+
name: 'localpart-letter-plus-numbers-allowed',
|
|
168
171
|
timeout: 5000,
|
|
169
172
|
|
|
170
173
|
async run({ assert }) {
|
|
174
|
+
// Short letter + numbers local parts are legitimate (real Gmail users like
|
|
175
|
+
// mi1925973, hk9526802) — the ^[a-z]{1,2}\d+$ pattern was removed in v5.5.6
|
|
176
|
+
// after NeverBounce confirmed real users were being blocked.
|
|
171
177
|
const result = await validate('a123@gmail.com');
|
|
172
178
|
|
|
173
|
-
assert.equal(result.valid,
|
|
174
|
-
assert.propertyEquals(result, 'checks.localPart.
|
|
179
|
+
assert.equal(result.valid, true, 'Single letter + numbers should be allowed');
|
|
180
|
+
assert.propertyEquals(result, 'checks.localPart.valid', true, 'localPart check should pass');
|
|
175
181
|
},
|
|
176
182
|
},
|
|
177
183
|
|
|
@@ -370,6 +376,49 @@ module.exports = {
|
|
|
370
376
|
},
|
|
371
377
|
},
|
|
372
378
|
|
|
379
|
+
// --- Typo domain checks ---
|
|
380
|
+
|
|
381
|
+
{
|
|
382
|
+
name: 'typo-gamil-blocked',
|
|
383
|
+
timeout: 5000,
|
|
384
|
+
|
|
385
|
+
async run({ assert }) {
|
|
386
|
+
const result = await validate('rachel.greene@gamil.com');
|
|
387
|
+
|
|
388
|
+
assert.equal(result.valid, false, 'gamil.com should be blocked as a typo of gmail.com');
|
|
389
|
+
assert.propertyEquals(result, 'checks.typo.valid', false, 'Typo check should fail');
|
|
390
|
+
assert.propertyEquals(result, 'checks.typo.matchedPrefix', 'gamil.', 'Should report the matched prefix');
|
|
391
|
+
assert.propertyEquals(result, 'checks.typo.reason', 'Likely misspelled domain', 'Should have human-readable reason');
|
|
392
|
+
},
|
|
393
|
+
},
|
|
394
|
+
|
|
395
|
+
{
|
|
396
|
+
name: 'typo-gmail-con-blocked',
|
|
397
|
+
timeout: 5000,
|
|
398
|
+
|
|
399
|
+
async run({ assert }) {
|
|
400
|
+
const result = await validate('rachel.greene@gmail.con');
|
|
401
|
+
|
|
402
|
+
assert.equal(result.valid, false, 'gmail.con should be blocked as a typo TLD');
|
|
403
|
+
assert.propertyEquals(result, 'checks.typo.valid', false, 'Typo check should fail');
|
|
404
|
+
},
|
|
405
|
+
},
|
|
406
|
+
|
|
407
|
+
{
|
|
408
|
+
name: 'typo-correct-domains-pass',
|
|
409
|
+
timeout: 5000,
|
|
410
|
+
|
|
411
|
+
async run({ assert }) {
|
|
412
|
+
const gmail = await validate('rachel.greene@gmail.com');
|
|
413
|
+
const hotmail = await validate('rachel.greene@hotmail.com');
|
|
414
|
+
|
|
415
|
+
assert.equal(gmail.valid, true, 'gmail.com should pass');
|
|
416
|
+
assert.propertyEquals(gmail, 'checks.typo.valid', true, 'Typo check should pass for gmail.com');
|
|
417
|
+
assert.equal(hotmail.valid, true, 'hotmail.com should pass');
|
|
418
|
+
assert.propertyEquals(hotmail, 'checks.typo.valid', true, 'Typo check should pass for hotmail.com');
|
|
419
|
+
},
|
|
420
|
+
},
|
|
421
|
+
|
|
373
422
|
// --- isCorporate helper ---
|
|
374
423
|
|
|
375
424
|
{
|
|
@@ -529,8 +578,51 @@ module.exports = {
|
|
|
529
578
|
timeout: 5000,
|
|
530
579
|
|
|
531
580
|
async run({ assert }) {
|
|
532
|
-
assert.deepEqual(DEFAULT_CHECKS, ['format', 'disposable', 'corporate', 'localPart'], 'DEFAULT_CHECKS should be
|
|
533
|
-
assert.deepEqual(ALL_CHECKS, ['format', 'disposable', 'corporate', 'localPart', 'mailbox'], 'ALL_CHECKS should
|
|
581
|
+
assert.deepEqual(DEFAULT_CHECKS, ['format', 'disposable', 'corporate', 'localPart', 'typo'], 'DEFAULT_CHECKS should be all free sync checks (no dns/mailbox)');
|
|
582
|
+
assert.deepEqual(ALL_CHECKS, ['format', 'disposable', 'corporate', 'localPart', 'typo', 'dns', 'mailbox'], 'ALL_CHECKS should add dns + mailbox');
|
|
583
|
+
},
|
|
584
|
+
},
|
|
585
|
+
|
|
586
|
+
// --- DNS check behavior ---
|
|
587
|
+
|
|
588
|
+
{
|
|
589
|
+
name: 'dns-not-in-default-checks',
|
|
590
|
+
timeout: 5000,
|
|
591
|
+
|
|
592
|
+
async run({ assert }) {
|
|
593
|
+
const result = await validate('rachel.greene@gmail.com');
|
|
594
|
+
|
|
595
|
+
assert.equal(result.valid, true, 'Should be valid');
|
|
596
|
+
assert.equal(result.checks.dns, undefined, 'DNS should not run with default checks (async/slow — opt-in)');
|
|
597
|
+
},
|
|
598
|
+
},
|
|
599
|
+
|
|
600
|
+
{
|
|
601
|
+
name: 'dns-valid-domain-passes',
|
|
602
|
+
timeout: 15000,
|
|
603
|
+
|
|
604
|
+
async run({ assert }) {
|
|
605
|
+
// Offline-safe: on network errors the dns check is skipped (valid stays true);
|
|
606
|
+
// only definitive no-MX/NXDOMAIN answers block.
|
|
607
|
+
const result = await validate('rachel.greene@gmail.com', { checks: ['format', 'dns'] });
|
|
608
|
+
|
|
609
|
+
assert.equal(result.valid, true, 'gmail.com should pass the DNS check');
|
|
610
|
+
assert.hasProperty(result, 'checks.dns', 'Should have dns check result');
|
|
611
|
+
},
|
|
612
|
+
},
|
|
613
|
+
|
|
614
|
+
{
|
|
615
|
+
name: 'dns-nonexistent-domain-fails',
|
|
616
|
+
timeout: 15000,
|
|
617
|
+
skip: !process.env.TEST_EXTENDED_MODE
|
|
618
|
+
? 'TEST_EXTENDED_MODE not set (requires live DNS resolution)'
|
|
619
|
+
: false,
|
|
620
|
+
|
|
621
|
+
async run({ assert }) {
|
|
622
|
+
const result = await validate('rachel.greene@thisdomaindoesnotexist99887766.com', { checks: ['format', 'dns'] });
|
|
623
|
+
|
|
624
|
+
assert.equal(result.valid, false, 'Nonexistent domain should fail the DNS check');
|
|
625
|
+
assert.propertyEquals(result, 'checks.dns.valid', false, 'DNS check should fail');
|
|
534
626
|
},
|
|
535
627
|
},
|
|
536
628
|
|
|
@@ -12,6 +12,8 @@
|
|
|
12
12
|
*
|
|
13
13
|
* SendGrid processor tests:
|
|
14
14
|
* - Various event types (group_unsubscribe, unsubscribe, spamreport, bounce, dropped)
|
|
15
|
+
* - bounce/dropped only revoke on bounce_classification='Invalid Address' (hard bounce);
|
|
16
|
+
* technical bounces are sender-side issues and must NOT revoke consent
|
|
15
17
|
* - Email lookup → user doc mutation with source='sendgrid'
|
|
16
18
|
* - Silent skip when email doesn't map to a user (shared SendGrid account scenario)
|
|
17
19
|
* - Batched events processed independently
|
|
@@ -25,13 +27,14 @@ function sgEventId(name) {
|
|
|
25
27
|
}
|
|
26
28
|
|
|
27
29
|
// Helper — build a SendGrid event payload
|
|
28
|
-
function sgEvent({ id, type, email, timestamp, asmGroupId }) {
|
|
30
|
+
function sgEvent({ id, type, email, timestamp, asmGroupId, bounceClassification }) {
|
|
29
31
|
return {
|
|
30
32
|
sg_event_id: id,
|
|
31
33
|
event: type,
|
|
32
34
|
email,
|
|
33
35
|
timestamp: timestamp || Math.floor(Date.now() / 1000),
|
|
34
36
|
...(asmGroupId !== undefined ? { asm_group_id: asmGroupId } : {}),
|
|
37
|
+
...(bounceClassification !== undefined ? { bounce_classification: bounceClassification } : {}),
|
|
35
38
|
};
|
|
36
39
|
}
|
|
37
40
|
|
|
@@ -167,20 +170,43 @@ module.exports = {
|
|
|
167
170
|
},
|
|
168
171
|
|
|
169
172
|
{
|
|
170
|
-
name: 'sendgrid-bounce-event-handled',
|
|
173
|
+
name: 'sendgrid-hard-bounce-event-handled',
|
|
171
174
|
auth: 'none',
|
|
172
175
|
async run({ http, firestore, assert, accounts }) {
|
|
173
176
|
const uid = accounts.basic.uid;
|
|
174
177
|
const email = accounts.basic.email;
|
|
175
|
-
const eventId = sgEventId('bounce');
|
|
178
|
+
const eventId = sgEventId('hard-bounce');
|
|
176
179
|
|
|
180
|
+
// Only hard bounces (bounce_classification='Invalid Address') revoke consent.
|
|
177
181
|
const response = await http.as('none').post(
|
|
178
182
|
`backend-manager/marketing/webhook?provider=sendgrid&key=${process.env.BACKEND_MANAGER_WEBHOOK_KEY}`,
|
|
179
|
-
[sgEvent({ id: eventId, type: 'bounce', email })]
|
|
183
|
+
[sgEvent({ id: eventId, type: 'bounce', email, bounceClassification: 'Invalid Address' })]
|
|
184
|
+
);
|
|
185
|
+
|
|
186
|
+
assert.isSuccess(response);
|
|
187
|
+
assert.propertyEquals(response, 'data.processed', 1, 'Hard bounce should be treated as a revoke');
|
|
188
|
+
|
|
189
|
+
const userDoc = await firestore.get(`users/${uid}`);
|
|
190
|
+
assert.equal(userDoc?.consent?.marketing?.status, 'revoked');
|
|
191
|
+
},
|
|
192
|
+
},
|
|
193
|
+
|
|
194
|
+
{
|
|
195
|
+
name: 'sendgrid-dropped-hard-bounce-handled',
|
|
196
|
+
auth: 'none',
|
|
197
|
+
async run({ http, firestore, assert, accounts }) {
|
|
198
|
+
const uid = accounts.basic.uid;
|
|
199
|
+
const email = accounts.basic.email;
|
|
200
|
+
const eventId = sgEventId('dropped');
|
|
201
|
+
|
|
202
|
+
// 'dropped' follows the same classification filter as 'bounce'.
|
|
203
|
+
const response = await http.as('none').post(
|
|
204
|
+
`backend-manager/marketing/webhook?provider=sendgrid&key=${process.env.BACKEND_MANAGER_WEBHOOK_KEY}`,
|
|
205
|
+
[sgEvent({ id: eventId, type: 'dropped', email, bounceClassification: 'Invalid Address' })]
|
|
180
206
|
);
|
|
181
207
|
|
|
182
208
|
assert.isSuccess(response);
|
|
183
|
-
assert.propertyEquals(response, 'data.processed', 1, '
|
|
209
|
+
assert.propertyEquals(response, 'data.processed', 1, 'Dropped with Invalid Address should be treated as a revoke');
|
|
184
210
|
|
|
185
211
|
const userDoc = await firestore.get(`users/${uid}`);
|
|
186
212
|
assert.equal(userDoc?.consent?.marketing?.status, 'revoked');
|
|
@@ -189,6 +215,45 @@ module.exports = {
|
|
|
189
215
|
|
|
190
216
|
// ─── SendGrid processor — events we ignore ───
|
|
191
217
|
|
|
218
|
+
{
|
|
219
|
+
name: 'sendgrid-technical-bounce-ignored',
|
|
220
|
+
auth: 'none',
|
|
221
|
+
async run({ http, assert, accounts }) {
|
|
222
|
+
const email = accounts.basic.email;
|
|
223
|
+
const eventId = sgEventId('technical-bounce');
|
|
224
|
+
|
|
225
|
+
// Technical bounces (DMARC, TLS, DNS) are sender-side issues — the recipient's
|
|
226
|
+
// mailbox is still valid, so consent must NOT be revoked.
|
|
227
|
+
const response = await http.as('none').post(
|
|
228
|
+
`backend-manager/marketing/webhook?provider=sendgrid&key=${process.env.BACKEND_MANAGER_WEBHOOK_KEY}`,
|
|
229
|
+
[sgEvent({ id: eventId, type: 'bounce', email, bounceClassification: 'Technical Failure' })]
|
|
230
|
+
);
|
|
231
|
+
|
|
232
|
+
assert.isSuccess(response, 'Should accept the request (not error) but ignore the event');
|
|
233
|
+
assert.propertyEquals(response, 'data.processed', 0, 'Technical bounce should not be processed');
|
|
234
|
+
assert.propertyEquals(response, 'data.skipped', 1, '1 event should be skipped');
|
|
235
|
+
},
|
|
236
|
+
},
|
|
237
|
+
|
|
238
|
+
{
|
|
239
|
+
name: 'sendgrid-bounce-without-classification-ignored',
|
|
240
|
+
auth: 'none',
|
|
241
|
+
async run({ http, assert, accounts }) {
|
|
242
|
+
const email = accounts.basic.email;
|
|
243
|
+
const eventId = sgEventId('unclassified-bounce');
|
|
244
|
+
|
|
245
|
+
// No bounce_classification — can't confirm a hard bounce, so skip.
|
|
246
|
+
const response = await http.as('none').post(
|
|
247
|
+
`backend-manager/marketing/webhook?provider=sendgrid&key=${process.env.BACKEND_MANAGER_WEBHOOK_KEY}`,
|
|
248
|
+
[sgEvent({ id: eventId, type: 'bounce', email })]
|
|
249
|
+
);
|
|
250
|
+
|
|
251
|
+
assert.isSuccess(response, 'Should accept the request (not error) but ignore the event');
|
|
252
|
+
assert.propertyEquals(response, 'data.processed', 0, 'Unclassified bounce should not be processed');
|
|
253
|
+
assert.propertyEquals(response, 'data.skipped', 1, '1 event should be skipped');
|
|
254
|
+
},
|
|
255
|
+
},
|
|
256
|
+
|
|
192
257
|
{
|
|
193
258
|
name: 'sendgrid-delivered-event-ignored',
|
|
194
259
|
auth: 'none',
|