backend-manager 5.2.2 → 5.2.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -14,6 +14,39 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
14
14
  - `Fixed` for any bug fixes.
15
15
  - `Security` in case of vulnerabilities.
16
16
 
17
+ # [5.2.5] - 2026-05-22
18
+
19
+ ### Added
20
+
21
+ - **`utilities.trim(input)`** in `src/manager/helpers/utilities.js`. Walks objects/arrays recursively and trims whitespace on every string. Does NOT strip HTML. Middleware uses it to clean up incoming request data without mangling URLs, Markdown, or any payload with `<`/`>`/`&`.
22
+ - **`findContact(email)`** on the SendGrid + Beehiiv providers, and **`findSubscriber(email, publicationId)`** on Beehiiv. Both extracted from the existing `removeContact`/`removeSubscriber` search-by-email logic so tests (and other call sites) can verify provider state without forcing a delete.
23
+ - **`test/marketing/consent-lifecycle.js`** — 5-phase live integration test (gated on `TEST_EXTENDED_MODE=true`) that walks two long-lived sentinel accounts through pre-check → sync → declined-stays-out → unsubscribe → validation-gate, asserting actual SendGrid + Beehiiv state at every step. Uses `pollProvider()` to handle SendGrid's async background jobs (upsert/delete can take 10-60s+ to surface).
24
+
25
+ ### Changed
26
+
27
+ - **Middleware no longer strips HTML by default.** The auto-`sanitize-html` pass on incoming request data was mangling legitimate input — URL query strings (`?a=1&b=2` → `?a=1&amp;b=2`), Markdown, and any payload containing `<`/`>`/`&`. The middleware now only trims whitespace by default. HTML sanitization is opt-in per route via `Manager.Middleware(req, res).run('route', { sanitize: true })`. Sanitize at the HTML-insertion site (`utilities.sanitize()` in your template/email render) rather than at the request boundary. The existing schema-level `sanitize: false` opt-out still works when route-level sanitize is on.
28
+ - **Test sentinel accounts `consent-granted` and `consent-declined` now use the `_test.allow_*` prefix** (`_test.allow_consent-granted@...`, `_test.allow_consent-declined@...`). The `_test.*` validation block from v5.2.3 also blocks the previous `_test.consent-*` names from reaching providers — `_test.allow_*` is the carved-out exception specifically for live-provider integration tests.
29
+ - **Beehiiv API timeouts bumped to 60s** on the two remaining stragglers (`findSubscriber`, `resolveSegmentIds`) to match the SendGrid+Beehiiv 60s convention from v5.2.3.
30
+
31
+ ### Removed
32
+
33
+ - **`cleanupMarketingProviders()` function and its pre-run/post-run hooks** in `src/test/test-accounts.js` + `src/test/runner.js`. No longer needed — the validation gate added in v5.2.3 blocks `_test.*` emails from reaching SendGrid + Beehiiv upstream, so there's nothing to clean up. The new `consent-lifecycle.js` test (which intentionally round-trips real contacts via `_test.allow_*`) manages its own pre-check force-clean inline.
34
+
35
+ # [5.2.3] - 2026-05-22
36
+
37
+ ### Added
38
+
39
+ - **`marketing.sendgrid.listId` in `templates/backend-manager-config.json`.** Empty-string placeholder for OMEGA's `sendgrid/ensure/list.js` to populate at brand-onboarding time. Mirrors the existing `marketing.beehiiv.publicationId` convention.
40
+ - **`_test.*` local-part block** in `src/manager/libraries/email/data/blocked-local-patterns.js`. Test-suite accounts (`_test.<scenario>@somiibo.com`) are now blocked from reaching SendGrid + Beehiiv. The carved-out exception is `_test.allow_*` — used for live-provider integration tests that intentionally need to round-trip a real contact.
41
+
42
+ ### Changed
43
+
44
+ - **`Marketing.add()` and `Marketing.sync()` now use the full `validate()` pipeline** instead of just `isCorporate()`. Single SSOT for "is this a valid marketing email" — runs format → disposable → corporate → localPart in one call. Stricter behavior: disposable-domain emails (mailinator etc.) and junk local-parts (`noreply`, `test*`, `_test.*`) are now blocked from marketing lists. They were previously waved through because the gate only checked corporate domains.
45
+ - **SendGrid list-ID lookup is now config-only.** `src/manager/libraries/email/providers/sendgrid.js#getListId()` reads `Manager.config.marketing.sendgrid.listId` and returns null if missing — no more runtime API call, no more fuzzy-match-by-brand-name. Old fuzzy logic kept commented out as `getListIdByFuzzyMatch()` backstop. **Brands must run OMEGA's sendgrid service to populate `listId` before this version sees their list assignments work** (without it, contacts land in SendGrid's global "All Contacts" pool, not the brand list).
46
+ - **Beehiiv publication-ID lookup is now config-only.** `src/manager/libraries/email/providers/beehiiv.js#getPublicationId()` reads `Manager.config.marketing.beehiiv.publicationId` and returns null if missing — same shape as SendGrid above. Old fuzzy logic kept commented out as backstop. Beehiiv side already preferred config when set, but now there's no API fallback.
47
+ - **`marketing.beehiiv.publicationId` is now an always-present empty string in the config template** (was a commented-out hint). Matches the SendGrid `listId` shape and means `Manager.config.marketing.beehiiv.publicationId` always returns `""` (never `undefined`) for legacy brands.
48
+ - **SendGrid API timeouts bumped from 10s → 60s** via a new top-level `SENDGRID_TIMEOUT_MS` constant in `sendgrid.js`. All 9 fetch sites updated. Catches the intermittent SendGrid backend hiccups that were dropping signups silently with "Request timed out".
49
+
17
50
  # [5.2.2] - 2026-05-21
18
51
 
19
52
  ### Added
package/CLAUDE.md CHANGED
@@ -76,7 +76,7 @@ See [docs/cli-firestore-auth.md](docs/cli-firestore-auth.md) and [docs/cli-logs.
76
76
  - **Firestore shorthand**: `admin.firestore().doc('users/abc123')` (path string) rather than `.collection('users').doc('abc123')`.
77
77
  - **Template strings for requires**: `` require(`${functionsDir}/node_modules/backend-manager`) `` rather than string concat.
78
78
  - **No backwards compatibility** unless explicitly requested.
79
- - **Routes receive sanitized data by default** see [docs/sanitization.md](docs/sanitization.md) for opt-out rules.
79
+ - **Routes receive whitespace-trimmed data; HTML is preserved.** Sanitize at the HTML-insertion site via `utilities.sanitize()`. Opt into middleware-level HTML strip per-route with `{ sanitize: true }`. See [docs/sanitization.md](docs/sanitization.md).
80
80
  - **Match schema names to route names** — if route is `myEndpoint`, schema is `myEndpoint`.
81
81
  - **Always use `assistant.respond()` for responses** — do NOT use `res.send()` directly.
82
82
  - **Add Firestore composite indexes** for any compound query (`where` + `orderBy`, or multiple `where`s) to `src/cli/commands/setup-tests/helpers/required-indexes.js` (the SSOT). Without the index, queries crash with `FAILED_PRECONDITION` in production.
@@ -113,7 +113,7 @@ Deep references live in `docs/`. **Whenever you make a behavioral change, update
113
113
 
114
114
  - [docs/routes.md](docs/routes.md) — recipes for new API commands, routes, event handlers, cron jobs (with code templates)
115
115
  - [docs/schemas.md](docs/schemas.md) — schema definition format, defaults vs premium overrides
116
- - [docs/sanitization.md](docs/sanitization.md) — automatic XSS sanitization, schema opt-out (`sanitize: false`), route-level opt-out, manual `utilities.sanitize()`
116
+ - [docs/sanitization.md](docs/sanitization.md) — middleware trim-only default; opt-in HTML strip (`{ sanitize: true }`) with per-field opt-out (`sanitize: false`); manual `utilities.sanitize()` for HTML-insertion sites
117
117
  - [docs/auth-hooks.md](docs/auth-hooks.md) — consumer hooks for `before-create`/`before-signin`/`on-create`/`on-delete` (blocking + non-blocking examples)
118
118
  - [docs/common-operations.md](docs/common-operations.md) — inside-the-handler patterns: authenticate, read/write Firestore, error handling, send response, `bm_api` hook
119
119
 
@@ -134,6 +134,6 @@ Deep references live in `docs/`. **Whenever you make a behavioral change, update
134
134
 
135
135
  ### Testing & CLI
136
136
 
137
- - [docs/testing.md](docs/testing.md) — running, filtering, log files, test types (standalone/suite/group), context object, assertions, auth levels. **For Firestore/Auth/local state, cleanup runs at the START of every run, never at the end** if you add a test that writes Firestore data, register the collection/namespace in the runner's pre-test wipe list, don't add a trailing cleanup step. **Exception:** third-party providers we can't wipe at start (e.g. SendGrid/Beehiiv contact lists) get a symmetric pre + post cleanup hook in the runner see `cleanupMarketingProviders` and [docs/consent.md](docs/consent.md). Don't pattern-match this exception for new Firestore code.
137
+ - [docs/testing.md](docs/testing.md) — running, filtering, log files, test types (standalone/suite/group), context object, assertions, auth levels. **All cleanup runs at the START of every run, never at the end.** If you add a test that writes Firestore data, register the collection/namespace in the runner's pre-test wipe list, don't add a trailing cleanup step. Marketing providers (SendGrid/Beehiiv) don't need a special exception `_test.*` emails are blocked at the validation layer so test signups never reach providers. The `_test.allow_*` carve-out exists only for the live-provider lifecycle test (`test/marketing/consent-lifecycle.js`), which manages its own teardown.
138
138
  - [docs/cli-firestore-auth.md](docs/cli-firestore-auth.md) — `npx mgr firestore:*` and `auth:*` commands, shared flags, examples
139
139
  - [docs/cli-logs.md](docs/cli-logs.md) — `npx mgr logs:read` / `logs:tail` with full flag reference and built-in Cloud Function names
package/docs/consent.md CHANGED
@@ -138,7 +138,7 @@ Each processor's `handleEvent` does the same shape of work:
138
138
  | SendGrid | `unsubscribe`, `group_unsubscribe`, `spamreport`, `bounce`, `dropped` |
139
139
  | Beehiiv | `subscription.unsubscribed`, `subscription.deleted`, `subscription.paused` |
140
140
 
141
- **Beehiiv publication filter.** Each Beehiiv event includes a `publication_id`. The processor compares this against the result of `beehiivProvider.getPublicationId()` (which reads `Manager.config.marketing.beehiiv.publicationId` or fuzzy-matches against the Beehiiv API by brand name). Mismatch → silent skip. This is how shared-publication events (e.g. devbeans shared by 6 brands) get routed correctly — each brand processes only events matching its own publication.
141
+ **Beehiiv publication filter.** Each Beehiiv event includes a `publication_id`. The processor compares this against `beehiivProvider.getPublicationId()`, which reads `Manager.config.marketing.beehiiv.publicationId` (populated at brand-onboarding time by OMEGA's `beehiiv/ensure/publication.js`). Mismatch → silent skip. This is how shared-publication events (e.g. devbeans shared by 6 brands) get routed correctly — each brand processes only events matching its own publication. Brands without `publicationId` in config silently skip all Beehiiv webhook events. The same convention applies to SendGrid: `marketing.sendgrid.listId` is populated by OMEGA's `sendgrid/ensure/list.js`.
142
142
 
143
143
  ## Parent forwarder (Phase E)
144
144
 
@@ -304,18 +304,13 @@ After the migration: optionally run a re-opt-in drip campaign to legally recover
304
304
 
305
305
  Run with `npx mgr test` (full suite) or `npx mgr test routes/marketing/webhook` (just the webhook tests).
306
306
 
307
- ### Provider-side cleanup (extended mode only)
307
+ ### Live-provider tests (extended mode only)
308
308
 
309
- Normally BEM test runs are self-contained because all data lives in the local emulator. Marketing tests are the exception when `TEST_EXTENDED_MODE=true` is set, the routes make real API calls to SendGrid + Beehiiv and leave real contacts behind.
309
+ Most BEM tests are self-contained against the local emulator. The marketing-consent system has one test that's an exception [test/marketing/consent-lifecycle.js](../test/marketing/consent-lifecycle.js) which makes real API calls to SendGrid + Beehiiv to verify the full round-trip works end-to-end.
310
310
 
311
- To keep this self-contained, the runner calls [`cleanupMarketingProviders`](../src/test/test-accounts.js) **twice** per run:
311
+ The validation pipeline (`src/manager/libraries/email/validation.js`) blocks all `_test.*` emails from reaching providers via the `/^_test\.(?!allow_)/` pattern in `blocked-local-patterns.js`. The two `_test.allow_*` sentinels (`_test.allow_consent-granted` and `_test.allow_consent-declined`) used by the lifecycle test bypass that gate intentionally, and the test cleans up after itself (phase-3 removes the granted contact via `Manager.Email().remove()`).
312
312
 
313
- 1. **Pre-run** scrubs leftovers from any previous run that crashed mid-suite.
314
- 2. **Post-run** — scrubs everything this run created. Best-effort (wrapped in try/catch — cleanup failures don't change the test result).
315
-
316
- Both fire ONLY when `TEST_EXTENDED_MODE=true`. In normal mode (the default), no provider calls happen and no cleanup is needed.
317
-
318
- This is the one exception to the "Firestore cleanup runs at start only" rule documented in [docs/testing.md](testing.md) — we can't wipe SendGrid/Beehiiv state in our own Firestore wipe, so we have to ask the providers to do it for us, twice for defense in depth.
313
+ The "all cleanup runs at start, never at the end" rule documented in [docs/testing.md](testing.md) applies to all test data, including third-party providers.
319
314
 
320
315
  ## Frontend pieces (cross-references)
321
316
 
@@ -1,16 +1,25 @@
1
1
  # Sanitization (XSS Prevention)
2
2
 
3
- BEM automatically sanitizes all incoming request data stripping HTML tags and trimming whitespace from every string field. This happens in the middleware pipeline before route handlers execute, so **routes receive clean data by default**.
3
+ BEM middleware always **trims** whitespace on incoming string fields (via `utilities.trim()`). HTML sanitization is **opt-in** it's not run by default because it mangles legitimate input like URL query strings (`&` `&amp;`) and Markdown.
4
+
5
+ The expectation is that you sanitize at the **HTML-insertion site** (in the template, in the email body, etc.) — not at the request boundary.
4
6
 
5
7
  ## How It Works
6
8
 
7
- 1. **Schema fields**: Sanitized per-field during the middleware pipeline. Fields can opt out with `sanitize: false` in the schema.
8
- 2. **Non-schema fields** (when `setupSettings: false` or `includeNonSchemaSettings: true`): All strings are sanitized with no opt-out.
9
- 3. The middleware uses `Manager.Utilities().sanitize()` under the hood.
9
+ 1. **Trimming**: Every string in `settings` is whitespace-trimmed by the middleware (objects + arrays walked recursively). Always on.
10
+ 2. **HTML sanitization**: Off by default. Opt in per-route with `{ sanitize: true }` on `Manager.Middleware(...).run(...)`.
11
+ 3. **Schemas** can mark individual fields with `sanitize: false` to skip the HTML strip for that field when route-level sanitize is enabled (used for fields that legitimately need raw HTML — rich-text editors, email templates).
12
+
13
+ ## Route-Level Opt-In
14
+
15
+ ```javascript
16
+ // In functions/index.js — enable HTML strip for a specific route
17
+ Manager.Middleware(req, res).run('my-route', { sanitize: true });
18
+ ```
10
19
 
11
- ## Schema Opt-Out
20
+ When enabled, every string in `settings` is run through `sanitize-html` (strip all tags) unless the schema marks the field with `sanitize: false`.
12
21
 
13
- For fields that legitimately need HTML (rich text, email templates, etc.), set `sanitize: false` in the schema:
22
+ ## Schema Field Opt-Out (when route-level sanitize is on)
14
23
 
15
24
  ```javascript
16
25
  // This field will NOT be sanitized — raw HTML is preserved
@@ -19,43 +28,42 @@ htmlContent: {
19
28
  default: '',
20
29
  sanitize: false,
21
30
  },
22
- // This field IS sanitized (default behavior, no flag needed)
31
+ // This field IS sanitized when route-level sanitize is enabled
23
32
  name: {
24
33
  types: ['string'],
25
34
  default: '',
26
35
  },
27
36
  ```
28
37
 
29
- ## Route-Level Opt-Out
38
+ ## Manual Sanitization (Recommended)
30
39
 
31
- Disable sanitization entirely for a route (rare only for routes that handle raw HTML everywhere):
32
-
33
- ```javascript
34
- // In functions/index.js
35
- Manager.Middleware(req, res).run('my-route', { sanitize: false });
36
- ```
37
-
38
- ## Manual Sanitization (Outside Middleware)
39
-
40
- For cron jobs, event handlers, or anywhere outside the request pipeline, use `utilities.sanitize()` directly:
40
+ For most use cases particularly anywhere you're inserting user-supplied content into HTML call `utilities.sanitize()` directly at the insertion site:
41
41
 
42
42
  ```javascript
43
43
  // Available in route context
44
- const clean = utilities.sanitize(untrustedData);
44
+ const safeHtml = utilities.sanitize(untrustedData);
45
45
 
46
46
  // Or via Manager
47
- const clean = Manager.Utilities().sanitize(untrustedData);
47
+ const safeHtml = Manager.Utilities().sanitize(untrustedData);
48
48
  ```
49
49
 
50
50
  Accepts any data type — strings, objects, arrays, primitives. Walks objects/arrays recursively, strips HTML from strings, passes everything else through unchanged.
51
51
 
52
- ## Route Handler Context
52
+ ## Why HTML Sanitization Is Not the Middleware Default
53
+
54
+ Stripping HTML from every incoming string at the request boundary is too aggressive — it corrupts legitimate input:
53
55
 
54
- The middleware injects these into every route handler:
56
+ - URL query strings: `https://example.com/?a=1&b=2` becomes `https://example.com/?a=1&amp;b=2`
57
+ - Markdown / code snippets with `<`, `>`, `&` characters get mangled
58
+ - API payloads round-tripped through the system get silently rewritten
59
+
60
+ Stored XSS comes from rendering, not from receiving. Sanitize at the render site (where you know the output context — HTML body, attribute, URL, JSON) instead of at the front door.
61
+
62
+ ## Route Handler Context
55
63
 
56
64
  ```javascript
57
65
  module.exports = async ({ Manager, assistant, analytics, usage, user, settings, libraries, utilities }) => {
58
- // settings — already sanitized by middleware
59
- // utilities — Manager.Utilities() instance for manual sanitization
66
+ // settings — whitespace-trimmed by middleware; HTML preserved unless route opts in via { sanitize: true }
67
+ // utilities — Manager.Utilities() instance for manual sanitize()/trim()
60
68
  };
61
69
  ```
package/docs/schemas.md CHANGED
@@ -36,4 +36,4 @@ module.exports = function (assistant, settings, options) {
36
36
 
37
37
  ## Field Sanitization
38
38
 
39
- By default, every string field in a schema is sanitized (HTML tags stripped, whitespace trimmed) during the middleware pipeline. To preserve raw HTML (rich text, email templates), set `sanitize: false` on the field. See [docs/sanitization.md](sanitization.md).
39
+ Middleware always trims whitespace on string fields. HTML sanitization is **opt-in per route** — `Manager.Middleware(req, res).run('my-route', { sanitize: true })`. When opted in, fields can individually opt back out with `sanitize: false`. See [docs/sanitization.md](sanitization.md).
package/docs/testing.md CHANGED
@@ -19,18 +19,19 @@ What the runner wipes pre-test (in [src/test/test-accounts.js](../src/test/test-
19
19
 
20
20
  1. **`meta/stats`** doc ensured (required for on-create batch writes).
21
21
  2. **`users/_test-*`** Firebase Auth users + Firestore docs (delete).
22
- 3. **Marketing providers** (SendGrid + Beehiiv) leftover test contacts removed. Runs only under `TEST_EXTENDED_MODE`. Runs **before** test users are recreated so a killed run can't leave a contact pinned to an about-to-be-recreated uid.
23
- 4. **Mixed Firestore collections** — `payments-orders`, `payments-webhooks`, `payments-intents`, `payments-disputes`, `marketing-webhooks`. Two-pass cleanup per collection:
22
+ 3. **Mixed Firestore collections** — `payments-orders`, `payments-webhooks`, `payments-intents`, `payments-disputes`, `marketing-webhooks`. Two-pass cleanup per collection:
24
23
  - Pass 1 — owner-keyed: `where('owner', 'in', [...testUids])` (batched at 30 uids per `in` query).
25
24
  - Pass 2 — id-keyed: any doc whose ID starts with `_test-` (catches ownerless test docs like dispute alerts and raw test webhooks).
26
- 5. **Test-only Firestore collections** — `_test`, `_test_query` — wiped in full.
27
- 6. **Realtime Database** — the `_test` namespace removed in full (`admin.database().ref('_test').remove()`).
25
+ 4. **Test-only Firestore collections** — `_test`, `_test_query` — wiped in full.
26
+ 5. **Realtime Database** — the `_test` namespace removed in full (`admin.database().ref('_test').remove()`).
28
27
 
29
- ### The one exception: third-party provider cleanup runs at start AND end
28
+ ### Marketing-provider cleanup
30
29
 
31
- The `cleanupMarketingProviders` call also fires **after** the suite completes (extended mode only). Reason: pre-run cleanup catches what a crashed previous run left behind, but during a normal-completion run we'd still leak real SendGrid/Beehiiv contacts until the NEXT run cleaned them up. By running cleanup post-suite too, each extended-mode run leaves the provider lists in the same state it found them.
30
+ Test signups never reach SendGrid + Beehiiv. The validation pipeline (`src/manager/libraries/email/validation.js`) blocks all `_test.*` emails at the marketing-library layer via the `/^_test\.(?!allow_)/` pattern in `blocked-local-patterns.js`.
32
31
 
33
- This is **specifically** for third-party state that lives outside the local emulator. **Do not use trailing cleanup for Firestore or Auth data** those follow the start-only rule because we control the local state and pre-run cleanup is enough.
32
+ The single exception is the `_test.allow_*` prefix. Two long-lived test accounts (`_test.allow_consent-granted@...` and `_test.allow_consent-declined@...`) intentionally round-trip through SendGrid + Beehiiv as the live-provider integration sentinels. They are exercised by `test/marketing/consent-lifecycle.js`, which manages its own setup, assertions, and teardown.
33
+
34
+ All cleanup follows the start-only rule. No trailing-cleanup exception.
34
35
 
35
36
  ### When adding a new test that writes data
36
37
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "backend-manager",
3
- "version": "5.2.2",
3
+ "version": "5.2.5",
4
4
  "description": "Quick tools for developing Firebase functions",
5
5
  "main": "src/manager/index.js",
6
6
  "bin": {
@@ -35,7 +35,7 @@ Middleware.prototype.run = function (libPath, options) {
35
35
  options.setupAnalytics = typeof options.setupAnalytics === 'boolean' ? options.setupAnalytics : true;
36
36
  options.setupUsage = typeof options.setupUsage === 'boolean' ? options.setupUsage : true;
37
37
  options.setupSettings = typeof options.setupSettings === 'undefined' ? true : options.setupSettings;
38
- options.sanitize = typeof options.sanitize === 'undefined' ? true : options.sanitize;
38
+ options.sanitize = typeof options.sanitize === 'undefined' ? false : options.sanitize;
39
39
  options.includeNonSchemaSettings = typeof options.includeNonSchemaSettings === 'undefined' ? false : options.includeNonSchemaSettings;
40
40
  options.schema = typeof options.schema === 'undefined' ? libPath : options.schema;
41
41
  options.parseMultipartFormData = typeof options.parseMultipartFormData === 'undefined' ? true : options.parseMultipartFormData;
@@ -177,13 +177,16 @@ Middleware.prototype.run = function (libPath, options) {
177
177
  assistant.settings = data;
178
178
  }
179
179
 
180
- // Sanitize settings trim whitespace and strip HTML from all strings
181
- // Respects sanitize: false on individual schema fields
180
+ // Trim whitespace on all string settings (always on harmless and useful).
181
+ assistant.settings = Manager.Utilities().trim(assistant.settings);
182
+
183
+ // Optional HTML strip (off by default — opt in with `{ sanitize: true }`).
184
+ // Sanitize at the HTML-insertion site instead unless you need a belt-and-suspenders pass here.
185
+ // Respects sanitize: false on individual schema fields.
182
186
  if (options.sanitize) {
183
187
  const schema = options.setupSettings ? Manager.Settings().schema : null;
184
188
  const utilities = Manager.Utilities();
185
189
 
186
- // Walk settings, skipping fields the schema marks as sanitize: false
187
190
  assistant.settings = sanitizeWithSchema(utilities, assistant.settings, schema);
188
191
  }
189
192
 
@@ -528,4 +528,35 @@ Utilities.prototype.sanitize = function (input) {
528
528
  return input;
529
529
  };
530
530
 
531
+ /**
532
+ * Trim whitespace from all strings in input. Walks objects/arrays recursively.
533
+ * Does NOT strip HTML — use sanitize() for that.
534
+ *
535
+ * @param {*} input - The data to trim (string, object, array, or primitive)
536
+ * @returns {*} Trimmed copy (objects/arrays) or trimmed value (strings)
537
+ */
538
+ Utilities.prototype.trim = function (input) {
539
+ if (input == null) {
540
+ return input;
541
+ }
542
+
543
+ if (typeof input === 'string') {
544
+ return input.trim();
545
+ }
546
+
547
+ if (Array.isArray(input)) {
548
+ return input.map(item => this.trim(item));
549
+ }
550
+
551
+ if (typeof input === 'object') {
552
+ const result = {};
553
+ for (const [key, value] of Object.entries(input)) {
554
+ result[key] = this.trim(value);
555
+ }
556
+ return result;
557
+ }
558
+
559
+ return input;
560
+ };
561
+
531
562
  module.exports = Utilities;
@@ -3,9 +3,14 @@
3
3
  * Kept as JS (not JSON) so patterns stay as native RegExp literals.
4
4
  */
5
5
  module.exports = [
6
- /^\d+$/, // All numeric: 123456
7
- /^(.)\1{2,}$/, // Repeating single char: aaaa, xxxx
8
- /^[a-z]{1,2}\d+$/, // Single letter + numbers: a123, x999
9
- /^test/, // Starts with test: test, testuser, test123, test.user
10
- /^example/, // Starts with example: example, exampleuser, example.user
6
+ /^\d+$/, // All numeric: 123456
7
+ /^(.)\1{2,}$/, // Repeating single char: aaaa, xxxx
8
+ /^[a-z]{1,2}\d+$/, // Single letter + numbers: a123, x999
9
+ /^test/, // Starts with test: test, testuser, test123, test.user
10
+ /^example/, // Starts with example: example, exampleuser, example.user
11
+ // Test-suite accounts use `_test.<scenario>@...` so they don't collide with
12
+ // real signups. Block them from reaching SendGrid/Beehiiv so live lists stay
13
+ // clean. `_test.allow_*` is the carved-out exception for live-provider
14
+ // integration tests that intentionally need to round-trip a real contact.
15
+ /^_test\.(?!allow_)/,
11
16
  ];
@@ -29,7 +29,7 @@ const md = new MarkdownIt({ html: true, breaks: true, linkify: true });
29
29
 
30
30
  const { TEMPLATES, GROUPS, SENDERS } = require('../constants.js');
31
31
  const { tagLinks } = require('../utm.js');
32
- const { isCorporate } = require('../validation.js');
32
+ const { validate } = require('../validation.js');
33
33
  const sendgridProvider = require('../providers/sendgrid.js');
34
34
  const beehiivProvider = require('../providers/beehiiv.js');
35
35
 
@@ -73,9 +73,13 @@ Marketing.prototype.add = async function (options) {
73
73
  return {};
74
74
  }
75
75
 
76
- if (isCorporate(email)) {
77
- assistant.warn(`Marketing.add(): Blocked corporate/social-media domain, skipping: ${email}`);
78
- return { blocked: 'corporate', email };
76
+ // SSOT for what "valid marketing email" means — format, disposable, corporate,
77
+ // localPart junk (incl. _test.* test-suite accounts). Single validate() call
78
+ // catches all of these before we hit SendGrid/Beehiiv.
79
+ const validation = await validate(email);
80
+ if (!validation.valid) {
81
+ assistant.warn(`Marketing.add(): Validation failed, skipping: ${email}`, validation.checks);
82
+ return { blocked: 'validation', email, checks: validation.checks };
79
83
  }
80
84
 
81
85
  if (assistant.isTesting() && !process.env.TEST_EXTENDED_MODE) {
@@ -157,9 +161,13 @@ Marketing.prototype.sync = async function (userDocOrUid) {
157
161
  return {};
158
162
  }
159
163
 
160
- if (isCorporate(email)) {
161
- assistant.warn(`Marketing.sync(): Blocked corporate/social-media domain, skipping: ${email}`);
162
- return { blocked: 'corporate', email };
164
+ // SSOT for what "valid marketing email" means — format, disposable, corporate,
165
+ // localPart junk (incl. _test.* test-suite accounts). Single validate() call
166
+ // catches all of these before we hit SendGrid/Beehiiv.
167
+ const validation = await validate(email);
168
+ if (!validation.valid) {
169
+ assistant.warn(`Marketing.sync(): Validation failed, skipping: ${email}`, validation.checks);
170
+ return { blocked: 'validation', email, checks: validation.checks };
163
171
  }
164
172
 
165
173
  if (assistant.isTesting() && !process.env.TEST_EXTENDED_MODE) {