backend-manager 5.0.203 → 5.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (80) hide show
  1. package/CHANGELOG.md +72 -0
  2. package/CLAUDE.md +100 -1529
  3. package/TODO-CHARGEBLAST.md +32 -0
  4. package/TODO-email-auth.md +14 -0
  5. package/docs/admin-post-route.md +24 -0
  6. package/docs/ai-library.md +23 -0
  7. package/docs/architecture.md +31 -0
  8. package/docs/auth-hooks.md +74 -0
  9. package/docs/cli-firestore-auth.md +59 -0
  10. package/docs/cli-logs.md +67 -0
  11. package/docs/code-patterns.md +67 -0
  12. package/docs/common-mistakes.md +11 -0
  13. package/docs/common-operations.md +64 -0
  14. package/docs/directory-structure.md +119 -0
  15. package/docs/environment-detection.md +7 -0
  16. package/docs/file-naming.md +11 -0
  17. package/docs/key-files.md +36 -0
  18. package/docs/marketing-campaigns.md +244 -0
  19. package/docs/marketing-fields.md +25 -0
  20. package/docs/mcp.md +95 -0
  21. package/docs/payment-system.md +325 -0
  22. package/docs/response-headers.md +7 -0
  23. package/docs/routes.md +126 -0
  24. package/docs/sanitization.md +61 -0
  25. package/docs/schemas.md +39 -0
  26. package/docs/stripe-webhook-forwarding.md +18 -0
  27. package/docs/testing.md +129 -0
  28. package/docs/usage-rate-limiting.md +67 -0
  29. package/package.json +8 -4
  30. package/scripts/update-disposable-domains.js +1 -1
  31. package/src/defaults/CHANGELOG.md +15 -0
  32. package/src/defaults/CLAUDE.md +8 -4
  33. package/src/defaults/docs/README.md +17 -0
  34. package/src/defaults/test/README.md +33 -0
  35. package/src/manager/events/cron/daily/marketing-newsletter-generate.js +48 -8
  36. package/src/manager/functions/core/actions/api/admin/create-post.js +3 -27
  37. package/src/manager/functions/core/actions/api/general/add-marketing-contact.js +5 -0
  38. package/src/manager/helpers/utilities.js +21 -0
  39. package/src/manager/index.js +1 -1
  40. package/src/manager/libraries/ai/index.js +162 -0
  41. package/src/manager/libraries/ai/providers/anthropic.js +193 -0
  42. package/src/manager/libraries/ai/providers/claude-code.js +206 -0
  43. package/src/manager/libraries/ai/providers/openai.js +934 -0
  44. package/src/manager/libraries/email/data/blocked-local-parts.json +55 -0
  45. package/src/manager/libraries/email/data/blocked-local-patterns.js +11 -0
  46. package/src/manager/libraries/email/data/corporate-domains.json +23 -0
  47. package/src/manager/libraries/{disposable-domains.json → email/data/disposable-domains.json} +3 -0
  48. package/src/manager/libraries/email/generators/lib/filter.js +179 -0
  49. package/src/manager/libraries/email/generators/lib/image-host.js +231 -0
  50. package/src/manager/libraries/email/generators/lib/mjml-template.js +83 -0
  51. package/src/manager/libraries/email/generators/lib/structure.js +278 -0
  52. package/src/manager/libraries/email/generators/lib/svg-illustrator.js +184 -0
  53. package/src/manager/libraries/email/generators/lib/templates/classic-schema.js +63 -0
  54. package/src/manager/libraries/email/generators/lib/templates/clean.js +82 -0
  55. package/src/manager/libraries/email/generators/lib/templates/editorial-helpers.js +100 -0
  56. package/src/manager/libraries/email/generators/lib/templates/editorial.js +317 -0
  57. package/src/manager/libraries/email/generators/lib/templates/field-report-helpers.js +138 -0
  58. package/src/manager/libraries/email/generators/lib/templates/field-report.js +497 -0
  59. package/src/manager/libraries/email/generators/lib/templates/index.js +28 -0
  60. package/src/manager/libraries/email/generators/lib/templates/shared.js +534 -0
  61. package/src/manager/libraries/email/generators/newsletter.js +377 -95
  62. package/src/manager/libraries/email/marketing/index.js +16 -2
  63. package/src/manager/libraries/email/providers/beehiiv.js +7 -3
  64. package/src/manager/libraries/email/validation.js +53 -38
  65. package/src/manager/libraries/openai.js +13 -932
  66. package/src/manager/routes/admin/post/deduplicate-image-alts.js +52 -0
  67. package/src/manager/routes/admin/post/post.js +10 -17
  68. package/src/manager/routes/marketing/contact/post.js +5 -1
  69. package/templates/_.env +4 -0
  70. package/templates/_.gitignore +1 -0
  71. package/templates/backend-manager-config.json +48 -4
  72. package/test/helpers/email-validation.js +141 -3
  73. package/test/helpers/slugify.js +394 -0
  74. package/test/marketing/fixtures/clean.json +31 -0
  75. package/test/marketing/fixtures/editorial.json +31 -0
  76. package/test/marketing/fixtures/field-report.json +54 -0
  77. package/test/marketing/newsletter-generate.js +731 -0
  78. package/test/marketing/newsletter-templates.js +512 -0
  79. package/test/routes/admin/deduplicate-image-alts.js +190 -0
  80. /package/src/manager/libraries/{custom-disposable-domains.json → email/data/custom-disposable-domains.json} +0 -0
package/CLAUDE.md CHANGED
@@ -1,1569 +1,140 @@
1
- # Backend Manager (BEM) - Claude Code Instructions
1
+ # Backend Manager (BEM)
2
2
 
3
- This document provides instructions for Claude Code when working with Backend Manager projects.
3
+ > **Note for contributors and Claude:** This file is the architectural overview — identity, top-level conventions, and a map to deep references. The **meat** (per-subsystem APIs, behavior tables, recipes) lives in `docs/<topic>.md`. When extending or adding content, write it in the matching `docs/*.md` file and cross-link from here — do NOT inline it. If a topic doesn't have a doc yet, create one. Goal: keep this file under 250 lines.
4
4
 
5
- ## Project Identity
5
+ ## Identity
6
6
 
7
- **Backend Manager (BEM)** is an NPM package that provides powerful backend features for Firebase Cloud Functions projects, including authentication, rate limiting, analytics, and more.
7
+ Backend Manager (BEM) is a comprehensive framework for building modern Firebase Cloud Functions backends. Sister project to Electron Manager (EM), Browser Extension Manager (BXM), and Ultimate Jekyll Manager (UJM). Provides a single `Manager.init(exports, {...})` bootstrap that wires built-in functions (`bm_api`, auth events, cron jobs), helper classes (Assistant, User, Analytics, Usage, Middleware, Settings, Utilities, Metadata), payment processor integrations (Stripe / PayPal), Firestore-trigger pipelines, marketing campaign automation, an MCP server, and a CLI for emulator/deploy/logs/auth/Firestore operations.
8
8
 
9
- **This repository** (`backend-manager`) is the BEM library itself. If you're working here, you're contributing to the library, not consuming it.
9
+ **This repository** is the BEM library itself. **Consumer projects** are Firebase projects that `require('backend-manager')` in their `functions/index.js`, with `backend-manager-config.json` + `service-account.json` alongside, plus optional `routes/`, `schemas/`, and `hooks/` directories for custom endpoints.
10
10
 
11
- **Consumer projects** are Firebase projects that `require('backend-manager')` in their `functions/index.js`. These have:
12
- - `functions/` directory with `index.js` that calls `Manager.init(exports, {...})`
13
- - `backend-manager-config.json` configuration file
14
- - `service-account.json` for Firebase credentials
15
- - Optional `routes/` and `schemas/` directories for custom endpoints
11
+ ## Recommended skills
16
12
 
17
- ## Architecture Overview
13
+ Two Claude Code skills are tailored to this project. Both auto-trigger on relevant keywords and can be invoked manually with `/<skill-name>`:
18
14
 
19
- ### Manager Class
20
- The core `Manager` class (in `src/manager/index.js`) extends EventEmitter and orchestrates all functionality:
21
- - Initializes Firebase Admin SDK
22
- - Sets up built-in Cloud Functions (`bm_api`, auth events, cron)
23
- - Provides factory methods for helper classes
24
- - Manages configuration from multiple sources
15
+ - **`BEM:patterns`** — SSOT for Backend Manager routes, schemas, tests, Firebase functions, Firestore rules, usage tracking patterns. Auto-loads on BEM-specific keywords (`route`, `schema`, `endpoint`, `bm_api`, `Manager.init`, `npx mgr test`, `gcloud logs`, etc.) and when touching files in `functions/routes/`, `functions/schemas/`, `functions/index.js`, `test/`, `src/cli/commands/`.
16
+ - **`js:patterns`** JavaScript/Node.js conventions: file structure, JSDoc, defensive coding (`?.` usage), template literals, `package.json` conventions. Auto-loads when creating new `.js` files or touching JS module structure.
25
17
 
26
- ### Dual-Mode Support
27
- BEM supports two deployment modes:
28
- - **Firebase Functions** (`projectType: 'firebase'`): Cloud Functions with Firebase triggers
29
- - **Custom Server** (`projectType: 'custom'`): Express server for non-Firebase deployments
18
+ ## Quick Start
30
19
 
31
- ### Helper Factory Pattern
32
- All helpers are accessed via factory methods on the Manager instance:
33
- ```javascript
34
- Manager.Assistant({ req, res }) // Request handler
35
- Manager.User(data) // User properties
36
- Manager.Analytics({ assistant }) // GA4 events
37
- Manager.Usage() // Rate limiting
38
- Manager.Middleware(req, res) // Request pipeline
39
- Manager.Settings() // Schema validation
40
- Manager.Utilities() // Batch operations
41
- Manager.Metadata(doc) // Timestamps/tags
42
- Manager.storage({ name }) // Local JSON storage (lowdb)
43
- ```
20
+ ### For Consuming Projects
44
21
 
45
- ## Directory Structure
22
+ 1. `npm install backend-manager --save-dev` (inside `functions/`)
23
+ 2. `npx mgr setup` — validates config, scaffolds defaults (CLAUDE.md, CHANGELOG.md, docs/, test/), provisions Firestore indexes
24
+ 3. `npx mgr emulator` — start Firebase emulators (auth/firestore/functions/database/storage)
25
+ 4. `npx mgr serve` — local serve with Stripe webhook forwarding (if `STRIPE_SECRET_KEY` is set)
26
+ 5. `npx mgr test` — runs framework + project test suites against an emulator
27
+ 6. `npx mgr deploy` — deploy Cloud Functions to Firebase
28
+ 7. `npx mgr logs:read` / `npx mgr logs:tail` — Cloud Function logs from Google Cloud Logging
46
29
 
47
- ### BEM Library (this repo)
48
- ```
49
- src/
50
- manager/
51
- index.js # Main Manager class
52
- helpers/ # Helper classes
53
- assistant.js # Request/response handling
54
- user.js # User property structure + schema
55
- analytics.js # GA4 integration
56
- usage.js # Rate limiting
57
- middleware.js # Request pipeline
58
- settings.js # Schema validation
59
- utilities.js # Batch operations
60
- metadata.js # Timestamps/tags
61
- libraries/
62
- payment/ # Shared payment utilities
63
- order-id.js # Order ID generation (XXXX-XXXX-XXXX)
64
- processors/ # Payment processor libraries
65
- stripe.js # Stripe SDK init, fetchResource, toUnified*, resolvePriceId
66
- paypal.js # PayPal fetchResource, toUnified* (custom_id parsing)
67
- test.js # Test processor (delegates to Stripe shapes)
68
- events/ # All event-driven code
69
- auth/ # Auth event handlers (hookable)
70
- before-create.js # Disposable email blocking + IP rate limiting
71
- before-signin.js # Activity update + sign-in analytics
72
- on-create.js # User doc creation
73
- on-delete.js # User doc deletion + marketing cleanup
74
- utils.js # Shared utilities (retryWrite, runAuthHook)
75
- cron/ # Cron job runners
76
- runner.js # Shared cron job runner (BEM + consumer hooks)
77
- daily.js # Daily cron entry point
78
- daily/{job}.js # Individual daily cron jobs
79
- frequent.js # Frequent cron entry point
80
- frequent/{job}.js # Individual frequent cron jobs
81
- firestore/ # Firestore triggers
82
- payments-webhooks/ # Webhook processing pipeline
83
- on-write.js # Orchestrator: fetch→transform→transition→write
84
- analytics.js # Payment analytics tracking (GA4, Meta, TikTok)
85
- transitions/ # State transition detection + handlers
86
- index.js # Transition detection logic
87
- send-email.js # Shared email helper for handlers
88
- subscription/ # Subscription transition handlers
89
- one-time/ # One-time payment transition handlers
90
- functions/core/ # Built-in functions
91
- actions/
92
- api.js # Main bm_api handler
93
- api/{category}/{action}.js # API command handlers
94
- routes/ # Built-in routes
95
- admin/
96
- post/ # POST /admin/post - Create blog posts via GitHub
97
- post.js # Extracts images, uploads to GitHub, rewrites body to @post/ format
98
- put.js # PUT /admin/post - Edit existing posts
99
- templates/
100
- post.html # Post frontmatter template
101
- payments/
102
- intent/ # POST /payments/intent
103
- post.js # Intent creation orchestrator
104
- processors/ # Per-processor intent creators
105
- stripe.js # Stripe Checkout Session creation
106
- paypal.js # PayPal subscription + one-time order creation
107
- test.js # Test processor (auto-fires webhooks)
108
- webhook/ # POST /payments/webhook
109
- post.js # Webhook ingestion + Firestore write
110
- processors/ # Per-processor webhook parsers
111
- stripe.js # Stripe event parsing + categorization
112
- paypal.js # PayPal event parsing + categorization
113
- test.js # Test processor (delegates to Stripe)
114
- cancel/ # POST /payments/cancel
115
- processors/
116
- stripe.js # Stripe cancel_at_period_end
117
- paypal.js # PayPal subscription cancel
118
- test.js # Test cancel (writes webhook doc)
119
- refund/ # POST /payments/refund
120
- processors/
121
- stripe.js # Stripe refund + immediate cancel
122
- paypal.js # PayPal refund + cancel
123
- test.js # Test refund (writes webhook doc)
124
- portal/ # POST /payments/portal
125
- processors/
126
- stripe.js # Stripe billing portal URL
127
- paypal.js # PayPal management URL
128
- schemas/ # Built-in schemas
129
- cli/
130
- index.js # CLI entry point
131
- commands/ # CLI commands
132
- test/
133
- test-accounts.js # Test account definitions (static + journey)
134
- templates/
135
- backend-manager-config.json # Config template
136
- ```
30
+ All `npx mgr <cmd>` aliases work: `npx bm <cmd>`, `npx bem <cmd>`, `npx backend-manager <cmd>`.
137
31
 
138
- ### Consumer Project Structure
139
- ```
140
- functions/
141
- index.js # Manager.init() + custom functions
142
- backend-manager-config.json # App configuration
143
- service-account.json # Firebase credentials
144
- routes/
145
- {endpoint}/
146
- index.js # All methods handler
147
- get.js # GET handler
148
- post.js # POST handler
149
- schemas/
150
- {endpoint}/
151
- index.js # Schema definition
152
- hooks/
153
- auth/
154
- before-create.js # Custom pre-signup checks (can block)
155
- before-signin.js # Custom pre-signin checks (can block)
156
- on-create.js # Post-signup side effects (non-blocking)
157
- on-delete.js # Post-deletion side effects (non-blocking)
158
- cron/
159
- daily/
160
- {job}.js # Custom daily jobs
161
- ```
32
+ > **Important:** All `npx mgr ...` commands MUST be run from the consumer project's `functions/` subdirectory. The binary lives in `functions/node_modules/.bin/`.
162
33
 
163
- ## Code Patterns
34
+ ### For Framework Development (This Repository)
164
35
 
165
- ### Short-Circuit Returns
166
- Use early returns instead of nested conditionals:
167
- ```javascript
168
- // CORRECT
169
- function handler(data) {
170
- if (!data) {
171
- return assistant.errorify('Missing data', { code: 400 });
172
- }
36
+ 1. `npm install` — install BEM's own deps
37
+ 2. `npm run prepare` build once: copies `src/` → `dist/` via prepare-package
38
+ 3. `npm run prepare:watch` — watch mode
39
+ 4. Test in a consumer project: from inside the consumer's `functions/` dir, run `npx mgr install local` (swaps BEM to the local repo via the `install` CLI). Reverse with `npx mgr install prod`.
173
40
 
174
- // Main logic here
175
- return assistant.respond({ success: true });
176
- }
41
+ ## Architecture
177
42
 
178
- // INCORRECT
179
- function handler(data) {
180
- if (data) {
181
- // Main logic here
182
- return assistant.respond({ success: true });
183
- }
184
- }
185
- ```
43
+ BEM exposes a single `Manager` class that orchestrates everything: it initializes Firebase Admin, wires built-in functions (`bm_api`, auth events, cron), and hands out helper instances via factory methods. Supports **two deployment modes** — Firebase Functions (`projectType: 'firebase'`) or Custom Server (`projectType: 'custom'`). See [docs/architecture.md](docs/architecture.md) for the full overview of the Manager class, dual-mode support, and helper factory pattern.
186
44
 
187
- ### Logical Operators on New Lines
188
- Place operators at the start of continuation lines:
189
- ```javascript
190
- // CORRECT
191
- const isValid = hasPermission
192
- || isAdmin
193
- || isOwner;
45
+ For the directory layout of both the BEM library and consumer projects, see [docs/directory-structure.md](docs/directory-structure.md).
194
46
 
195
- // INCORRECT
196
- const isValid = hasPermission ||
197
- isAdmin ||
198
- isOwner;
199
- ```
47
+ ## CLI
200
48
 
201
- ### Firestore Document Access
202
- Use shorthand `.doc()` path:
203
- ```javascript
204
- // CORRECT
205
- admin.firestore().doc('users/abc123')
49
+ `npx mgr <command>` (aliases `bm`, `bem`, `backend-manager`):
206
50
 
207
- // INCORRECT
208
- admin.firestore().collection('users').doc('abc123')
209
- ```
51
+ | Command | Description |
52
+ |---|---|
53
+ | `setup` | Validate config, scaffold defaults (CLAUDE.md, CHANGELOG.md, docs/, test/), provision Firestore indexes |
54
+ | `emulator` | Start Firebase emulators (auth/firestore/functions/database/storage) |
55
+ | `serve` | Local Firebase serve (with auto Stripe webhook forwarding if keys set) |
56
+ | `watch` | Auto-reload functions on file change |
57
+ | `deploy` | Deploy Cloud Functions to Firebase |
58
+ | `test` | Run framework + project test suites against an emulator |
59
+ | `mcp` | Start the stdio MCP server (for Claude Code / Claude Desktop) |
60
+ | `firestore:get/set/query/delete` | Direct Firestore reads/writes from the terminal |
61
+ | `auth:get/list/delete/set-claims` | Manage Auth users from the terminal |
62
+ | `logs:read` / `logs:tail` | Cloud Function logs from Google Cloud Logging |
63
+ | `stripe` | Standalone Stripe CLI webhook forwarding |
64
+ | `indexes` | Sync required Firestore indexes into `firestore.indexes.json` |
65
+ | `firebase-init` | Run Firebase Admin SDK initialization helper |
66
+ | `clean` | Remove generated artifacts (logs, test outputs) |
67
+ | `version` | Print BEM version |
210
68
 
211
- ### Template Strings for Requires
212
- ```javascript
213
- // CORRECT
214
- require(`${functionsDir}/node_modules/backend-manager`)
69
+ See [docs/cli-firestore-auth.md](docs/cli-firestore-auth.md) and [docs/cli-logs.md](docs/cli-logs.md) for full flag references.
215
70
 
216
- // INCORRECT
217
- require(functionsDir + '/node_modules/backend-manager')
218
- ```
71
+ ## File Conventions
219
72
 
220
- ### Prefer fs-jetpack
221
- Use `fs-jetpack` over `fs` or `fs-extra` for file operations.
73
+ - **CommonJS** throughout. `prepare-package` copies `src/` → `dist/` 1:1 (no transforms).
74
+ - **`fs-jetpack`** over `fs` / `fs-extra` for file operations.
75
+ - One `module.exports = ...` per file.
76
+ - **Short-circuit early returns** rather than nested ifs.
77
+ - **Logical operators at the start of continuation lines** (`|| condB` on a new line, not `condA ||` trailing).
78
+ - **Firestore shorthand**: `admin.firestore().doc('users/abc123')` (path string) rather than `.collection('users').doc('abc123')`.
79
+ - **Template strings for requires**: `` require(`${functionsDir}/node_modules/backend-manager`) `` rather than string concat.
80
+ - **No backwards compatibility** unless explicitly requested.
81
+ - **Routes receive sanitized data by default** — see [docs/sanitization.md](docs/sanitization.md) for opt-out rules.
82
+ - **Match schema names to route names** — if route is `myEndpoint`, schema is `myEndpoint`.
83
+ - **Always use `assistant.respond()` for responses** — do NOT use `res.send()` directly.
84
+ - **Add Firestore composite indexes** for any compound query (`where` + `orderBy`, or multiple `where`s) to `src/cli/commands/setup-tests/helpers/required-indexes.js` (the SSOT). Without the index, queries crash with `FAILED_PRECONDITION` in production.
222
85
 
223
- ## Sanitization (XSS Prevention)
86
+ See [docs/code-patterns.md](docs/code-patterns.md) for code-pattern detail, [docs/common-mistakes.md](docs/common-mistakes.md) for the full anti-pattern checklist, and [docs/file-naming.md](docs/file-naming.md) for the naming table (routes / schemas / API commands / events / cron jobs / hooks).
224
87
 
225
- BEM automatically sanitizes all incoming request data — stripping HTML tags and trimming whitespace from every string field. This happens in the middleware pipeline before route handlers execute, so **routes receive clean data by default**.
88
+ ## Doc-update parity
226
89
 
227
- ### How It Works
228
- 1. **Schema fields**: Sanitized per-field during the middleware pipeline. Fields can opt out with `sanitize: false` in the schema.
229
- 2. **Non-schema fields** (when `setupSettings: false` or `includeNonSchemaSettings: true`): All strings are sanitized with no opt-out.
230
- 3. The middleware uses `Manager.Utilities().sanitize()` under the hood.
90
+ Whenever you make a behavioral change (new command, new flag, new pattern, removed feature), update:
231
91
 
232
- ### Schema Opt-Out
233
- For fields that legitimately need HTML (rich text, email templates, etc.), set `sanitize: false` in the schema:
234
- ```javascript
235
- // This field will NOT be sanitized — raw HTML is preserved
236
- htmlContent: {
237
- types: ['string'],
238
- default: '',
239
- sanitize: false,
240
- },
241
- // This field IS sanitized (default behavior, no flag needed)
242
- name: {
243
- types: ['string'],
244
- default: '',
245
- },
246
- ```
92
+ 1. **`README.md`** — user-facing summary
93
+ 2. **`CLAUDE.md`** (this file) architecture overview, one paragraph or cross-link
94
+ 3. **`docs/<topic>.md`** — the meat. If a topic doesn't have a doc yet, create one.
95
+ 4. **`CHANGELOG.md`** if the project keeps one
247
96
 
248
- ### Route-Level Opt-Out
249
- Disable sanitization entirely for a route (rare — only for routes that handle raw HTML everywhere):
250
- ```javascript
251
- // In functions/index.js
252
- Manager.Middleware(req, res).run('my-route', { sanitize: false });
253
- ```
97
+ Don't ship behavioral changes with stale docs. Validate first, then document — write docs that describe shipped reality, not intentions.
254
98
 
255
- ### Manual Sanitization (Outside Middleware)
256
- For cron jobs, event handlers, or anywhere outside the request pipeline, use `utilities.sanitize()` directly:
257
- ```javascript
258
- // Available in route context
259
- const clean = utilities.sanitize(untrustedData);
99
+ ## Documentation
260
100
 
261
- // Or via Manager
262
- const clean = Manager.Utilities().sanitize(untrustedData);
263
- ```
264
- Accepts any data type — strings, objects, arrays, primitives. Walks objects/arrays recursively, strips HTML from strings, passes everything else through unchanged.
101
+ Deep references live in `docs/`. **Whenever you make a behavioral change, update both this overview AND the relevant `docs/*.md` deep reference.**
265
102
 
266
- ### Route Handler Context
267
- The middleware injects these into every route handler:
268
- ```javascript
269
- module.exports = async ({ Manager, assistant, analytics, usage, user, settings, libraries, utilities }) => {
270
- // settings — already sanitized by middleware
271
- // utilities — Manager.Utilities() instance for manual sanitization
272
- };
273
- ```
103
+ ### Architecture & Conventions
274
104
 
275
- ## Creating New Components
105
+ - [docs/architecture.md](docs/architecture.md) Manager class, dual-mode (firebase/custom), helper factory pattern
106
+ - [docs/directory-structure.md](docs/directory-structure.md) — BEM library + consumer project layouts
107
+ - [docs/code-patterns.md](docs/code-patterns.md) — short-circuit returns, logical operators on new lines, Firestore shorthand, template-string requires, fs-jetpack preference
108
+ - [docs/file-naming.md](docs/file-naming.md) — naming table for routes, schemas, API commands, events, cron jobs, hooks
109
+ - [docs/common-mistakes.md](docs/common-mistakes.md) — anti-pattern checklist (don't modify Manager internals, always await, increment-before-update, etc.)
110
+ - [docs/key-files.md](docs/key-files.md) — quick lookup for the most-touched files (Manager, helpers, auth events, cron, payment processors, CLI commands)
111
+ - [docs/environment-detection.md](docs/environment-detection.md) — `assistant.isDevelopment/isProduction/isTesting()`
112
+ - [docs/response-headers.md](docs/response-headers.md) — automatic `bm-properties` header
276
113
 
277
- ### New API Command
114
+ ### Building Routes & Components
278
115
 
279
- Create `src/manager/functions/core/actions/api/{category}/{action}.js`:
116
+ - [docs/routes.md](docs/routes.md) — recipes for new API commands, routes, event handlers, cron jobs (with code templates)
117
+ - [docs/schemas.md](docs/schemas.md) — schema definition format, defaults vs premium overrides
118
+ - [docs/sanitization.md](docs/sanitization.md) — automatic XSS sanitization, schema opt-out (`sanitize: false`), route-level opt-out, manual `utilities.sanitize()`
119
+ - [docs/auth-hooks.md](docs/auth-hooks.md) — consumer hooks for `before-create`/`before-signin`/`on-create`/`on-delete` (blocking + non-blocking examples)
120
+ - [docs/common-operations.md](docs/common-operations.md) — inside-the-handler patterns: authenticate, read/write Firestore, error handling, send response, `bm_api` hook
280
121
 
281
- ```javascript
282
- function Module() {}
122
+ ### Built-in Routes
283
123
 
284
- Module.prototype.main = function () {
285
- const self = this;
286
- const Manager = self.Manager;
287
- const Api = self.Api;
288
- const assistant = self.assistant;
289
- const payload = self.payload;
124
+ - [docs/admin-post-route.md](docs/admin-post-route.md) `POST/PUT /admin/post` blog creation via GitHub (image extraction + `@post/` rewriting)
125
+ - [docs/payment-system.md](docs/payment-system.md) full payment pipeline: Intent → Webhook → On-Write → Transition; subscription model, statuses, `resolveSubscription()`, transition handlers, processor interface, product config, test processor
126
+ - [docs/marketing-campaigns.md](docs/marketing-campaigns.md) campaign CRUD routes, recurring campaigns, generator pipeline (newsletter), template-owned schemas, asset hosting, seed campaigns
127
+ - [docs/mcp.md](docs/mcp.md) Model Context Protocol server: 19 tools, stdio + HTTP transports, OAuth, Claude Chat/Code configuration
290
128
 
291
- return new Promise(async function(resolve, reject) {
292
- // Validate input
293
- if (!payload.data.payload.requiredField) {
294
- return reject(assistant.errorify('Missing required field', { code: 400 }));
295
- }
129
+ ### Subsystems & Libraries
296
130
 
297
- // Business logic here
298
- const result = { success: true };
131
+ - [docs/usage-rate-limiting.md](docs/usage-rate-limiting.md) usage tracking, monthly/daily caps, `setUser()` + mirrors for proxy usage, reset schedule
132
+ - [docs/ai-library.md](docs/ai-library.md) `Manager.AI()` unified entry for OpenAI + Anthropic
133
+ - [docs/marketing-fields.md](docs/marketing-fields.md) — adding custom fields to SendGrid + Beehiiv via the BEM/OMEGA SSOT pair
134
+ - [docs/stripe-webhook-forwarding.md](docs/stripe-webhook-forwarding.md) — auto-started Stripe CLI forwarding for local dev
299
135
 
300
- // Log and return
301
- assistant.log('Action completed', result);
302
- return resolve({ data: result });
303
- });
304
- };
136
+ ### Testing & CLI
305
137
 
306
- module.exports = Module;
307
- ```
308
-
309
- ### New Route (Consumer Project)
310
-
311
- Create `routes/{name}/index.js`:
312
-
313
- ```javascript
314
- function Route() {}
315
-
316
- Route.prototype.main = async function (assistant) {
317
- const Manager = assistant.Manager;
318
- const usage = assistant.usage;
319
- const user = assistant.usage.user;
320
- const analytics = assistant.analytics;
321
- const settings = assistant.settings;
322
-
323
- // Check authentication if needed
324
- if (!user.authenticated) {
325
- return assistant.respond('Authentication required', { code: 401 });
326
- }
327
-
328
- // Track usage
329
- await usage.validate('requests');
330
- usage.increment('requests');
331
- await usage.update();
332
-
333
- // Send response
334
- assistant.respond({ success: true, data: settings });
335
- };
336
-
337
- module.exports = Route;
338
- ```
339
-
340
- ### New Schema (Consumer Project)
341
-
342
- Create `schemas/{name}/index.js`:
343
-
344
- ```javascript
345
- module.exports = function (assistant, settings, options) {
346
- const user = options.user;
347
-
348
- return {
349
- defaults: {
350
- fieldName: {
351
- types: ['string'],
352
- default: 'default value',
353
- required: false,
354
- },
355
- numericField: {
356
- types: ['number'],
357
- default: 10,
358
- min: 1,
359
- max: 100,
360
- },
361
- },
362
- // Override for premium users
363
- premium: {
364
- numericField: {
365
- max: 1000,
366
- },
367
- },
368
- };
369
- };
370
- ```
371
-
372
- ### New Event Handler
373
-
374
- Create `src/manager/functions/core/events/{type}/{event}.js`:
375
-
376
- ```javascript
377
- function Module() {}
378
-
379
- Module.prototype.init = function (Manager, payload) {
380
- const self = this;
381
- self.Manager = Manager;
382
- self.assistant = Manager.Assistant();
383
- self.libraries = Manager.libraries;
384
- self.user = payload.user;
385
- self.context = payload.context;
386
- return self;
387
- };
388
-
389
- Module.prototype.main = function () {
390
- const self = this;
391
- const Manager = self.Manager;
392
- const assistant = self.assistant;
393
-
394
- return new Promise(async function(resolve, reject) {
395
- const { admin } = self.libraries;
396
-
397
- assistant.log('Event triggered', self.user);
398
-
399
- // Event logic here
400
-
401
- return resolve(self);
402
- });
403
- };
404
-
405
- module.exports = Module;
406
- ```
407
-
408
- ### New Cron Job (Consumer Project)
409
-
410
- Create `hooks/cron/daily/{job}.js`:
411
-
412
- ```javascript
413
- function Job() {}
414
-
415
- Job.prototype.main = function () {
416
- const self = this;
417
- const Manager = self.Manager;
418
- const assistant = self.assistant;
419
-
420
- return new Promise(async function(resolve, reject) {
421
- assistant.log('Running daily job...');
422
-
423
- // Job logic here
424
-
425
- return resolve();
426
- });
427
- };
428
-
429
- module.exports = Job;
430
- ```
431
-
432
- ### Auth Hooks (Consumer Project)
433
-
434
- Auth hooks let consumer projects inject custom logic into BEM's auth event lifecycle. BEM runs its core handler first, then looks for a matching hook at `hooks/auth/{event-name}.js`.
435
-
436
- | Hook | File | Behavior |
437
- |------|------|----------|
438
- | `before-create` | `hooks/auth/before-create.js` | Runs after BEM's disposable email + rate limit checks. **Can throw `HttpsError` to block signup.** |
439
- | `before-signin` | `hooks/auth/before-signin.js` | Runs after BEM's activity update. **Can throw `HttpsError` to block sign-in.** |
440
- | `on-create` | `hooks/auth/on-create.js` | Runs after BEM creates the user doc. **Non-blocking** — errors are caught and logged. |
441
- | `on-delete` | `hooks/auth/on-delete.js` | Runs after BEM deletes the user doc. **Non-blocking** — errors are caught and logged. |
442
-
443
- Hook signature (same as BEM's internal handlers):
444
- ```javascript
445
- module.exports = async ({ Manager, assistant, user, context, libraries }) => {
446
- // user: AuthUserRecord (uid, email, providerData, etc.)
447
- // context: AuthEventContext for blocking functions (ipAddress, userAgent, additionalUserInfo)
448
- // EventContext for triggers (eventId, eventType, timestamp — no IP/userAgent)
449
- // libraries: { admin, functions, ... }
450
- };
451
- ```
452
-
453
- #### Blocking hook example (before-create)
454
-
455
- ```javascript
456
- // hooks/auth/before-create.js — Only allow Google OAuth signups
457
- const ENFORCE = true;
458
-
459
- const ALLOWED_PROVIDERS = ['google.com'];
460
-
461
- module.exports = async ({ assistant, user, context, libraries }) => {
462
- if (!ENFORCE) { return; }
463
-
464
- const { functions } = libraries;
465
- const provider = context.additionalUserInfo?.providerId;
466
-
467
- if (!ALLOWED_PROVIDERS.includes(provider)) {
468
- assistant.error(`hook/before-create: Blocked provider '${provider}' for ${user.email}`);
469
- throw new functions.auth.HttpsError('permission-denied', 'Please sign up with Google.');
470
- }
471
- };
472
- ```
473
-
474
- #### Non-blocking hook example (on-create)
475
-
476
- ```javascript
477
- // hooks/auth/on-create.js — Auto-delete spam referrals
478
- const powertools = require('node-powertools');
479
-
480
- const ENFORCE = true;
481
- const BLOCKED_AFFILIATE_CODES = ['iLvQjmvm'];
482
-
483
- module.exports = async ({ Manager, assistant, user, context, libraries }) => {
484
- if (!ENFORCE) { return; }
485
-
486
- const { admin } = libraries;
487
- const uid = user.uid;
488
-
489
- // Poll until signup route attaches attribution.affiliate.code
490
- let referredBy = null;
491
-
492
- await powertools.poll(async () => {
493
- const userDoc = await admin.firestore().doc(`users/${uid}`).get().catch(() => null);
494
- if (!userDoc?.exists) { return true; }
495
- referredBy = userDoc.data()?.attribution?.affiliate?.code;
496
- return !!referredBy;
497
- }, { interval: 10000, timeout: 60000 }).catch(() => {});
498
-
499
- if (!referredBy || !BLOCKED_AFFILIATE_CODES.includes(referredBy)) { return; }
500
-
501
- // Delete spam account (triggers on-delete for cleanup)
502
- await admin.auth().deleteUser(uid).catch(e => assistant.error('Delete failed:', e));
503
- };
504
- ```
505
-
506
- ## Common Operations
507
-
508
- ### Authenticate User
509
- ```javascript
510
- const user = await assistant.authenticate();
511
- if (!user.authenticated) {
512
- return assistant.errorify('Authentication required', { code: 401 });
513
- }
514
- ```
515
-
516
- ### Read/Write Firestore
517
- ```javascript
518
- const { admin } = Manager.libraries;
519
-
520
- // Read
521
- const doc = await admin.firestore().doc('users/abc123').get();
522
- const data = doc.data();
523
-
524
- // Write
525
- await admin.firestore().doc('users/abc123').set({ field: 'value' }, { merge: true });
526
- ```
527
-
528
- ### Handle Errors
529
- ```javascript
530
- // Send error response
531
- assistant.errorify('Something went wrong', { code: 500, sentry: true });
532
-
533
- // Or throw to reject
534
- return reject(assistant.errorify('Bad request', { code: 400 }));
535
- ```
536
-
537
- ### Send Response
538
- ```javascript
539
- // Success
540
- assistant.respond({ success: true, data: result });
541
-
542
- // With custom status
543
- assistant.respond({ created: true }, { code: 201 });
544
-
545
- // Redirect
546
- assistant.respond('https://example.com', { code: 302 });
547
- ```
548
-
549
- ### Use Hooks (Consumer Project)
550
- ```javascript
551
- Manager.handlers.bm_api = function (mod, position) {
552
- const assistant = mod.assistant;
553
- const command = assistant.request.data.command;
554
-
555
- return new Promise(async function(resolve, reject) {
556
- if (position === 'pre' && command === 'user:sign-up') {
557
- // Before sign-up logic
558
- }
559
- return resolve();
560
- });
561
- };
562
- ```
563
-
564
- ## File Naming Conventions
565
-
566
- | Type | Location | Naming |
567
- |------|----------|--------|
568
- | Routes | `routes/{name}/` | `index.js` or `{method}.js` |
569
- | Schemas | `schemas/{name}/` | `index.js` or `{method}.js` |
570
- | API Commands | `actions/api/{category}/` | `{action}.js` |
571
- | Auth Events | `events/auth/` | `{event}.js` |
572
- | Auth Hooks (consumer) | `hooks/auth/` | `{event}.js` |
573
- | Cron Jobs (BEM) | `events/cron/daily/` | `{job}.js` |
574
- | Cron Jobs (consumer) | `hooks/cron/daily/` | `{job}.js` |
575
-
576
- ## Admin Post Route
577
-
578
- The `POST /admin/post` route creates blog posts via GitHub's API. It handles image extraction, upload, and body rewriting.
579
-
580
- ### Image Processing Flow
581
- 1. Receives markdown body with external image URLs (e.g., `![alt](https://images.unsplash.com/...)`)
582
- 2. Extracts all `![alt](url)` patterns from the body using regex
583
- 3. Downloads each image and uploads it to `src/assets/images/blog/post-{id}/` on GitHub
584
- 4. **Rewrites the body** to replace external URLs with `@post/{filename}` format
585
- 5. The `@post/` prefix is resolved at Jekyll build time by `jekyll-uj-powertools` to the full path
586
-
587
- ### Key Details
588
- - Image filenames are derived from `hyphenate(alt_text)` + downloaded extension
589
- - Header image (`headerImageURL`) is uploaded but NOT rewritten in the body (it's in frontmatter)
590
- - Failed image downloads are skipped — the original external URL stays in the body
591
- - The `extractImages()` function returns a URL mapping used for body rewriting
592
-
593
- ### Files
594
- - `src/manager/routes/admin/post/post.js` — POST handler (create)
595
- - `src/manager/routes/admin/post/put.js` — PUT handler (edit)
596
- - `src/manager/routes/admin/post/templates/post.html` — Post template
597
-
598
- ## Testing
599
-
600
- ### Running Tests
601
- ```bash
602
- # Option 1: Two terminals
603
- npx mgr emulator # Terminal 1 - keeps emulator running
604
- npx mgr test # Terminal 2 - runs tests
605
-
606
- # Option 2: Single command (auto-starts emulator)
607
- npx mgr test
608
- ```
609
-
610
- ### Log Files
611
- BEM CLI commands automatically save all output to log files in `functions/` while still streaming to the console:
612
- - **`functions/serve.log`** — Output from `npx mgr serve` (Firebase serve)
613
- - **`functions/emulator.log`** — Full emulator output (Firebase emulator + Cloud Functions logs)
614
- - **`functions/test.log`** — Test runner output (when running against an existing emulator)
615
- - **`functions/logs.log`** — Cloud Function logs from `npx mgr logs:read` or `npx mgr logs:tail` (raw JSON for `read`, streaming text for `tail`)
616
-
617
- When `npx mgr test` starts its own emulator, logs go to `emulator.log` (since it delegates to the emulator command). When running against an already-running emulator, logs go to `test.log`.
618
-
619
- These files are overwritten on each run and are gitignored (`*.log`). Use them to search for errors, debug webhook pipelines, or review full function output after a test run.
620
-
621
- ### Filtering Tests
622
- ```bash
623
- npx mgr test rules/ # Run rules tests (both BEM and project)
624
- npx mgr test bem:rules/ # Only BEM's rules tests
625
- npx mgr test project:rules/ # Only project's rules tests
626
- npx mgr test user/ admin/ # Multiple paths
627
- ```
628
-
629
- ### Test Locations
630
- - **BEM core tests:** `test/`
631
- - **Project tests:** `functions/test/bem/`
632
-
633
- Use `bem:` or `project:` prefix to filter by source.
634
-
635
- ### Test Types
636
-
637
- | Type | Use When | Behavior |
638
- |------|----------|----------|
639
- | Standalone | Single logical test | Runs once |
640
- | Suite (`type: 'suite'`) | Sequential dependent tests | Shared state, stops on failure |
641
- | Group (`type: 'group'`) | Multiple independent tests | Continues on failure |
642
-
643
- ### Standalone Test
644
- ```javascript
645
- module.exports = {
646
- description: 'Test name',
647
- auth: 'none', // none, user, admin, premium-active, premium-expired
648
- timeout: 10000,
649
- async run({ http, assert, accounts, firestore, state, waitFor }) { },
650
- async cleanup({ ... }) { }, // Optional
651
- };
652
- ```
653
-
654
- ### Suite (Sequential with Shared State)
655
- ```javascript
656
- module.exports = {
657
- description: 'Suite name',
658
- type: 'suite',
659
- tests: [
660
- { name: 'step-1', async run({ state }) { state.value = 'shared'; } },
661
- { name: 'step-2', async run({ state }) { /* state.value available */ } },
662
- ],
663
- };
664
- ```
665
-
666
- ### Group (Independent Tests)
667
- ```javascript
668
- module.exports = {
669
- description: 'Group name',
670
- type: 'group',
671
- tests: [
672
- { name: 'test-1', auth: 'admin', async run({ http, assert }) { } },
673
- { name: 'test-2', auth: 'none', async run({ http, assert }) { } },
674
- ],
675
- };
676
- ```
677
-
678
- ### Context Object
679
- | Property | Description |
680
- |----------|-------------|
681
- | `http` | HTTP client (`http.command()`, `http.as('admin').command()`) |
682
- | `assert` | Assertion helpers (see below) |
683
- | `accounts` | Test accounts `{ basic, admin, premium-active, ... }` |
684
- | `firestore` | Direct DB access (`get`, `set`, `delete`, `exists`) |
685
- | `state` | Shared state (suites only) |
686
- | `waitFor` | Polling helper `waitFor(condition, timeout, interval)` |
687
-
688
- ### Assert Methods
689
- ```javascript
690
- assert.ok(value, message) // Truthy
691
- assert.equal(a, b, message) // Strict equality
692
- assert.notEqual(a, b, message) // Not equal
693
- assert.deepEqual(a, b, message) // Deep equality
694
- assert.match(value, /regex/, message) // Regex match
695
- assert.isSuccess(response, message) // Response success
696
- assert.isError(response, code, message) // Response error with code
697
- assert.hasProperty(obj, 'path.to.prop', msg) // Property exists
698
- assert.propertyEquals(obj, 'path', value, msg) // Property value
699
- assert.isType(value, 'string', message) // Type check
700
- assert.contains(array, value, message) // Array includes
701
- assert.inRange(value, min, max, message) // Number range
702
- assert.fail(message) // Explicit fail
703
- ```
704
-
705
- ### Auth Levels
706
- `none`, `user`/`basic`, `admin`, `premium-active`, `premium-expired`
707
-
708
- ### Key Test Files
709
- | File | Purpose |
710
- |------|---------|
711
- | `src/test/runner.js` | Test runner |
712
- | `test/` | BEM core tests |
713
- | `src/test/utils/assertions.js` | Assert helpers |
714
- | `src/test/utils/http-client.js` | HTTP client |
715
- | `src/test/test-accounts.js` | Test account definitions |
716
-
717
- ## Stripe Webhook Forwarding
718
-
719
- BEM auto-starts Stripe CLI webhook forwarding when running `npx mgr serve` or `npx mgr emulator`. This forwards Stripe test webhooks to the local server so the full payment pipeline works end-to-end during development.
720
-
721
- **Requirements:**
722
- - `STRIPE_SECRET_KEY` set in `functions/.env`
723
- - `BACKEND_MANAGER_KEY` set in `functions/.env`
724
- - [Stripe CLI](https://stripe.com/docs/stripe-cli) installed
725
-
726
- **Standalone usage:**
727
- ```bash
728
- npx mgr stripe
729
- ```
730
-
731
- If any prerequisite is missing, webhook forwarding is silently skipped with an info message.
732
-
733
- The forwarding URL is: `http://localhost:{hostingPort}/backend-manager/payments/webhook?processor=stripe&key={BACKEND_MANAGER_KEY}`
734
-
735
- ## CLI Utility Commands
736
-
737
- Quick commands for reading/writing Firestore and managing Auth users directly from the terminal. Works in any BEM consumer project (requires `functions/service-account.json` for production, or `--emulator` for local).
738
-
739
- **IMPORTANT: All CLI commands (`npx mgr ...`) MUST be run from the consumer project's `functions/` subdirectory** (e.g., `cd /path/to/my-project/functions && npx mgr ...`). The `mgr` binary lives in `functions/node_modules/.bin/` — running from the project root or any other directory will fail.
740
-
741
- ### Firestore Commands
742
-
743
- ```bash
744
- npx mgr firestore:get <path> # Read a document
745
- npx mgr firestore:set <path> '<json>' # Write/merge a document
746
- npx mgr firestore:set <path> '<json>' --no-merge # Overwrite a document entirely
747
- npx mgr firestore:query <collection> # Query a collection (default limit 25)
748
- --where "field==value" # Filter (repeatable for AND)
749
- --orderBy "field:desc" # Sort
750
- --limit N # Limit results
751
- npx mgr firestore:delete <path> # Delete a document (prompts for confirmation)
752
- ```
753
-
754
- ### Auth Commands
755
-
756
- ```bash
757
- npx mgr auth:get <uid-or-email> # Get user by UID or email (auto-detected via @)
758
- npx mgr auth:list [--limit N] [--page-token T] # List users (default 100)
759
- npx mgr auth:delete <uid-or-email> # Delete user (prompts for confirmation)
760
- npx mgr auth:set-claims <uid-or-email> '<json>' # Set custom claims
761
- ```
762
-
763
- ### Logs Commands
764
-
765
- Fetch or stream Cloud Function logs from Google Cloud Logging. Requires `gcloud` CLI installed and authenticated. Auto-resolves the project ID from `service-account.json`, `.firebaserc`, or `GCLOUD_PROJECT`.
766
-
767
- ```bash
768
- npx mgr logs:read # Read last 1h of logs (default: 300 entries, newest first)
769
- npx mgr logs:read --fn bm_api # Filter by function name
770
- npx mgr logs:read --fn bm_api --severity ERROR # Filter by severity (DEBUG, INFO, WARNING, ERROR, CRITICAL)
771
- npx mgr logs:read --since 2d --limit 100 # Custom time range and limit
772
- npx mgr logs:read --search "72.134.242.25" # Search textPayload for a string (IP, email, error, etc.)
773
- npx mgr logs:read --fn bm_authBeforeCreate --search "ian@example.com" --since 7d # Combined filters
774
- npx mgr logs:read --order asc # Oldest first (default: desc/newest first)
775
- npx mgr logs:read --filter 'jsonPayload.level="error"' # Raw gcloud filter passthrough
776
- npx mgr logs:tail # Stream live logs
777
- npx mgr logs:tail --fn bm_paymentsWebhookOnWrite # Stream filtered live logs
778
- ```
779
-
780
- Both commands save output to `functions/logs.log` (overwritten on each run). `logs:read` saves raw JSON; `logs:tail` streams text.
781
-
782
- **Cloud Logs vs Local Logs:** These commands query **production** Google Cloud Logging. For **local/dev** logs, read `functions/serve.log` (from `npx mgr serve`) or `functions/emulator.log` (from `npx mgr test`) directly — they are plain text files, not gcloud.
783
-
784
- | Flag | Description | Default | Commands |
785
- |------|-------------|---------|----------|
786
- | `--fn <name>` | Filter by Cloud Function name (see table below) | all | both |
787
- | `--severity <level>` | Minimum severity: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL` | all | both |
788
- | `--search <text>` | Search textPayload for a substring (IP, email, uid, error message) | none | both |
789
- | `--filter <expr>` | Raw gcloud logging filter expression (appended to built-in filters) | none | both |
790
- | `--since <duration>` | Time range (`30m`, `1h`, `2d`, `1w`) | `1h` | read only |
791
- | `--limit <n>` | Max entries | `300` | read only |
792
- | `--order <dir>` | Sort order: `asc` (oldest first) or `desc` (newest first) | `desc` | read only |
793
- | `--interval <sec>` | Polling interval in seconds | `5` | tail only |
794
- | `--raw` | Output raw JSON | false | both |
795
-
796
- #### `--fn` Function Name Reference
797
-
798
- The `--fn` flag uses the **deployed Cloud Function name**, not the route path.
799
-
800
- **BEM built-in functions (always deployed):**
801
-
802
- | Function name | Type | Description |
803
- |---------------|------|-------------|
804
- | `bm_api` | HTTPS | Main API router — all consumer routes (GET/POST/PUT/DELETE) go through this |
805
- | `bm_authBeforeCreate` | Auth blocking | Before user creation: disposable email blocking, IP rate limiting, consumer hooks |
806
- | `bm_authBeforeSignIn` | Auth blocking | Before sign-in: consumer hooks |
807
- | `bm_authOnCreate` | Auth event | After user creation: user doc setup |
808
- | `bm_authOnDelete` | Auth event | After user deletion |
809
- | `bm_paymentsWebhookOnWrite` | Firestore trigger | Processes payment webhooks |
810
- | `bm_paymentsDisputeOnWrite` | Firestore trigger | Processes payment disputes |
811
- | `bm_notificationsOnWrite` | Firestore trigger | Sends push notifications |
812
- | `bm_cronDaily` | Scheduled | Daily cron (midnight UTC) |
813
- | `bm_cronFrequent` | Scheduled | Frequent cron (every 10 min) |
814
-
815
- **Consumer-defined functions** use the export name from `functions/index.js` (e.g., `exports.items = ...` → `--fn items`).
816
-
817
- **Quick lookup — which function to query:**
818
- - API route errors → `--fn bm_api`
819
- - Signup/auth blocked → `--fn bm_authBeforeCreate`
820
- - Sign-in issues → `--fn bm_authBeforeSignIn`
821
- - User doc not created → `--fn bm_authOnCreate`
822
- - Payment not processing → `--fn bm_paymentsWebhookOnWrite`
823
- - Cron job issues → `--fn bm_cronDaily` or `--fn bm_cronFrequent`
824
-
825
- ### Shared Flags
826
-
827
- | Flag | Description |
828
- |------|-------------|
829
- | `--emulator` | Target local emulator instead of production |
830
- | `--force` | Skip confirmation on destructive operations |
831
- | `--raw` | Compact JSON output (for piping to `jq` etc.) |
832
-
833
- ### Examples
834
-
835
- ```bash
836
- # Read a user document from production
837
- npx mgr firestore:get users/abc123
838
-
839
- # Write to emulator
840
- npx mgr firestore:set users/test123 '{"name":"Test User"}' --emulator
841
-
842
- # Query with filters
843
- npx mgr firestore:query users --where "subscription.status==active" --limit 10
844
-
845
- # Look up auth user by email
846
- npx mgr auth:get user@example.com
847
-
848
- # Set admin claims
849
- npx mgr auth:set-claims user@example.com '{"admin":true}'
850
-
851
- # Delete from emulator (no confirmation needed)
852
- npx mgr firestore:delete users/test123 --emulator
853
- ```
854
-
855
- ## Usage & Rate Limiting
856
-
857
- ### Overview
858
-
859
- Usage is tracked per-metric (e.g., `requests`, `sponsorships`) with four fields:
860
- - `monthly`: Current month's count, reset on the 1st of each month by cron
861
- - `daily`: Current day's count, reset every day by cron
862
- - `total`: All-time count, never resets
863
- - `last`: Object with `id`, `timestamp`, `timestampUNIX` of the last usage event
864
-
865
- ### Limits & Daily Caps
866
-
867
- Limits are always specified as **monthly** values in product config (e.g., `limits.requests = 100` means 100/month).
868
-
869
- By default, limits are enforced with **daily caps** to prevent users from burning their entire monthly quota in a single day. Two checks are applied:
870
-
871
- 1. **Flat daily cap**: `ceil(monthlyLimit / daysInMonth)` — max uses per day
872
- - e.g., 100/month in a 31-day month = `ceil(100/31)` = 4/day
873
- 2. **Proportional monthly cap**: `ceil(monthlyLimit * dayOfMonth / daysInMonth)` — running total
874
- - Prevents accumulating too much too fast even within daily limits
875
- - e.g., Day 15 of a 30-day month with 100/month limit = max 50 used so far
876
-
877
- Products can opt out of daily caps by setting `rateLimit: 'monthly'` (default is `'daily'`):
878
- ```json
879
- {
880
- "id": "basic",
881
- "limits": { "requests": 100 },
882
- "rateLimit": "monthly"
883
- }
884
- ```
885
-
886
- ### Proxy Usage (setUser + Mirrors)
887
-
888
- Sometimes usage must be billed to a different user than the one making the request (e.g., anonymous visitors consuming an agent owner's credits). Use `setUser()` to swap the target and `addMirror()` / `setMirrors()` to write usage to additional Firestore docs:
889
-
890
- ```js
891
- // Switch usage target to the agent owner (fetches their user doc)
892
- await usage.setUser(ownerUid);
893
-
894
- // Also write usage data to the agent doc
895
- usage.addMirror(`agents/${agentId}`);
896
-
897
- // Now validate, increment, and update all operate on the owner's data
898
- // update() writes to users/{ownerUid} AND agents/{agentId} in parallel
899
- await usage.validate('credits');
900
- usage.increment('credits');
901
- await usage.update();
902
- ```
903
-
904
- **Methods:**
905
- - `setUser(uid)` — async, fetches `users/{uid}` from Firestore, replaces `self.user`, sets `useUnauthenticatedStorage = false`
906
- - `setMirrors(paths)` — sync, overwrites the mirror array with the given paths
907
- - `addMirror(path)` — sync, appends a single path to the mirror array
908
-
909
- Mirrors are write-only — `update()` writes `{ usage: self.user.usage }` (merge) to each mirror path. No reads are performed on mirrors.
910
-
911
- ### Reset Schedule
912
-
913
- | Target | Frequency | What happens |
914
- |--------|-----------|-------------|
915
- | Local storage | Daily | Cleared entirely |
916
- | `usage` collection (unauthenticated) | Daily | Deleted entirely |
917
- | User doc `usage.*.daily` (authenticated) | Daily | Reset to 0 |
918
- | User doc `usage.*.monthly` (authenticated) | Monthly (1st) | Reset to 0 |
919
-
920
- The daily cron (`reset-usage.js`) runs at midnight UTC. It collects all users with non-zero counters across all metrics, then performs a single write per user to reset daily (and monthly on the 1st).
921
-
922
- ## Subscription System
923
-
924
- ### Subscription Statuses
925
-
926
- | Status | Meaning | User can delete account? |
927
- |--------|---------|--------------------------|
928
- | `active` | Subscription is current and valid (includes trialing) | No (unless `product.id === 'basic'`) |
929
- | `suspended` | Payment failed (Stripe: `past_due`, `unpaid`) | No |
930
- | `cancelled` | Subscription terminated (Stripe: `canceled`, `incomplete`, `incomplete_expired`) | Yes |
931
-
932
- ### Stripe Status Mapping
933
-
934
- | Stripe Status | `subscription.status` | Notes |
935
- |---|---|---|
936
- | `active` | `active` | Normal active subscription |
937
- | `trialing` | `active` | `trial.claimed = true` |
938
- | `past_due` | `suspended` | Payment failed, retrying |
939
- | `unpaid` | `suspended` | Payment failed |
940
- | `canceled` | `cancelled` | Subscription terminated |
941
- | `incomplete` | `cancelled` | Never completed initial payment |
942
- | `incomplete_expired` | `cancelled` | Expired before completion |
943
- | `active` + `cancel_at_period_end` | `active` | `cancellation.pending = true` |
944
-
945
- ### Unified Subscription Object (`users/{uid}.subscription`)
946
-
947
- ```javascript
948
- subscription: {
949
- product: {
950
- id: 'basic', // product ID from config ('basic', 'premium', etc.)
951
- name: 'Basic', // display name from config
952
- },
953
- status: 'active', // 'active' | 'suspended' | 'cancelled'
954
- expires: { timestamp, timestampUNIX },
955
- trial: {
956
- claimed: false, // has user EVER used a trial
957
- expires: { timestamp, timestampUNIX },
958
- },
959
- cancellation: {
960
- pending: false, // true = cancel at period end
961
- date: { timestamp, timestampUNIX },
962
- },
963
- payment: {
964
- processor: null, // 'stripe' | 'paypal' | etc.
965
- orderId: null, // BEM order ID (e.g., '1234-5678-9012')
966
- resourceId: null, // provider subscription ID (e.g., 'sub_xxx')
967
- frequency: null, // 'monthly' | 'annually' | 'weekly' | 'daily'
968
- price: 0, // resolved from config (number, e.g., 4.99)
969
- startDate: { timestamp, timestampUNIX },
970
- updatedBy: {
971
- event: { name: null, id: null },
972
- date: { timestamp, timestampUNIX },
973
- },
974
- },
975
- }
976
- ```
977
-
978
- ### Access Check Patterns
979
-
980
- ```javascript
981
- // Is premium (paid)?
982
- user.subscription.status === 'active' && user.subscription.product.id !== 'basic'
983
-
984
- // Is on trial?
985
- user.subscription.trial.claimed && user.subscription.status === 'active'
986
-
987
- // Has pending cancellation?
988
- user.subscription.cancellation.pending === true
989
-
990
- // Payment failed?
991
- user.subscription.status === 'suspended'
992
- ```
993
-
994
- ### resolveSubscription(account)
995
-
996
- `User.resolveSubscription(account)` is a static method on the User helper that derives calculated subscription fields from raw account data. It returns only fields that require derivation logic — raw data (product.id, status, trial, cancellation) lives on the account object directly.
997
-
998
- ```javascript
999
- const User = require('backend-manager/src/manager/helpers/user');
1000
-
1001
- const resolved = User.resolveSubscription(account);
1002
- // Returns: { plan, active, trialing, cancelling }
1003
- ```
1004
-
1005
- | Field | Type | Description |
1006
- |-------|------|-------------|
1007
- | `plan` | `string` | Effective plan ID the user has access to RIGHT NOW (`'basic'` if cancelled/suspended) |
1008
- | `active` | `boolean` | User has active access (active, trialing, or cancelling) |
1009
- | `trialing` | `boolean` | In an active trial (status `'active'` + `trial.claimed` + unexpired `trial.expires`) |
1010
- | `cancelling` | `boolean` | Cancellation pending (status `'active'` + `cancellation.pending` + NOT trialing) |
1011
-
1012
- Accepts either a raw Firestore account object or a resolved `User` instance (checks both `account.subscription` and `account.properties.subscription`).
1013
-
1014
- **Unified with web-manager**: The same function exists as `auth.resolveSubscription(account)` in web-manager (`modules/auth.js`) with identical logic and return shape.
1015
-
1016
- **Use this instead of manual access checks** — it centralizes all the derivation logic in one place:
1017
- ```javascript
1018
- // ✅ PREFERRED — use resolveSubscription
1019
- const resolved = User.resolveSubscription(user);
1020
- if (resolved.active) { /* has access */ }
1021
-
1022
- // ❌ AVOID — manual checks that duplicate logic
1023
- if (user.subscription.status === 'active' && user.subscription.product.id !== 'basic') { /* ... */ }
1024
- ```
1025
-
1026
- ## Payment Transition Handlers
1027
-
1028
- ### Overview
1029
-
1030
- When a webhook changes a subscription or processes a one-time payment, BEM detects the state transition and dispatches to a handler file. Handlers are fire-and-forget (non-blocking) — they run after the transition is detected but before or during the Firestore writes. Handler failures never block webhook processing.
1031
-
1032
- Handlers are skipped during tests unless `TEST_EXTENDED_MODE` is set.
1033
-
1034
- ### Transition Detection
1035
-
1036
- The `transitions/index.js` module compares the **before** state (current `users/{uid}.subscription`) with the **after** state (new unified subscription) to detect what changed.
1037
-
1038
- ### Subscription Transitions
1039
-
1040
- | Transition | Before → After | File |
1041
- |---|---|---|
1042
- | `new-subscription` | basic/null → active paid | `transitions/subscription/new-subscription.js` |
1043
- | `payment-failed` | active → suspended | `transitions/subscription/payment-failed.js` |
1044
- | `payment-recovered` | suspended → active | `transitions/subscription/payment-recovered.js` |
1045
- | `cancellation-requested` | pending=false → pending=true | `transitions/subscription/cancellation-requested.js` |
1046
- | `subscription-cancelled` | non-cancelled → cancelled | `transitions/subscription/subscription-cancelled.js` |
1047
- | `plan-changed` | active product A → active product B | `transitions/subscription/plan-changed.js` |
1048
-
1049
- Note: Trials are NOT a separate transition. The `new-subscription` handler checks `after.trial.claimed` to determine if the subscription started with a trial.
1050
-
1051
- ### One-Time Transitions
1052
-
1053
- | Transition | Event Type | File |
1054
- |---|---|---|
1055
- | `purchase-completed` | `checkout.session.completed` | `transitions/one-time/purchase-completed.js` |
1056
- | `purchase-failed` | `invoice.payment_failed` | `transitions/one-time/purchase-failed.js` |
1057
-
1058
- ### Handler Interface
1059
-
1060
- All handlers are in `src/manager/events/firestore/payments-webhooks/transitions/` and export a single async function:
1061
-
1062
- ```javascript
1063
- module.exports = async function ({ before, after, uid, userDoc, admin, assistant, Manager, eventType, eventId }) {
1064
- // before: previous subscription state (null for new/one-time)
1065
- // after: new unified state (subscription or one-time)
1066
- // userDoc: full user document data
1067
- // eventType: original webhook event type (e.g., 'customer.subscription.updated')
1068
- // eventId: webhook event ID
1069
- };
1070
- ```
1071
-
1072
- ### Creating a New Transition Handler
1073
-
1074
- 1. Add detection logic in `transitions/index.js` (in priority order)
1075
- 2. Create handler file in `transitions/{category}/{name}.js`
1076
- 3. Handler receives full context — use `assistant.log()` for logging, `Manager.project.apiUrl` for API calls
1077
-
1078
- ## Payment System Architecture
1079
-
1080
- ### Pipeline
1081
-
1082
- The payment system follows a linear pipeline: **Intent → Webhook → On-Write → Transition**.
1083
-
1084
- 1. **Intent** (`POST /payments/intent`): Client requests a payment session. BEM validates the product, generates an order ID (`XXXX-XXXX-XXXX`), and delegates to the processor module (e.g., Stripe creates a Checkout Session). Saves to `payments-intents/{orderId}`.
1085
-
1086
- 2. **Webhook** (`POST /payments/webhook?processor=X&key=Y`): Processor sends event data. BEM parses and categorizes the event (`subscription` or `one-time`), extracts the UID, and saves to `payments-webhooks/{eventId}` with `status: 'pending'`.
1087
-
1088
- 3. **On-Write** (Firestore trigger on `payments-webhooks/{eventId}`): Fetches the latest resource from the processor API (not stale webhook data), transforms it into a unified object, detects state transitions, dispatches handlers, tracks analytics, and writes to `users/{uid}.subscription` (subscriptions) and `payments-orders/{orderId}`.
1089
-
1090
- 4. **Transitions** (fire-and-forget): Handler files run asynchronously after detection. Failures never block webhook processing. Skipped during tests unless `TEST_EXTENDED_MODE` is set.
1091
-
1092
- ### 3-Layer Architecture
1093
-
1094
- The payment system is cleanly separated into three independent layers:
1095
-
1096
- | Layer | Purpose | Tests |
1097
- |-------|---------|-------|
1098
- | **Processor input** (Stripe, PayPal, Test) | Parse raw webhooks + transform to unified shape | Helper tests per processor (`payment/stripe/to-unified-subscription.js`, `payment/paypal/to-unified-one-time.js`, etc.) |
1099
- | **Unified pipeline** (processor-agnostic) | Transition detection, Firestore writes, analytics | Journey tests (`journey-payments-*.js`) |
1100
- | **Transition handlers** (fire-and-forget) | Emails, notifications, side effects | Skipped during tests unless `TEST_EXTENDED_MODE` |
1101
-
1102
- Each processor transforms its raw data into the **same unified shape**. Once data enters the pipeline, the code doesn't know or care which processor it came from. This means:
1103
- - Adding a new processor = implement the processor interface (below). The pipeline handles the rest.
1104
- - Journey tests use the `test` processor but exercise the full unified pipeline end-to-end.
1105
- - Processor-specific tests only need to verify correct transformation to the unified shape.
1106
-
1107
- ### Processor Interface
1108
-
1109
- Each processor implements three modules:
1110
-
1111
- **Intent processor** (`routes/payments/intent/processors/{processor}.js`):
1112
- ```javascript
1113
- module.exports = {
1114
- async createIntent({ uid, orderId, product, productId, frequency, trial, confirmationUrl, cancelUrl, Manager, assistant }) {
1115
- return { id, url, raw };
1116
- },
1117
- };
1118
- ```
1119
-
1120
- **Webhook processor** (`routes/payments/webhook/processors/{processor}.js`):
1121
- ```javascript
1122
- module.exports = {
1123
- isSupported(eventType) { return boolean; },
1124
- parseWebhook(req) { return { eventId, eventType, category, resourceType, resourceId, raw, uid }; },
1125
- };
1126
- ```
1127
-
1128
- **Cancel processor** (`routes/payments/cancel/processors/{processor}.js`):
1129
- ```javascript
1130
- module.exports = {
1131
- async cancelAtPeriodEnd({ resourceId, uid, subscription, assistant }) { /* cancel at end of period */ },
1132
- };
1133
- ```
1134
-
1135
- **Refund processor** (`routes/payments/refund/processors/{processor}.js`):
1136
- ```javascript
1137
- module.exports = {
1138
- async processRefund({ resourceId, uid, subscription, assistant }) {
1139
- return { amount, currency, full };
1140
- },
1141
- };
1142
- ```
1143
-
1144
- **Portal processor** (`routes/payments/portal/processors/{processor}.js`):
1145
- ```javascript
1146
- module.exports = {
1147
- async createPortalSession({ resourceId, uid, returnUrl, assistant }) {
1148
- return { url };
1149
- },
1150
- };
1151
- ```
1152
-
1153
- **Shared library** (`libraries/payment/processors/{processor}.js`):
1154
- ```javascript
1155
- module.exports = {
1156
- init() { /* return SDK instance */ },
1157
- async fetchResource(resourceType, resourceId, rawFallback, context) { /* return resource */ },
1158
- getOrderId(resource) { /* return orderId string or null */ },
1159
- toUnifiedSubscription(rawSubscription, options) { /* return unified object */ },
1160
- toUnifiedOneTime(rawResource, options) { /* return unified object */ },
1161
- };
1162
- ```
1163
-
1164
- ### Product Resolution
1165
-
1166
- Products are resolved differently per processor, but always end up matching a product in `config.payment.products`:
1167
-
1168
- | Processor | Resolution chain | Stable ID |
1169
- |-----------|-----------------|-----------|
1170
- | **Stripe** | `sub.items.data[0].price.product` or `raw.plan.product` → match `product.stripe.productId` or `legacyProductIds` | `prod_xxx` |
1171
- | **PayPal** | `sub → plan_id → plan → product_id` → match `product.paypal.productId` | PayPal catalog product ID |
1172
- | **Test** | Uses `product.stripe.productId` in Stripe-shaped data | Same as Stripe |
1173
-
1174
- Falls back to `{ id: 'basic' }` if no match found.
1175
-
1176
- ### Processor-Specific Details
1177
-
1178
- **Stripe:** Uses `metadata.uid` and `metadata.orderId` on subscriptions for UID/order resolution.
1179
-
1180
- **PayPal:** Uses `custom_id` field on subscriptions with format `uid:{uid},orderId:{orderId}`. Product resolution fetches the plan from the subscription, then gets `product_id` from the plan. Plans are scoped by `product_id` query param to avoid cross-brand matches on shared PayPal accounts.
1181
-
1182
- ### Product Configuration
1183
-
1184
- Products are defined in `backend-manager-config.json` under `payment.products`:
1185
-
1186
- ```javascript
1187
- payment: {
1188
- processors: {
1189
- stripe: { publishableKey: 'pk_live_...' },
1190
- paypal: { clientId: 'ARvf...' },
1191
- },
1192
- products: [
1193
- {
1194
- id: 'basic', // Free tier (no prices, no processor keys)
1195
- name: 'Basic',
1196
- type: 'subscription',
1197
- limits: { requests: 100 },
1198
- },
1199
- {
1200
- id: 'premium', // Paid subscription
1201
- name: 'Premium',
1202
- type: 'subscription',
1203
- limits: { requests: 1000 },
1204
- trial: { days: 14 },
1205
- prices: { monthly: 4.99, annually: 49.99 }, // Flat numbers; also supports 'weekly' and 'daily'
1206
- stripe: { productId: 'prod_xxx', legacyProductIds: ['prod_OLD'] },
1207
- paypal: { productId: 'PROD-abc123' },
1208
- },
1209
- {
1210
- id: 'credits-100', // One-time purchase
1211
- name: '100 Credits',
1212
- type: 'one-time',
1213
- prices: { once: 9.99 },
1214
- stripe: { productId: 'prod_yyy' },
1215
- paypal: { productId: null },
1216
- },
1217
- ],
1218
- }
1219
- ```
1220
-
1221
- Key rules:
1222
- - `prices` contains **flat numbers only** — no processor-specific IDs
1223
- - Processor IDs live at the product level: `stripe: { productId }`, `paypal: { productId }`
1224
- - `stripe.productId` is stable — never changes even when prices change
1225
- - `stripe.legacyProductIds` maps old pre-migration Stripe products to this product
1226
- - Price IDs (Stripe `price_xxx`, PayPal plan IDs) are **resolved at runtime** by matching amount + interval against active prices on the processor's product
1227
- - `basic` product has no `prices` and no processor keys — it's the free tier
1228
- - `archived: true` stops offering a product to new subscribers while keeping it resolvable for existing ones
1229
-
1230
- ### Firestore Collections
1231
-
1232
- | Collection | Key | Purpose |
1233
- |---|---|---|
1234
- | `payments-intents/{orderId}` | Order ID | Intent metadata (processor, product, status) |
1235
- | `payments-webhooks/{eventId}` | Processor event ID | Webhook processing state + transition result |
1236
- | `payments-orders/{orderId}` | Order ID | Unified order data (single source of truth for orders) |
1237
- | `users/{uid}.subscription` | User UID | Current subscription state (subscriptions only) |
1238
-
1239
- ### Test Processor
1240
-
1241
- The `test` processor generates Stripe-shaped data and auto-fires webhooks to the local server. Only available in non-production environments. Use `processor: 'test'` in intent requests during testing. The test webhook processor delegates to Stripe's parser since it generates Stripe-shaped payloads.
1242
-
1243
- ## Marketing Custom Fields
1244
-
1245
- BEM syncs user data to marketing providers (SendGrid, Beehiiv) as custom fields. Field definitions live in a single dictionary; OMEGA provisions them in each provider.
1246
-
1247
- ### Adding a New Field
1248
-
1249
- 1. Add the field to `FIELDS` in `src/manager/libraries/email/constants.js` — the key IS the field name in both providers. Set `source`, `path`, `type`.
1250
- 2. Add matching entry in OMEGA's `src/lib/bem-fields.js` with `name`, `display`, `type`. If Beehiiv has it built-in (e.g., country, utm_source), set `beehiivBuiltIn: true`.
1251
- 3. Run OMEGA: `npm start -- --service=sendgrid,beehiiv --brand=X`
1252
- 4. BEM resolves field IDs at runtime — no provider code changes needed.
1253
-
1254
- ### How It Works
1255
-
1256
- - **SendGrid**: `resolveFieldIds()` fetches field definitions from the SendGrid API, builds a name-to-ID cache, and maps values to SendGrid's auto-generated IDs (e.g., `brand_id` maps to `e35_T`).
1257
- - **Beehiiv**: BEM uses the key directly as the custom field name — no ID resolution needed.
1258
- - **OMEGA**: The `ensure/custom-fields.js` handlers are idempotent — they fetch existing fields and only create what is missing.
1259
-
1260
- ### Key Files
1261
-
1262
- | Purpose | File |
1263
- |---------|------|
1264
- | Field dictionary (BEM SSOT) | `src/manager/libraries/email/constants.js` |
1265
- | Field provisioning list (OMEGA SSOT) | `omega-manager/src/lib/bem-fields.js` |
1266
- | SendGrid provisioning | `omega-manager/src/services/sendgrid/ensure/custom-fields.js` |
1267
- | Beehiiv provisioning | `omega-manager/src/services/beehiiv/ensure/custom-fields.js` |
1268
-
1269
- ## Marketing Campaign System
1270
-
1271
- ### Campaign CRUD Routes (admin-only)
1272
-
1273
- | Method | Endpoint | Purpose |
1274
- |--------|----------|---------|
1275
- | POST | `/marketing/campaign` | Create campaign (immediate or scheduled) |
1276
- | GET | `/marketing/campaign` | List/filter campaigns by date range, status, type |
1277
- | PUT | `/marketing/campaign` | Update pending campaigns (reschedule, edit) |
1278
- | DELETE | `/marketing/campaign` | Delete pending campaigns |
1279
-
1280
- ### Firestore Collection: `marketing-campaigns/{id}`
1281
-
1282
- ```javascript
1283
- {
1284
- settings: { name, subject, preheader, content, template, sender, segments, excludeSegments, ... },
1285
- sendAt: 1743465600, // Unix timestamp (any format accepted, normalized on create)
1286
- status: 'pending', // pending | sent | failed
1287
- type: 'email', // email | push
1288
- recurrence: { pattern, hour, day }, // Optional — makes it recurring
1289
- generator: 'newsletter', // Optional — runs content generator before sending
1290
- recurringId: '_recurring-sale', // Present on history docs (links to parent template)
1291
- generatedFrom: '_recurring-newsletter', // Present on generated docs
1292
- results: { sendgrid: {...}, beehiiv: {...} },
1293
- metadata: { created: {...}, updated: {...} },
1294
- }
1295
- ```
1296
-
1297
- ### Campaign Types
1298
-
1299
- - **Email**: dispatches to SendGrid (Single Send) + Beehiiv (Post) via `mailer.sendCampaign()`
1300
- - **Push**: dispatches to FCM via `notification.send()` (shared library)
1301
- - Content is **markdown** — converted to HTML at send time. Template variables resolved before conversion.
1302
-
1303
- ### Recurring Campaigns
1304
-
1305
- Campaigns with a `recurrence` field repeat automatically:
1306
- - Cron fires → creates a **history doc** (same collection, `recurringId` set) → advances `sendAt` to next occurrence
1307
- - Status stays `pending` on the recurring template, history docs are `sent`/`failed`
1308
- - `_` prefix on IDs groups them at top of Firestore console
1309
-
1310
- Recurrence patterns: `daily`, `weekly`, `monthly`, `quarterly`, `yearly`
1311
-
1312
- ### Generator Campaigns
1313
-
1314
- Campaigns with a `generator` field don't send directly. A daily cron pre-generates content 24 hours before `sendAt`:
1315
- 1. Daily cron finds generator campaigns due within 24 hours
1316
- 2. Runs the generator module (e.g., `generators/newsletter.js`)
1317
- 3. Creates a NEW standalone `pending` campaign with generated content
1318
- 4. Advances the recurring template's `sendAt`
1319
- 5. Generated campaign appears on calendar for review, sent by frequent cron when due
1320
-
1321
- ### Template Variables
1322
-
1323
- Resolved at send time via `powertools.template()`. Single braces `{var}` for campaign-level, double `{{var}}` for SendGrid template-level.
1324
-
1325
- | Variable | Example Output |
1326
- |----------|---------------|
1327
- | `{brand.name}` | Somiibo |
1328
- | `{brand.id}` | somiibo |
1329
- | `{brand.url}` | https://somiibo.com |
1330
- | `{season.name}` | Winter, Spring, Summer, Fall |
1331
- | `{holiday.name}` | Black Friday, Christmas, Valentine's Day, etc. |
1332
- | `{date.month}` | November |
1333
- | `{date.year}` | 2026 |
1334
- | `{date.full}` | March 17, 2026 |
1335
-
1336
- ### UTM Auto-Tagging
1337
-
1338
- `libraries/email/utm.js` scans HTML for `<a href>` matching the brand's domain and appends UTM params. Applied to both marketing campaigns and transactional emails.
1339
-
1340
- Defaults: `utm_source=brand.id`, `utm_medium=email`, `utm_campaign=name`, `utm_content=type`. Override via `settings.utm` object.
1341
-
1342
- ### Segments SSOT
1343
-
1344
- `SEGMENTS` dictionary in `constants.js` — 22 segment definitions. OMEGA creates them in SendGrid, BEM resolves keys to provider IDs at runtime via `resolveSegmentIds()` (cached).
1345
-
1346
- | Category | Segments |
1347
- |----------|----------|
1348
- | Subscription (9) | `subscription_free`, `subscription_paid`, `subscription_trialing`, `subscription_cancelling`, `subscription_suspended`, `subscription_cancelled`, `subscription_churned`, `subscription_ever_paid`, `subscription_never_paid` |
1349
- | Lifecycle (5) | `lifecycle_7d`, `lifecycle_30d`, `lifecycle_90d`, `lifecycle_6m`, `lifecycle_1y` |
1350
- | Engagement (5) | `engagement_active_30d`, `engagement_active_90d`, `engagement_inactive_90d`, `engagement_inactive_5m`, `engagement_inactive_6m` |
1351
-
1352
- Campaigns reference segments by SSOT key: `segments: ['subscription_free']`. Auto-translated to provider IDs.
1353
-
1354
- ### Contact Pruning
1355
-
1356
- `cron/daily/marketing-prune.js` — runs 1st of each month. Two stages:
1357
- 1. **Re-engagement**: send email to `engagement_inactive_5m` (excluding `engagement_inactive_6m`)
1358
- 2. **Prune**: export `engagement_inactive_6m` contacts, bulk delete from SendGrid + Beehiiv. Never prunes paying customers.
1359
-
1360
- ### Newsletter Generator
1361
-
1362
- `generators/newsletter.js` — pulls content from parent server, AI assembles branded newsletter.
1363
- 1. Fetch sources: `GET {parentUrl}/newsletter/sources?category=X&claimFor=brandId` (atomic claim)
1364
- 2. AI assembly: GPT-4o-mini generates subject, preheader, and markdown content
1365
- 3. Mark used: `PUT {parentUrl}/newsletter/sources` per source
1366
-
1367
- ### Seed Campaigns
1368
-
1369
- Created by `npx mgr setup` (idempotent, enforced fields checked every run):
1370
-
1371
- | ID | Type | Description |
1372
- |----|------|-------------|
1373
- | `_recurring-sale` | email (sendgrid) | Seasonal sale targeting free + cancelled + churned users |
1374
- | `_recurring-newsletter` | email (beehiiv) | AI-generated newsletter from parent server sources |
1375
-
1376
- ### Marketing Config
1377
-
1378
- ```javascript
1379
- marketing: {
1380
- sendgrid: { enabled: true },
1381
- beehiiv: { enabled: false, publicationId: 'pub_xxxxx' },
1382
- prune: { enabled: true },
1383
- newsletter: { enabled: false, categories: ['social-media', 'marketing'] },
1384
- }
1385
- ```
1386
-
1387
- ### Key Marketing Files
1388
-
1389
- | Purpose | File |
1390
- |---------|------|
1391
- | Marketing library | `src/manager/libraries/email/marketing/index.js` |
1392
- | Field + segment SSOT | `src/manager/libraries/email/constants.js` |
1393
- | UTM tagging | `src/manager/libraries/email/utm.js` |
1394
- | Newsletter generator | `src/manager/libraries/email/generators/newsletter.js` |
1395
- | Notification library | `src/manager/libraries/notification.js` |
1396
- | SendGrid provider | `src/manager/libraries/email/providers/sendgrid.js` |
1397
- | Beehiiv provider | `src/manager/libraries/email/providers/beehiiv.js` |
1398
- | Campaign routes | `src/manager/routes/marketing/campaign/{get,post,put,delete}.js` |
1399
- | Campaign cron | `src/manager/cron/frequent/marketing-campaigns.js` |
1400
- | Newsletter pre-gen cron | `src/manager/cron/daily/marketing-newsletter-generate.js` |
1401
- | Pruning cron | `src/manager/cron/daily/marketing-prune.js` |
1402
- | Seed campaigns | `src/cli/commands/setup-tests/helpers/seed-campaigns.js` |
1403
-
1404
- ## Common Mistakes to Avoid
1405
-
1406
- 1. **Don't modify Manager internals directly** - Use factory methods and public APIs
1407
-
1408
- 2. **Always use `assistant.respond()` for responses** - Don't use `res.send()` directly
1409
-
1410
- 3. **Match schema names to route names** - If route is `myEndpoint`, schema should be `myEndpoint`
1411
-
1412
- 4. **Always await async operations** - Don't forget `await` on Firestore operations
1413
-
1414
- 5. **Handle errors properly** - Use `assistant.errorify()` with appropriate status codes
1415
-
1416
- 6. **Don't call `respond()` multiple times** - Only one response per request
1417
-
1418
- 7. **Use short-circuit returns** - Return early from error conditions
1419
-
1420
- 8. **Increment usage before update** - Call `usage.increment()` then `usage.update()`
1421
-
1422
- 9. **Add Firestore composite indexes for new compound queries** - Any new Firestore query using multiple `.where()` clauses or `.where()` + `.orderBy()` requires a composite index. Add it to `src/cli/commands/setup-tests/helpers/required-indexes.js` (the SSOT). Consumer projects pick these up via `npx mgr setup`, which syncs them into `firestore.indexes.json`. Without the index, the query will crash with `FAILED_PRECONDITION` in production.
1423
-
1424
- ## Key Files Reference
1425
-
1426
- | Purpose | File |
1427
- |---------|------|
1428
- | Main Manager class | `src/manager/index.js` |
1429
- | Request/response handling | `src/manager/helpers/assistant.js` |
1430
- | Middleware pipeline | `src/manager/helpers/middleware.js` |
1431
- | Schema validation | `src/manager/helpers/settings.js` |
1432
- | Rate limiting | `src/manager/helpers/usage.js` |
1433
- | User properties + schema | `src/manager/helpers/user.js` |
1434
- | Batch utilities | `src/manager/helpers/utilities.js` |
1435
- | Auth: before-create | `src/manager/events/auth/before-create.js` |
1436
- | Auth: before-signin | `src/manager/events/auth/before-signin.js` |
1437
- | Auth: on-create | `src/manager/events/auth/on-create.js` |
1438
- | Auth: on-delete | `src/manager/events/auth/on-delete.js` |
1439
- | Auth: shared utilities | `src/manager/events/auth/utils.js` |
1440
- | Cron runner | `src/manager/events/cron/runner.js` |
1441
- | Main API handler | `src/manager/functions/core/actions/api.js` |
1442
- | Config template | `templates/backend-manager-config.json` |
1443
- | CLI entry | `src/cli/index.js` |
1444
- | Stripe webhook forwarding | `src/cli/commands/stripe.js` |
1445
- | Firebase init helper (CLI) | `src/cli/commands/firebase-init.js` |
1446
- | Firestore CLI commands | `src/cli/commands/firestore.js` |
1447
- | Auth CLI commands | `src/cli/commands/auth.js` |
1448
- | Logs CLI commands | `src/cli/commands/logs.js` |
1449
- | Intent creation | `src/manager/routes/payments/intent/post.js` |
1450
- | Webhook ingestion | `src/manager/routes/payments/webhook/post.js` |
1451
- | Webhook processing (on-write) | `src/manager/events/firestore/payments-webhooks/on-write.js` |
1452
- | Payment analytics | `src/manager/events/firestore/payments-webhooks/analytics.js` |
1453
- | Transition detection | `src/manager/events/firestore/payments-webhooks/transitions/index.js` |
1454
- | Payment processor libraries | `src/manager/libraries/payment/processors/` |
1455
- | Stripe library | `src/manager/libraries/payment/processors/stripe.js` |
1456
- | PayPal library | `src/manager/libraries/payment/processors/paypal.js` |
1457
- | Order ID generator | `src/manager/libraries/payment/order-id.js` |
1458
- | Required Firestore indexes (SSOT) | `src/cli/commands/setup-tests/helpers/required-indexes.js` |
1459
- | Test accounts | `src/test/test-accounts.js` |
1460
-
1461
- ## Environment Detection
1462
-
1463
- ```javascript
1464
- assistant.isDevelopment() // true when ENVIRONMENT !== 'production' or in emulator
1465
- assistant.isProduction() // true when ENVIRONMENT === 'production'
1466
- assistant.isTesting() // true when running tests (via npx mgr test)
1467
- ```
1468
-
1469
- ## Model Context Protocol (MCP)
1470
-
1471
- BEM includes a built-in MCP server that exposes BEM routes as tools for Claude Chat, Claude Code, and other MCP clients.
1472
-
1473
- ### Architecture
1474
-
1475
- Two transport modes:
1476
- - **Stdio** (local): `npx mgr mcp` — for Claude Code / Claude Desktop
1477
- - **Streamable HTTP** (remote): `POST /backend-manager/mcp` — for Claude Chat (stateless, Firebase Functions compatible)
1478
-
1479
- ### Available Tools (19)
1480
-
1481
- | Tool | Route | Description |
1482
- |------|-------|-------------|
1483
- | `firestore_read` | `GET /admin/firestore` | Read a Firestore document by path |
1484
- | `firestore_write` | `POST /admin/firestore` | Write/merge a Firestore document |
1485
- | `firestore_query` | `POST /admin/firestore/query` | Query a collection with where/orderBy/limit |
1486
- | `send_email` | `POST /admin/email` | Send transactional email via SendGrid |
1487
- | `send_notification` | `POST /admin/notification` | Send push notification via FCM |
1488
- | `get_user` | `GET /user` | Get authenticated user info |
1489
- | `get_subscription` | `GET /user/subscription` | Get subscription info for a user |
1490
- | `sync_users` | `POST /admin/users/sync` | Sync user data across systems |
1491
- | `list_campaigns` | `GET /marketing/campaign` | List marketing campaigns |
1492
- | `create_campaign` | `POST /marketing/campaign` | Create a marketing campaign |
1493
- | `get_stats` | `GET /admin/stats` | Get system statistics |
1494
- | `cancel_subscription` | `POST /payments/cancel` | Cancel subscription at period end |
1495
- | `refund_payment` | `POST /payments/refund` | Process a refund |
1496
- | `run_cron` | `POST /admin/cron` | Trigger a cron job by ID |
1497
- | `create_post` | `POST /admin/post` | Create a blog post |
1498
- | `create_backup` | `POST /admin/backup` | Create a Firestore backup |
1499
- | `run_hook` | `POST /admin/hook` | Execute a custom hook |
1500
- | `generate_uuid` | `POST /general/uuid` | Generate a UUID |
1501
- | `health_check` | `GET /test/health` | Check server health |
1502
-
1503
- ### Authentication
1504
-
1505
- - **Stdio (local):** Reads `BACKEND_MANAGER_KEY` from `functions/.env` automatically
1506
- - **HTTP (remote):** OAuth 2.1 Authorization Code flow with PKCE. Claude Chat handles the flow — user pastes BEM key once on the authorize page. If `OAuth Client ID` is set to the BEM key in the connector config, the authorize step auto-approves.
1507
-
1508
- ### Hosting Rewrites
1509
-
1510
- The `npx mgr setup` command automatically adds required Firebase Hosting rewrites for MCP OAuth:
1511
- ```json
1512
- {
1513
- "source": "{/backend-manager,/backend-manager/**,/.well-known/oauth-protected-resource,/.well-known/oauth-authorization-server,/authorize,/token}",
1514
- "function": "bm_api"
1515
- }
1516
- ```
1517
-
1518
- ### CLI Usage
1519
-
1520
- ```bash
1521
- npx mgr mcp # Start stdio MCP server (for Claude Code)
1522
- ```
1523
-
1524
- ### Claude Code Configuration
1525
-
1526
- Add to `.claude/settings.json`:
1527
- ```json
1528
- {
1529
- "mcpServers": {
1530
- "backend-manager": {
1531
- "command": "npx",
1532
- "args": ["bm", "mcp"],
1533
- "cwd": "/path/to/consumer-project"
1534
- }
1535
- }
1536
- }
1537
- ```
1538
-
1539
- ### Claude Chat Configuration
1540
-
1541
- 1. Go to Settings → Custom Connectors → Add
1542
- 2. **URL:** `https://api.yourdomain.com/backend-manager/mcp`
1543
- 3. **OAuth Client ID:** your `BACKEND_MANAGER_KEY` (enables auto-approve)
1544
- 4. **OAuth Client Secret:** your `BACKEND_MANAGER_KEY`
1545
-
1546
- ### Key Files
1547
-
1548
- | Purpose | File |
1549
- |---------|------|
1550
- | Tool definitions | `src/mcp/tools.js` |
1551
- | HTTP handler (stateless + OAuth) | `src/mcp/handler.js` |
1552
- | Stdio server | `src/mcp/index.js` |
1553
- | HTTP client | `src/mcp/client.js` |
1554
- | CLI command | `src/cli/commands/mcp.js` |
1555
- | MCP route interception | `src/manager/index.js` (`_handleMcp`, `resolveMcpRoutePath`) |
1556
- | Hosting rewrites setup | `src/cli/commands/setup-tests/hosting-rewrites.js` |
1557
-
1558
- ### Adding New Tools
1559
-
1560
- 1. Add the tool definition to `src/mcp/tools.js` with `name`, `description`, `method`, `path`, and `inputSchema`
1561
- 2. The tool automatically maps to the corresponding BEM route via the HTTP client — no handler code needed
1562
-
1563
- ## Response Headers
1564
-
1565
- BEM automatically sets `bm-properties` header with:
1566
- - `code`: HTTP status code
1567
- - `tag`: Function name and execution ID
1568
- - `usage`: Current usage stats
1569
- - `schema`: Resolved schema info
138
+ - [docs/testing.md](docs/testing.md) running, filtering, log files, test types (standalone/suite/group), context object, assertions, auth levels
139
+ - [docs/cli-firestore-auth.md](docs/cli-firestore-auth.md) — `npx mgr firestore:*` and `auth:*` commands, shared flags, examples
140
+ - [docs/cli-logs.md](docs/cli-logs.md) — `npx mgr logs:read` / `logs:tail` with full flag reference and built-in Cloud Function names