backend-manager 5.5.4 → 5.6.1

package/CHANGELOG.md

@@ -14,6 +14,35 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - `Fixed` for any bug fixes.
 - `Security` in case of vulnerabilities.
 
+# [5.6.1] - 2026-06-11
+
+### Added
+- **`docs/audit.md` — full-audit check catalog (`/omega:bem audit`).** ID'd, severity-graded checks with scope auto-detect (consumer vs framework via `functions/package.json`): mirrored universal checks (U-01..U-14 — tests at every surface, sanitization, secrets incl. `service-account.json`, config canon, doc parity, dead/legacy patterns, dep health, …), BEM-specific checks (BEM-01..BEM-09 — name-matched context-object schemas, the required-vs-default footgun, ownership checks + `assistant.respond()`, index.js/rewrites wiring, Firestore canon, usage helper + rate limits, composite indexes, auth gates, rules coverage), and framework-repo checks (F-01..F-04). Findings persist to `functions/.temp/audit/claude-audit.md`; fixes run as a severity-ordered TodoWrite loop ending with a green `npx mgr test`. Wired to the `omega:bem` router's Audit process; `docs/audit.md` is mirrored across UJM/BXM/EM. Indexed in CLAUDE.md.
+
+### Changed
+- **package.json `keywords` corrected** — replaced the thin generic set (`cli`, `backend manager`, `firebase`) with accurate, discovery-oriented ones (`firebase`, `firebase-functions`, `cloud-functions`, `firestore`, `backend`, `serverless`, `api`, `express`, `cli`). npm-listing metadata only; no behavior change. Mirrored across UJM/BXM/EM.
+
+# [5.6.0] - 2026-06-11
+
+### Added
+- **`docs/migration.md` (action-skill consolidation).** The standalone `BEM:migrate` skill was deleted and folded into `omega:bem` as a process checklist; its playbooks landed in the repo as `docs/migration.md` — env-var migration (runtime config → top-level `functions/.env` keys with the full mapping table), legacy code conversion (`Manager.config.*` → `process.env.*`), and route/schema migration to the current context-object/flat formats. Indexed in CLAUDE.md.
+- **Dev-process guidance relaxed: only `npx mgr serve` / `npx mgr emulator` are off-limits.** The "NEVER run" rule in CLAUDE.md now prohibits only the long-running dev processes (instruct the user to start them if they aren't running; read `functions/*.log`, never tail) — `npx mgr test` is fine to run (it auto-starts its own emulator).
+- **Skills-as-routers migration — `docs/firestore.md` (new) + `routes.md`/`schemas.md` rewrites + `test-framework.md`/`usage-rate-limiting.md` extensions.** Framework facts migrated from the `omega:bem` skill into the repo so they version-match the installed package, with stale content corrected against source: `routes.md`'s consumer-route recipe now shows the CURRENT context-object handler (`module.exports = async ({ Manager, assistant, user, settings, ... })` — middleware calls `routeHandler(context)`; the old `Route.prototype.main` constructor example was stale) plus the CRUD method-file table, ownership checks, plural naming, firebase.json rewrite syntax + first-match ordering, and the `functions/index.js` entry pattern; `schemas.md` rewritten to the current contract (context object `{ assistant, user, data, method, headers, geolocation, client }` → FLAT schema with in-function plan branching — the old positional-args + `defaults:/premium:` tier examples were stale) plus field properties (`value`/`clean`/function `required`), the required-vs-default footgun, and the ID-generation/path-extraction patterns; new `firestore.md` carries the Firestore conventions (`.doc('col/id')` style, NO subcollections, ~500-doc cursor-paginated batch reads, `metadata.{created,updated}` timestamps, mirror-the-doc response format + delete-don't-redact); `test-framework.md` gains the `rules` client reference (`asAccount`/`expectSuccess`/`expectFailure` + seed-as-admin pattern), the journey-account isolation rule (CRITICAL — never use shared accounts with the `test` processor), naming conventions, and common patterns (auth rejection, concurrent-duplicate rejection, suite cleanup); `usage-rate-limiting.md` gains the core API table (`validate`/`increment`/`update`/`getLimit`/`getProduct`/`getUsage`) and the never-write-usage-manually rules. All indexed in CLAUDE.md; the skill is now a thin router (pointers + hard rules + process checklists).
+- **`--extended` CLI flag for cross-framework parity.** `npx mgr test --extended` is now the CLI shorthand for the shared, unprefixed `TEST_EXTENDED_MODE` env var (standardized across BEM/BXM/UJM/EM) — equivalent to `TEST_EXTENDED_MODE=true npx mgr test`. The flag is read pre-flight in `src/cli/commands/test.js` (before the `captureSyncedEnv`/`writeTestMode` pre-flight), so it propagates to BOTH the test-runner subprocess (`{ ...process.env }`) AND the live emulator (via `.temp/test-mode.json`) exactly like the env var. The env-var path keeps working — env var OR flag opts into REAL external services (default: skipped).
+- **Framework self-test from the repo (bundled fixture + `BEM_TEST_BOOT_PROJECT`).** `npx mgr test` run from the backend-manager repo now boots a bundled fixture Firebase project ([`src/test/fixtures/firebase-project/`](src/test/fixtures/firebase-project)) and runs a `test/boot/` smoke suite (emulator boots → fixture `Manager.init()` wires `bm_api` → health returns 200), instead of failing with "no firebase.json". Brings BEM into parity with BXM's `BXM_TEST_BOOT_PROJECT` / UJM's `UJ_TEST_BOOT_PROJECT`. The runner symlinks the local `backend-manager` (+ `firebase-admin`/`firebase-functions`) into the fixture, injects the fixture admin keys, and generates a throwaway emulator-only service account at runtime (all gitignored). `BEM_TEST_BOOT_PROJECT=<path>` overrides the fixture to self-test against a real consumer. The `boot/` smoke is excluded from consumer runs; the full `routes`/`events`/`rules` suites still run against a real consumer as before.
+- **Docs parity — new `docs/build-system.md` + `docs/logging.md`.** `build-system.md` documents BEM's deliberate outlier status (no consumer build pipeline; framework prepare-package; deploy flow); `logging.md` is now the SSOT for the `functions/*.log` file table (extracted from test-framework.md, which keeps a pointer — mirrors EM/BXM/UJM). Both indexed in CLAUDE.md → Documentation.
+- **Test coverage convention (docs).** New mirrored "Test coverage" sections in `CLAUDE.md`, `docs/test-framework.md`, `src/defaults/CLAUDE.md`, and `src/defaults/test/README.md` — every feature ships with tests at every surface it exposes (logic via handler suites, wiring via `http.as(...)` round-trips, rules suites when Firestore rules change); a surface is skipped only when the feature genuinely doesn't have one. UI coverage explicitly lives in the consuming frontends. Mirrored across EM/BXM/UJM.
+- **Universal `mgr:` test source prefix.** `npx mgr test` now accepts the universal cross-framework `mgr:` prefix (alias for `bem:`) to run framework-only tests, complementing the existing `bem:` and `project:` prefixes. `npx mgr test mgr:` runs all framework tests, `npx mgr test mgr:<path>` runs framework tests matching a path, and multiple space-separated targets compose (e.g. `npx mgr test bem:rules project:routes`).
+
+### Changed
+- **Router skill renamed `BEM:patterns` → `omega:bem`** — all framework skills now live under the `omega:` namespace (`omega:em`/`omega:bxm`/`omega:ujm`/`omega:bem` + the `omega:main` hub). CLAUDE.md's Recommended skills section updated.
+- **`docs/testing.md` renamed `docs/test-framework.md`** (H1 `# Testing` → `# Test Framework`) for cross-framework doc-file parity — EM/BXM/UJM all name their test reference `docs/test-framework.md`, and the mirrored docs must match down to the file name. All references updated (`CLAUDE.md`, `README.md`, `docs/*.md` cross-links, `src/defaults/CLAUDE.md`, `src/defaults/test/_init.js`, historical CHANGELOG links).
+- **Log files renamed for cross-framework parity.** `functions/serve.log` → `functions/dev.log` (the `npx mgr serve` dev-server output) and `functions/logs.log` → `functions/production.log` (the `npx mgr logs` Cloud Logging output). The `dev`/`test` names now match EM/BXM/UJM; `emulator.log` and `test.log` are unchanged. BEM logs still live in `functions/` (not `logs/`) — that directory is a deliberate exception so they sit beside firebase-tools' own `*-debug.log` files. The watcher reset sentinel `serve.log.reset` is correspondingly `dev.log.reset` (internal, in `.temp/`).
+
+### Fixed
+- **`npm publish` vs the fixture's runtime symlinks.** New `prepublishOnly` script removes `src/test/fixtures/firebase-project/functions/node_modules` before packing — the self-test's `backend-manager` symlink points back at the repo root, and prepare-package's publish-time cleanup walk (`jetpack.find` with a top-level-only `!node_modules/**` exclusion) followed the cycle until `ENAMETOOLONG`. The symlinks are throwaway runtime artifacts; the next self-test run regenerates them via `linkFixtureDeps()`.
+- **Fixture `.firebaserc` re-included over the global `.gitignore` rule** (`!src/test/fixtures/firebase-project/.firebaserc`) — the emulator boots with no `--project` flag and resolves the demo project from `.firebaserc`, so a fresh clone's self-test would have failed without it.
+
 # [5.5.4] - 2026-06-09
 
 ### Fixed
@@ -61,7 +90,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 ### Added
 - **Push notification campaigns.** Marketing campaign POST route and cron now handle `type: 'push'` with brand-aware icon/clickAction defaults and test-mode owner filtering.
 - **`filters` field** in campaign schema for push notification targeting.
-- **Test filtering docs** in CLAUDE.md and docs/testing.md (`npx mgr test <path>`, `bem:`, `project:` prefixes).
+- **Test filtering docs** in CLAUDE.md and docs/test-framework.md (`npx mgr test <path>`, `bem:`, `project:` prefixes).
 
 ### Fixed
 - **Email migration shims (temporary).** Old template names (`default` → `card`, `core/engagement/feedback` → `feedback`) and old data format (`data.body` → `data.content`) mapped for queued emails saved before the MJML migration.
@@ -163,7 +192,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - **AI tools passthrough (nested) — `ai.request({ tools: { list, choice } })`.** `tools.list` is an array of tool definitions passed to the OpenAI Responses API verbatim — built-in hosted tools (e.g. `{ type: 'web_search' }`) OR custom function tools (`{ type: 'function', name, parameters }`); `tools.choice` *(optional)* maps to `tool_choice`. Opt-in — omitted/empty means no tools and identical behavior to a plain request. Primary use is OpenAI's built-in **web search** so the model finds and cites real, currently-live URLs instead of hallucinating them. When tools are active the response `output` may carry tool-call items (e.g. `web_search_call`) + `url_citation` annotations; the message-text extractor ignores non-message items so `r.content` is unaffected. See [docs/ai-library.md](docs/ai-library.md).
 - **`image-illustrator.js` — newsletter section illustrations now generated as flat-vector PNGs via `gpt-image-2` by default**, replacing the SVG-author-then-rasterize approach as the default. Clean flat 2D vector style (Stripe / Linear / undraw.co aesthetic) built from the brand palette (`content.theme.{primary,secondary,accent}Color`), white background, no text. The legacy `svg-illustrator.js` method is still available per-brand via `marketing.beehiiv.content.method.image = 'svg'`. Both return the same `{ png: Buffer, fallback, meta }` contract, so `image-host.js` / `uploadAssets` are unchanged. Validated end-to-end against live `gpt-image-2` with Somiibo's real config. See [docs/marketing-campaigns.md](docs/marketing-campaigns.md).
 - **Full emulator-Firestore flush before every test run.** `deleteTestUsers()` now calls `flushEmulatorFirestore()` — `listCollections()` + `recursiveDelete()` on the entire emulator DB — before recreating test accounts. The emulator DB is 100% test data, so a full flush is the simplest correct clean slate; there are no per-collection allowlists to maintain. Guarded to run ONLY when `FIRESTORE_EMULATOR_HOST` is set, so it can never touch a real project.
-- **`test/_init.js` pre-test lifecycle hook.** The test runner loads an optional `test/_init.js` from BOTH test roots (BEM core + consumer project) and runs it before any test (it is not run as a test itself). The module **must export a function** — `module.exports = (ctx) => ({ ... })` — called with `{ config, Manager }` and returning the hook object. It may declare `accounts` (array of extra test accounts `{ id, uid, email, properties }`, created/fetched/deleted on the same path as the built-ins so a project has a user per lifecycle) and `async setup({ admin, config, accounts, Manager, assistant })` (reseed fixtures into the freshly-flushed DB, after account creation). There is **no `cleanup` hook** — the whole DB is flushed each run and each test cleans up after itself. A default boilerplate `test/_init.js` now ships via `src/defaults/` (copied into consumers on first `npx mgr setup`, never overwriting an existing one). Mirrored across all four OMEGA frameworks. See `docs/testing.md`.
+- **`test/_init.js` pre-test lifecycle hook.** The test runner loads an optional `test/_init.js` from BOTH test roots (BEM core + consumer project) and runs it before any test (it is not run as a test itself). The module **must export a function** — `module.exports = (ctx) => ({ ... })` — called with `{ config, Manager }` and returning the hook object. It may declare `accounts` (array of extra test accounts `{ id, uid, email, properties }`, created/fetched/deleted on the same path as the built-ins so a project has a user per lifecycle) and `async setup({ admin, config, accounts, Manager, assistant })` (reseed fixtures into the freshly-flushed DB, after account creation). There is **no `cleanup` hook** — the whole DB is flushed each run and each test cleans up after itself. A default boilerplate `test/_init.js` now ships via `src/defaults/` (copied into consumers on first `npx mgr setup`, never overwriting an existing one). Mirrored across all four OMEGA frameworks. See `docs/test-framework.md`.
 
 ### Changed
 - **Environment detection consolidated onto the Manager as SSOT.** `getEnvironment()` returns exactly one of `development | testing | production` (mutually exclusive, testing wins), read **live** from `process.env` on every call (no caching). `assistant.isDevelopment/isProduction/isTesting/getEnvironment` forward to the Manager. Fixes a bug where a cached environment made `getApiUrl()` resolve to the production URL inside the test runner.
@@ -243,7 +272,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - **`AI` `claude-code` provider rewritten for serverside use** (`src/manager/libraries/ai/providers/claude-code.js`). Was a local-only wrapper around `@anthropic-ai/claude-agent-sdk` that spawned the `claude` binary (`forceLoginMethod: 'claudeai'`, keychain OAuth) — would not run in Cloud Functions. Now calls the Claude Messages API over plain HTTPS via `@anthropic-ai/sdk` using the OAuth token as `Authorization: Bearer` + `anthropic-beta: oauth-2025-04-20`, so it bills the Claude Pro/Max subscription (not API credits) and runs anywhere Node runs. Token resolution: `options.apiKey` → `config.claude_code.oauth_token` → `process.env.CLAUDE_CODE_OAUTH_TOKEN`. Renewal is a manual yearly `claude setup-token` (no auto-refresh). Verified live (text + JSON-schema paths). See `docs/ai-library.md`.
 - **Dependency bumps**: `@anthropic-ai/claude-agent-sdk` ^0.3.152 → ^0.3.153, `stripe` ^22.1.1 → ^22.2.0.
 - **`templates/_.env`**: consolidate OpenAI/Anthropic keys under an "AI" section and add `CLAUDE_CODE_OAUTH_TOKEN`.
-- **Marketing webhook tests** (`test/routes/marketing/webhook.js`): renamed `*-duplicate-event-skipped` → `*-reprocessed-idempotently` (assert re-delivery reprocesses and the user stays revoked); `sendgrid-event-without-eventId-*` now asserts the event is processed; beehiiv idempotency variant skips when no publication is configured. Updated `docs/consent.md` and `docs/testing.md` to describe the no-ledger design.
+- **Marketing webhook tests** (`test/routes/marketing/webhook.js`): renamed `*-duplicate-event-skipped` → `*-reprocessed-idempotently` (assert re-delivery reprocesses and the user stays revoked); `sendgrid-event-without-eventId-*` now asserts the event is processed; beehiiv idempotency variant skips when no publication is configured. Updated `docs/consent.md` and `docs/test-framework.md` to describe the no-ledger design.
 
 # [5.2.12] - 2026-05-27
 
@@ -384,7 +413,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - **Cross-provider unsubscribe webhooks (Phase E).** New `POST /marketing/webhook?provider={sendgrid|beehiiv}&key=...` dispatcher with per-provider processor modules. SendGrid events (`unsubscribe`, `group_unsubscribe`, `spamreport`, `bounce`, `dropped`) and Beehiiv events (`subscription.unsubscribed`, `.deleted`, `.paused`) flip `consent.marketing.status` to `revoked`, attribute via `source`, and propagate to the OTHER provider. Idempotent via `marketing-webhooks/{eventId}` docs.
 - **Parent BEM forwarder.** `POST /marketing/webhook/forward` lets a parent BEM (one with `config.parent === 'self'`) fan webhook events out to sibling brands sharing a SendGrid account or Beehiiv publication. New `Manager.getParentUrl()`, `Manager.getParentApiUrl()`, `Manager.isParent()` helpers — children store the parent's `brand.url` with NO `api.` subdomain and the helper inserts `api.` at call time.
 - **Self-contained TEST_EXTENDED_MODE.** `src/test/runner.js` + `src/test/test-accounts.js` now do pre + post-run cleanup of SendGrid/Beehiiv contacts (the only third-party state we can't wipe at start). Pure Firestore/Auth state is still wiped only at start, per existing convention.
-- **New docs.** `docs/consent.md` (consent system + webhook flows + migration template). `docs/testing.md` updated with the post-run-cleanup exception.
+- **New docs.** `docs/consent.md` (consent system + webhook flows + migration template). `docs/test-framework.md` updated with the post-run-cleanup exception.
 
 ### Changed
 

package/CLAUDE.md

@@ -10,7 +10,7 @@ Backend Manager (BEM) is a comprehensive framework for building modern Firebase
 
 ## Recommended skills
 
-- **`BEM:patterns`** — SSOT for Backend Manager routes, schemas, tests, Firebase functions, Firestore rules, usage tracking patterns. Auto-loads on BEM-specific keywords (`route`, `schema`, `endpoint`, `bm_api`, `Manager.init`, `npx mgr test`, `gcloud logs`, etc.) and when touching files in `functions/routes/`, `functions/schemas/`, `functions/index.js`, `test/`, `src/cli/commands/`.
+- **`omega:bem`** — router skill. Auto-loads on BEM-specific keywords (`route`, `schema`, `endpoint`, `bm_api`, `Manager.init`, `npx mgr test`, `gcloud logs`, etc.) and points back to this CLAUDE.md + `docs/` (the SSOT), carrying only Claude-workflow hard rules and process checklists.
 - **`js:patterns`** — JavaScript/Node.js conventions: file structure, JSDoc, defensive coding (`?.` usage), template literals, `package.json` conventions. Auto-loads when creating new `.js` files or touching JS module structure.
 
 ## Quick Start
@@ -21,11 +21,15 @@ Backend Manager (BEM) is a comprehensive framework for building modern Firebase
 2. `npx mgr setup` — validates config, scaffolds defaults (CLAUDE.md, CHANGELOG.md, docs/, test/), provisions Firestore indexes
 3. `npx mgr emulator` — start Firebase emulators (auth/firestore/functions/database/storage)
 4. `npx mgr serve` — local serve with Stripe webhook forwarding (if `STRIPE_SECRET_KEY` is set)
-5. `npx mgr test` — runs framework + project test suites against an emulator
-   - `npx mgr test email/transactional` — run a specific test by path (relative to `test/`)
-   - `npx mgr test bem:email/templates` — run only BEM framework tests matching a path
-   - `npx mgr test project:routes/custom` — run only consumer project tests matching a path
-   - Prefix with `TEST_EXTENDED_MODE=true` for tests that hit real external APIs (SendGrid, OpenAI, etc.)
+5. `npx mgr test` — runs framework + project test suites against an emulator. Positional target(s) select which test FILES run, by source + path (multiple space-separated targets compose):
+   - `npx mgr test` — everything (framework + project suites)
+   - `npx mgr test email/transactional` — bare path (no prefix): both sources, matched by path (relative to `test/`)
+   - `npx mgr test mgr:` / `npx mgr test bem:` — ONLY framework tests (`mgr:` is the universal cross-framework alias for the manager's own tests; `bem:` is the equivalent BEM-specific alias)
+   - `npx mgr test mgr:email/templates` / `npx mgr test bem:email/templates` — only framework tests matching a path
+   - `npx mgr test project:` — ONLY project tests (all of them)
+   - `npx mgr test project:routes/custom` — only consumer project tests matching a path
+   - `npx mgr test bem:rules project:routes` — multiple targets compose (runs both selections)
+   - Pass `--extended` (or prefix `TEST_EXTENDED_MODE=true`) for tests that hit real external APIs (SendGrid, OpenAI, etc.). `--extended` is the CLI shorthand for the shared, unprefixed `TEST_EXTENDED_MODE` env var standardized across BEM/BXM/UJM/EM; BEM propagates it to BOTH the runner subprocess and the live emulator. See [docs/test-framework.md](docs/test-framework.md#extended-mode-test_extended_mode).
 6. `npx mgr deploy` — deploy Cloud Functions to Firebase
 7. `npx mgr logs:read` / `npx mgr logs:tail` — Cloud Function logs from Google Cloud Logging
 
@@ -38,7 +42,7 @@ All `npx mgr <cmd>` aliases work: `npx bm <cmd>`, `npx bem <cmd>`, `npx backend-
 1. `npm install` — install BEM's own deps
 2. `npm run prepare` — build once: copies `src/` → `dist/` via prepare-package
 3. `npm run prepare:watch` — watch mode
-4. Test in a consumer project: from inside the consumer, run `npx mgr install dev` to swap BEM to this local repo — required whenever you edit the framework source and want the consumer to pick up the changes (the consumer otherwise keeps its installed `node_modules/backend-manager`). Reverse with `npx mgr install live`. If `npx mgr` then errors with "could not determine executable to run", the local install skipped bin-linking — re-run `npm install` to relink, or call `node node_modules/backend-manager/bin/backend-manager <cmd>` directly.
+4. Test in the **designated test consumer** — `../ultimate-jekyll-backend` is BEM's consumer for validating framework changes end-to-end (exercise any consumer-level flow there freely: emulator, tests, deploy paths). From inside it, run `npx mgr install dev` to swap BEM to this local repo — required whenever you edit the framework source and want the consumer to pick up the changes (the consumer otherwise keeps its installed `node_modules/backend-manager`). Reverse with `npx mgr install live`. If `npx mgr` then errors with "could not determine executable to run", the local install skipped bin-linking — re-run `npm install` to relink, or call `node node_modules/backend-manager/bin/backend-manager <cmd>` directly.
 
 ## Architecture
 
@@ -46,6 +50,14 @@ BEM exposes a single `Manager` class that orchestrates everything: it initialize
 
 For the directory layout of both the BEM library and consumer projects, see [docs/directory-structure.md](docs/directory-structure.md).
 
+### Test framework
+
+`npx mgr test` runs framework + project suites against a **real Firebase emulator** (real Firestore/Auth — never mocked). Suites are organized by concern (`test/routes/`, `test/events/`, `test/rules/`, …) rather than runtime layers. See [docs/test-framework.md](docs/test-framework.md).
+
+### Test coverage
+
+Every feature ships with tests at EVERY surface it exposes — logic (`test/routes/`/`test/events/` handler suites against the real emulator), wiring (route round-trips over `http.as(...)` — registration, auth gates, schema validation; this IS BEM's end-to-end), and rules (Firestore security-rules suites when rules change). BEM has no UI layer — a feature's UI coverage lives in the consuming frontend (UJM/BXM/EM). Skip a surface ONLY when the feature genuinely doesn't have one; "the handler test already covers it" is NOT a reason to skip the route round-trip. See [docs/test-framework.md](docs/test-framework.md).
+
 ## CLI
 
 `npx mgr <command>` (aliases `bm`, `bem`, `backend-manager`):
@@ -70,10 +82,17 @@ For the directory layout of both the BEM library and consumer projects, see [doc
 
 See [docs/cli-firestore-auth.md](docs/cli-firestore-auth.md) and [docs/cli-logs.md](docs/cli-logs.md) for full flag references.
 
+## Dependency Resolution
+
+- **Consumer code can use `Manager.require(name)`** to load any BEM dependency from BEM's own module context (static + prototype). Consumer projects do NOT need to install BEM's transitive deps directly.
+- **web-manager owns Firebase on the client side.** Consumer frontend code (UJM pages, BXM popup/options, EM renderers) NEVER imports Firebase directly — `firebase.firestore()` → `webManager.firestore()`, `firebase.auth()` → `webManager.auth()`. BEM backend code uses `firebase-admin` directly (server-side is different). The three frontend frameworks (EM/BXM/UJM) additionally resolve deps via webpack `resolve.modules`.
+
 ## Development Workflow
 
-- **🚫 NEVER run `npm start` / `npm test` / `npx mgr emulator`** unless the user explicitly asks. Assume the user is already running the emulator or dev process. Running these commands kills the user's process and wastes time. Instead, **check output logs** after editing files to confirm the change took effect.
+- **🚫 NEVER run `npx mgr serve` / `npx mgr emulator`** — they're the user's long-running dev processes. Assume they're already running; if they aren't, **instruct the user to run them** rather than running them yourself (running them again kills theirs). To see output, **read the `functions/*.log` files** (`dev.log`, `emulator.log`, `test.log`) — never tail/attach to the process. Running `npx mgr test` is fine (it auto-starts its own emulator if needed).
+- **Where the output logs live:** BEM CLI commands tee output to `<projectDir>/functions/` (not `logs/` — BEM's deliberate exception, co-located with firebase-tools' `*-debug.log`): `dev.log` (`npx mgr serve`), `emulator.log` (`npx mgr emulator` / test with own emulator), `test.log` (`npx mgr test`), `production.log` (`npx mgr logs`). The `dev`/`test` names match EM/BXM/UJM; see [docs/test-framework.md](docs/test-framework.md#log-files).
 - **If the user reports an error**, check the emulator/test output for the root cause before guessing.
+- **Live-test UI changes via CDP.** When working on admin dashboards or browser-facing endpoints, use the `chrome-devtools` MCP tools (screenshots, click, evaluate JS, console logs) to verify the change works in the running browser. See `~/.claude/mcp-server/servers/chrome-devtools/CLAUDE.md`.
 
 ## File Conventions
 
@@ -112,9 +131,11 @@ Deep references live in `docs/`. **Whenever you make a behavioral change, update
 
 - [docs/architecture.md](docs/architecture.md) — Manager class, dual-mode (firebase/custom), helper factory pattern
 - [docs/directory-structure.md](docs/directory-structure.md) — BEM library + consumer project layouts
+- [docs/build-system.md](docs/build-system.md) — no consumer build (deliberate outlier), framework prepare-package, deploy pipeline
 - [docs/code-patterns.md](docs/code-patterns.md) — short-circuit returns, logical operators on new lines, Firestore shorthand, template-string requires, fs-jetpack preference
 - [docs/file-naming.md](docs/file-naming.md) — naming table for routes, schemas, API commands, events, cron jobs, hooks
 - [docs/common-mistakes.md](docs/common-mistakes.md) — anti-pattern checklist (don't modify Manager internals, always await, increment-before-update, etc.)
+- [docs/audit.md](docs/audit.md) — full-audit check catalog (U-xx universal / BEM-xx / F-xx IDs with severity + scope), protocol + fix loop
 - [docs/key-files.md](docs/key-files.md) — quick lookup for the most-touched files (Manager, helpers, auth events, cron, payment processors, CLI commands)
 - [docs/cli-output.md](docs/cli-output.md) — shared CLI styling module (`src/cli/utils/ui.js`): OMEGA-style banner/dividers/sections/status symbols + the `Summary` block; used by `setup`, adoptable by other commands
 - [docs/environment-detection.md](docs/environment-detection.md) — `getEnvironment()` returns `'development' | 'testing' | 'production'` (mutually exclusive); gate side effects on the INTENTIONAL check (`isProduction()` for prod-only, `isDevelopment() || isTesting()` for local-or-test) — never `!isDevelopment()`. Plus the URL helper convention (always `Manager.getApiUrl()` — auto-resolves local in dev+test, never read `project.apiUrl`)
@@ -122,8 +143,10 @@ Deep references live in `docs/`. **Whenever you make a behavioral change, update
 
 ### Building Routes & Components
 
-- [docs/routes.md](docs/routes.md) — recipes for new API commands, routes, event handlers, cron jobs (with code templates)
-- [docs/schemas.md](docs/schemas.md) — schema definition format, defaults vs premium overrides
+- [docs/routes.md](docs/routes.md) — recipes for new API commands, routes (context-object handlers, CRUD method files, ownership checks, firebase.json rewrites + ordering, functions/index.js entry), event handlers, cron jobs
+- [docs/schemas.md](docs/schemas.md) — schema contract (context object → flat schema, in-function plan branching), field properties, ID generation + path extraction, required-vs-default footgun
+- [docs/firestore.md](docs/firestore.md) — path style, NO subcollections, batch reads (~500 cursor pagination), `metadata.{created,updated}` timestamps, response format + redaction
+- [docs/migration.md](docs/migration.md) — legacy-project migration: runtime config → top-level env vars, `Manager.config.*` → `process.env.*`, constructor routes / tiered schemas → current format
 - [docs/sanitization.md](docs/sanitization.md) — middleware trim-only default; opt-in HTML strip (`{ sanitize: true }`) with per-field opt-out (`sanitize: false`); manual `utilities.sanitize()` for HTML-insertion sites
 - [docs/auth-hooks.md](docs/auth-hooks.md) — consumer hooks for `before-create`/`before-signin`/`on-create`/`on-delete` (blocking + non-blocking examples)
 - [docs/common-operations.md](docs/common-operations.md) — inside-the-handler patterns: authenticate, read/write Firestore, error handling, send response, `bm_api` hook
@@ -146,6 +169,8 @@ Deep references live in `docs/`. **Whenever you make a behavioral change, update
 
 ### Testing & CLI
 
-- [docs/testing.md](docs/testing.md) — running, filtering, log files, test types (standalone/suite/group), context object, assertions, auth levels. **NEVER mock — test against the real emulator.** No `mockManager`/`mockAdmin`/fake `firestore`/stubbed `assistant`; every `run()` gets the real `Manager`/`assistant`/`firestore`/`http`/`accounts` — use them. Pure functions (zero I/O) are the only thing you call directly; anything touching Firestore or an external API runs for real. Real external APIs (OpenAI/PayPal/GitHub/SendGrid/Stripe) are gated behind `TEST_EXTENDED_MODE` in-source (not mocked), and anything an extended test creates externally must be cleaned up by the test. **Each test file `module.exports` a `{ description, type, tests }` object — NOT raw Mocha (`describe`/`it`/`beforeEach`); those globals are not injected and the file fails to load. Split tests one-file-per-concern under `test/<area>/`, never one giant `test/test.js`.** **All cleanup runs at the START of every run, never at the end** — the runner flushes the ENTIRE emulator Firestore before every run, so there's nothing to register; seed any needed fixtures in `test/_init.js`'s `setup()`, and never add a trailing cleanup step. Marketing providers (SendGrid/Beehiiv) don't need a special exception — `_test.*` emails are blocked at the validation layer so test signups never reach providers. The `_test.allow_*` carve-out exists only for the live-provider lifecycle test (`test/marketing/consent-lifecycle.js`), which manages its own teardown.
+- [docs/test-framework.md](docs/test-framework.md) — running, filtering, log files, test types (standalone/suite/group), context object, assertions, auth levels. **NEVER mock — test against the real emulator.** No `mockManager`/`mockAdmin`/fake `firestore`/stubbed `assistant`; every `run()` gets the real `Manager`/`assistant`/`firestore`/`http`/`accounts` — use them. Pure functions (zero I/O) are the only thing you call directly; anything touching Firestore or an external API runs for real. Real external APIs (OpenAI/PayPal/GitHub/SendGrid/Stripe) are gated behind `TEST_EXTENDED_MODE` in-source (not mocked) — opt in with `--extended` or `TEST_EXTENDED_MODE=true` (shared, unprefixed across BEM/BXM/UJM/EM; propagates to BOTH runner + emulator) — and anything an extended test creates externally must be cleaned up by the test. **Each test file `module.exports` a `{ description, type, tests }` object — NOT raw Mocha (`describe`/`it`/`beforeEach`); those globals are not injected and the file fails to load. Split tests one-file-per-concern under `test/<area>/`, never one giant `test/test.js`.** **All cleanup runs at the START of every run, never at the end** — the runner flushes the ENTIRE emulator Firestore before every run, so there's nothing to register; seed any needed fixtures in `test/_init.js`'s `setup()`, and never add a trailing cleanup step. Marketing providers (SendGrid/Beehiiv) don't need a special exception — `_test.*` emails are blocked at the validation layer so test signups never reach providers. The `_test.allow_*` carve-out exists only for the live-provider lifecycle test (`test/marketing/consent-lifecycle.js`), which manages its own teardown.
+- [docs/test-boot-layer.md](docs/test-boot-layer.md) — the `boot/` smoke layer: framework self-test from the repo via the bundled fixture project + `BEM_TEST_BOOT_PROJECT` (BEM's analog of BXM/UJM `*_TEST_BOOT_PROJECT`)
 - [docs/cli-firestore-auth.md](docs/cli-firestore-auth.md) — `npx mgr firestore:*` and `auth:*` commands, shared flags, examples
 - [docs/cli-logs.md](docs/cli-logs.md) — `npx mgr logs:read` / `logs:tail` with full flag reference and built-in Cloud Function names
+- [docs/logging.md](docs/logging.md) — `functions/*.log` file table (the `functions/` location exception), `production.log`

package/README.md

@@ -850,18 +850,19 @@ npx mgr test
 
 ### Extended Mode (real APIs)
 
-Set `TEST_EXTENDED_MODE=true` on the **test command** to opt into real external API calls (SendGrid, Beehiiv, Stripe webhook handlers, marketing libraries). The flag flows automatically to the running emulator via `<projectRoot>/.temp/test-mode.json` — no need to set it on the emulator too:
+Pass `--extended` (or set `TEST_EXTENDED_MODE=true`) on the **test command** to opt into real external API calls (SendGrid, Beehiiv, Stripe webhook handlers, marketing libraries). `--extended` is the CLI shorthand for the shared, unprefixed `TEST_EXTENDED_MODE` env var standardized across BEM/BXM/UJM/EM — the two forms are equivalent. The mode flows automatically to BOTH the test-runner subprocess and the running emulator (via `<projectRoot>/.temp/test-mode.json`) — no need to set it on the emulator too:
 
 ```bash
 # Terminal 1 — start once, no flag needed
 npx mgr emulator
 
 # Terminal 2 — toggle freely between runs
-TEST_EXTENDED_MODE=true npx mgr test ...   # extended mode
+npx mgr test --extended ...                 # extended mode (--extended sets TEST_EXTENDED_MODE)
+TEST_EXTENDED_MODE=true npx mgr test ...    # identical — the env-var form
 npx mgr test ...                            # normal mode (next run flips back)
 ```
 
-See [docs/testing.md](docs/testing.md#extended-mode-test_extended_mode) for the full mechanism.
+See [docs/test-framework.md](docs/test-framework.md#extended-mode-test_extended_mode) for the full mechanism.
 
 ### Filtering Tests
 
@@ -875,10 +876,10 @@ npx mgr test user/ admin/       # Multiple paths
 ### Log Files
 
 BEM CLI commands automatically save output to log files in the project's `functions/` directory (alongside firebase-tools' own `*-debug.log` files so everything is grep-able from one place):
-- **`functions/serve.log`** — Output from `npx mgr serve`
+- **`functions/dev.log`** — Output from `npx mgr serve` (BEM's local dev server)
 - **`functions/emulator.log`** — Full emulator + Cloud Functions output (`npx mgr emulator`)
 - **`functions/test.log`** — Test runner output (`npx mgr test`, when running against an existing emulator)
-- **`functions/logs.log`** — Cloud Function logs (`npx mgr logs:read` or `npx mgr logs:tail`)
+- **`functions/production.log`** — Production Cloud Function logs (`npx mgr logs:read` or `npx mgr logs:tail`)
 
 Logs are overwritten on each run and gitignored via `*.log`. Use them to debug failing tests or review function output. Transient internal artifacts (reset sentinels, watch trigger, `test-mode.json`) live separately in `<projectDir>/.temp/`.
 

package/docs/audit.md

@@ -0,0 +1,70 @@
+# Audit Workflow
+
+Full-project audit for BEM — runs against a CONSUMER backend or the FRAMEWORK repo itself (scope auto-detected). Invoked via the `omega:bem` skill (`/omega:bem audit`) or any "audit this backend/project" request.
+
+Every check has a stable ID, a severity, and a scope. Findings are reported as `ID @ file:line`, fixed one at a time, then re-verified. The tables below do NOT restate the rules — each check links to the doc that owns the rule and the fix.
+
+## Protocol
+
+1. **Detect scope** — read `package.json` (consumer: `functions/package.json`): `name` is `backend-manager` → **framework audit** (U + BEM + F checks); `backend-manager` in (dev)dependencies → **consumer audit** (U + BEM checks).
+2. **Run the catalog** — every check matching the scope. Search with Grep/Glob/Read over `functions/` (`routes/`, `schemas/`, `hooks/`, `index.js`), `test/`, and config files; ALWAYS exclude `node_modules/`, `dist/`, `_legacy/`, `_backup/`. Record each finding as `ID @ file:line` + a one-line description.
+3. **Persist the report** — write the findings list to `functions/.temp/audit/claude-audit.md` (BEM's `functions/`-local convention, like its logs) so a long fix loop survives session breaks. Summarize counts by severity in chat.
+4. **Fix loop** — TodoWrite per finding, highest severity first, ONE at a time: mark in-progress → root cause → fix → verify → complete. Ask before structural or destructive fixes (file deletions, schema reshapes, data migrations).
+5. **Re-verify** — re-run every check that produced findings until clean; finish with `npx mgr test` from `functions/` (must be green — it auto-starts its own emulator if needed).
+6. **Doc parity** — if fixes changed behavior, update README / CLAUDE.md / `docs/<topic>.md` / CHANGELOG in the same change set.
+
+Severity: **CRIT** security or broken functionality · **HIGH** hard-rule violation · **MED** convention drift · **LOW** optional improvement.
+Scope: **C** consumer · **F** framework repo · **B** both.
+
+## Universal checks (U-xx)
+
+Mirrored across all four OMEGA frameworks (UJM / BEM / BXM / EM) — same ID means the same check everywhere.
+
+| ID | Sev | Scope | Check |
+|----|-----|-------|-------|
+| U-01 | HIGH | B | Every feature has tests at EVERY surface it exposes — handler suites + `http.as(...)` route round-trips + rules suites; never mocked, real emulator only ([test-framework.md](test-framework.md)) |
+| U-02 | HIGH | B | Test hygiene — side-effect tests use dedicated `journey-*` accounts; real-external-API tests gated behind `TEST_EXTENDED_MODE` in-source (not mocked); files export `{ description, type, tests }` (no raw Mocha); no trailing cleanup steps ([test-framework.md](test-framework.md)) |
+| U-03 | CRIT | B | Sanitization — middleware is trim-only by default; HTML strip via opt-in `{ sanitize: true }`; every HTML-insertion site calls `utilities.sanitize()` ([sanitization.md](sanitization.md)) |
+| U-04 | HIGH | B | Firebase ownership — server code uses `firebase-admin` via Manager (correct here); NO client `firebase` SDK in functions code; consuming frontends go through web-manager ([CLAUDE.md](../CLAUDE.md) §Dependency Resolution) |
+| U-05 | HIGH | C | No BEM transitive deps installed directly in `functions/package.json` — use `Manager.require(name)` ([CLAUDE.md](../CLAUDE.md) §Dependency Resolution) |
+| U-06 | HIGH | B | Env behavior gated on the INTENTIONAL check — `isProduction()` or `isDevelopment() \|\| isTesting()`, never `!isDevelopment()`; always `Manager.getApiUrl()`, never the cached `Manager.project.apiUrl` ([environment-detection.md](environment-detection.md)) |
+| U-07 | HIGH | B | Config canon — `backend-manager-config.json` matches the documented shape; canonical cross-framework blocks (`brand`, payment products, …) not reinvented ([architecture.md](architecture.md), [payment-system.md](payment-system.md)) |
+| U-08 | CRIT | B | No private credentials committed — `service-account.json`, `.env` secrets, API keys (Stripe `sk_`, SendGrid `SG.`, …); `.gitignore` covers them. (The Firebase WEB `apiKey` is public by design — do NOT flag it.) |
+| U-09 | HIGH | B | Source discipline — no live code referencing `_legacy/` / `_backup/`; framework edits in `src/` (never `dist/`) ([common-mistakes.md](common-mistakes.md)) |
+| U-10 | MED | B | Doc parity — README / CLAUDE.md / `docs/` / CHANGELOG match shipped behavior; CLAUDE.md < 250 lines; the docs index lists every `docs/*.md`; no stale names for renamed commands/patterns |
+| U-11 | MED | B | SSOT/DRY — no duplicated constants/config/logic; one authoritative home per value, imported everywhere else |
+| U-12 | MED | B | JS conventions — file structure, JSDoc, short-circuit returns, leading logical operators, `fs-jetpack`, one `module.exports` per file ([code-patterns.md](code-patterns.md) + global `js:patterns` skill) |
+| U-13 | MED | B | Dead code & stale patterns — no orphaned files nothing imports; no leftovers of migrated-away formats (constructor routes, tiered schemas, `Manager.config.*` reads — [migration.md](migration.md)); inventory TODO/FIXME (report only) |
+| U-14 | LOW | B | Dependency health — review `npm outdated` / `npm audit` (in `functions/`); apply fixes via the `general:update-packages` workflow (includes supply-chain checks) |
+
+## BEM-specific checks
+
+| ID | Sev | Scope | Check |
+|----|-----|-------|-------|
+| BEM-01 | HIGH | B | Every custom route has a name-matched schema; handlers are context-object exports (`async ({ Manager, assistant, … }) => {}`) — no legacy constructor routes ([routes.md](routes.md), [schemas.md](schemas.md)) |
+| BEM-02 | HIGH | B | Schema field rules — never `required: true` + `default` together (required is checked BEFORE defaults; use `min: 1` for path-extracted IDs); flat schema with in-function plan branching, no tier arrays ([schemas.md](schemas.md)) |
+| BEM-03 | HIGH | B | Route handlers — ownership checks on PUT/DELETE; plural-noun route names; `assistant.respond()` only (never `res.send()`) ([routes.md](routes.md), [common-operations.md](common-operations.md)) |
+| BEM-04 | HIGH | C | Wiring — every route exported in `functions/index.js`; `firebase.json` rewrites use bracket syntax, ordered most-specific-first ([routes.md](routes.md)) |
+| BEM-05 | HIGH | B | Firestore canon — NO subcollections; path-string `.doc('users/abc')`; batched collection reads (~500, cursor pagination); timestamps under `metadata.{created,updated}`; mirror-the-doc responses; delete-don't-redact ([firestore.md](firestore.md)) |
+| BEM-06 | HIGH | B | Usage — never read/write `{doc}.usage.*` manually, always the `usage` helper; expensive/abusable routes carry usage validation or rate limiting ([usage-rate-limiting.md](usage-rate-limiting.md)) |
+| BEM-07 | MED | B | Composite indexes — every compound query (`where` + `orderBy`, multiple `where`s) is registered in the required-indexes SSOT ([CLAUDE.md](../CLAUDE.md) §File Conventions) |
+| BEM-08 | HIGH | B | Auth gates — routes resolve the caller via `assistant`/`user` before acting; admin-only routes verify admin status ([common-operations.md](common-operations.md), [routes.md](routes.md)) |
+| BEM-09 | HIGH | B | Rules coverage — `firestore.rules` changes ship a rules suite (`rules.asAccount` / `expectSuccess` / `expectFailure`) ([test-framework.md](test-framework.md)) |
+
+## Framework-repo checks (F-xx)
+
+Only when auditing the BEM repo itself. Mirrored across the four frameworks.
+
+| ID | Sev | Check |
+|----|-----|-------|
+| F-01 | MED | Sister parity — mirrored sections (config shapes, test contract, CLAUDE.md skeleton, shared env/test conventions) in sync with UJM / BXM / EM; deviations are deliberate and documented |
+| F-02 | HIGH | Consumer-shipped defaults in sync — what `npx mgr setup` scaffolds matches current conventions and docs |
+| F-03 | MED | Docs completeness — every `docs/*.md` indexed in CLAUDE.md; every subsystem has a doc; no "(planned)" links for things that have shipped |
+| F-04 | HIGH | `npx mgr test mgr:` green before treating the audit as complete |
+
+## See also
+
+- [schemas.md](schemas.md) — the required-vs-default footgun behind BEM-02
+- [firestore.md](firestore.md) — the data canon behind BEM-05
+- [migration.md](migration.md) — the legacy formats U-13 hunts for
+- [test-framework.md](test-framework.md) — the surfaces behind U-01 / U-02 / BEM-09

package/docs/build-system.md

@@ -0,0 +1,29 @@
+# Build System
+
+BEM is the deliberate outlier among the four OMEGA frameworks: **consumer projects have no build pipeline**. There is no gulp, no webpack, no bundling — `functions/` runs as-is in the Firebase emulator locally and deploys as-is to Cloud Functions.
+
+## Pipeline overview
+
+| Stage | Command | What happens |
+|---|---|---|
+| Local dev | `npx mgr emulator` / `npx mgr serve` | Functions run directly from `functions/` source in the Firebase emulator |
+| Watch | `npx mgr watch` | Auto-reload functions on file change |
+| Ship | `npx mgr deploy` | `functions/` deploys as-is to Firebase Cloud Functions |
+
+There are no build modes — environment behavior is governed by emulator vs production, not a build flag. See [environment-detection.md](environment-detection.md).
+
+## prepare-package (framework-side)
+
+The BEM library itself has one build step: `npm run prepare` copies `src/` → `dist/` via prepare-package (`npm run prepare:watch` for watch mode). Consumers always require from `dist/`. This mirrors the framework-side prepare step in EM/BXM/UJM.
+
+At publish time, a `prepublishOnly` script first removes the self-test fixture's runtime `node_modules` (`src/test/fixtures/firebase-project/functions/node_modules`) — its `backend-manager` symlink points back at the repo root, which would send prepare-package's publish-time cleanup walk into an infinite cycle. The symlinks are throwaway; the next `npx mgr test` self-test regenerates them (see [test-boot-layer.md](test-boot-layer.md)).
+
+## Log files
+
+CLI commands tee output to `functions/*.log` (`dev.log`, `emulator.log`, `test.log`, `production.log`). Full reference: [logging.md](logging.md).
+
+## See also
+
+- [architecture.md](architecture.md) — Manager class, dual-mode support (`firebase` / `custom`)
+- [directory-structure.md](directory-structure.md) — BEM library + consumer project layout
+- [test-framework.md](test-framework.md) — the emulator-based test harness

package/docs/cli-logs.md

@@ -19,9 +19,9 @@ npx mgr logs:tail                                     # Stream live logs
 npx mgr logs:tail --fn bm_paymentsWebhookOnWrite      # Stream filtered live logs
 ```
 
-Both commands save output to `functions/logs.log` (overwritten on each run). `logs:read` saves raw JSON; `logs:tail` streams text.
+Both commands save output to `functions/production.log` (overwritten on each run). `logs:read` saves raw JSON; `logs:tail` streams text.
 
-**Cloud Logs vs Local Logs:** These commands query **production** Google Cloud Logging. For **local/dev** logs, read `functions/serve.log` (from `npx mgr serve`) or `functions/emulator.log` (from `npx mgr test`) directly — they are plain text files, not gcloud.
+**Cloud Logs vs Local Logs:** These commands query **production** Google Cloud Logging. For **local/dev** logs, read `functions/dev.log` (from `npx mgr serve`) or `functions/emulator.log` (from `npx mgr test`) directly — they are plain text files, not gcloud.
 
 ## Flags
 

package/docs/common-mistakes.md

@@ -9,4 +9,4 @@
 7. **Use short-circuit returns** — Return early from error conditions
 8. **Increment usage before update** — Call `usage.increment()` then `usage.update()`
 9. **Add Firestore composite indexes for new compound queries** — Any new Firestore query using multiple `.where()` clauses or `.where()` + `.orderBy()` requires a composite index. Add it to `src/cli/commands/setup-tests/helpers/required-indexes.js` (the SSOT). Consumer projects pick these up via `npx mgr setup`, which syncs them into `firestore.indexes.json`. Without the index, the query will crash with `FAILED_PRECONDITION` in production.
-10. **Don't put test data cleanup at the END of a test** — End-of-test cleanup doesn't fire when the previous run was killed mid-execution, so the next run inherits stale state. ALL test-data cleanup belongs in the runner's pre-test phase ([../src/test/test-accounts.js](../src/test/test-accounts.js) `deleteTestUsers()` + [../src/test/runner.js](../src/test/runner.js) `setupAccounts()`), which **flushes the entire emulator Firestore before every run** — so there's nothing to register when you add a test that writes data. Seed any required fixtures in `test/_init.js`'s `setup()` (runs after the flush). The only acceptable trailing cleanup is within-run state isolation (one test removes a doc so the NEXT test in the same run sees a clean slate) — that's not preparing the next run, it's intra-run housekeeping. See [testing.md](testing.md) "Test Data Cleanup".
+10. **Don't put test data cleanup at the END of a test** — End-of-test cleanup doesn't fire when the previous run was killed mid-execution, so the next run inherits stale state. ALL test-data cleanup belongs in the runner's pre-test phase ([../src/test/test-accounts.js](../src/test/test-accounts.js) `deleteTestUsers()` + [../src/test/runner.js](../src/test/runner.js) `setupAccounts()`), which **flushes the entire emulator Firestore before every run** — so there's nothing to register when you add a test that writes data. Seed any required fixtures in `test/_init.js`'s `setup()` (runs after the flush). The only acceptable trailing cleanup is within-run state isolation (one test removes a doc so the NEXT test in the same run sees a clean slate) — that's not preparing the next run, it's intra-run housekeeping. See [test-framework.md](test-framework.md) "Test Data Cleanup".

package/docs/consent.md

@@ -138,7 +138,9 @@ Each processor's `handleEvent` does the same shape of work:
 
 | Provider | Event types treated as revoke |
 |---|---|
-| SendGrid | `unsubscribe`, `group_unsubscribe`, `spamreport`, `bounce`, `dropped` |
+| SendGrid | `unsubscribe`, `group_unsubscribe`, `spamreport`, `bounce`\*, `dropped`\* |
+
+\* `bounce` and `dropped` only revoke consent when `bounce_classification` is `'Invalid Address'` (hard bounce). Technical bounces (DMARC, TLS, DNS, reputation) are sender-side issues — the recipient's email is still valid, so consent is preserved.
 | Beehiiv | `subscription.unsubscribed`, `subscription.deleted`, `subscription.paused` |
 
 **Beehiiv publication filter.** Each Beehiiv event includes a `publication_id`. The processor compares this against `beehiivProvider.getPublicationId()`, which reads `Manager.config.marketing.newsletter.publicationId` (populated at brand-onboarding time by OMEGA's `beehiiv/ensure/publication.js`). Mismatch → silent skip. This is how shared-publication events (e.g. devbeans shared by 6 brands) get routed correctly — each brand processes only events matching its own publication. Brands without `publicationId` in config silently skip all Beehiiv webhook events. The same convention applies to SendGrid: `marketing.campaigns.listId` is populated by OMEGA's `sendgrid/ensure/list.js`.
@@ -313,7 +315,7 @@ Most BEM tests are self-contained against the local emulator. The marketing-cons
 
 The validation pipeline (`src/manager/libraries/email/validation.js`) blocks all `_test.*` emails from reaching providers via the `/^_test\.(?!allow_)/` pattern in `blocked-local-patterns.js`. The two `_test.allow_*` sentinels (`_test.allow_consent-granted` and `_test.allow_consent-declined`) used by the lifecycle test bypass that gate intentionally, and the test cleans up after itself (phase-3 removes the granted contact via `Manager.Email().remove()`).
 
-The "all cleanup runs at start, never at the end" rule documented in [docs/testing.md](testing.md) applies to all test data, including third-party providers.
+The "all cleanup runs at start, never at the end" rule documented in [docs/test-framework.md](test-framework.md) applies to all test data, including third-party providers.
 
 ## Frontend pieces (cross-references)
 

package/docs/email-system.md

@@ -56,6 +56,27 @@ After `build()`, `send()` delivers via SendGrid, handles scheduled sends (>71h
 
 `_sendCampaignSendGrid()` follows the same prepare → render → deliver pattern. Content comes from `data.content.message` (markdown) — same location as transactional callers. Key difference: audience targeting uses brand-scoped dynamic segments (see [marketing-campaigns.md](marketing-campaigns.md)).
 
+### Email Validation Pipeline (`validation.js`)
+
+All marketing contact operations (`add`, `sync`) pass through `validate()` before reaching providers. Checks run in order; the first failure short-circuits.
+
+| # | Check | What it catches | Cost | Default |
+|---|---|---|---|---|
+| 1 | `format` | Regex: must have `@`, domain, no spaces | Free | Yes |
+| 2 | `disposable` | ~7k known disposable domains (vendor list + custom additions) | Free | Yes |
+| 3 | `corporate` | Social/corporate domains (instagram.com, facebook.com, etc.) | Free | Yes |
+| 4 | `localPart` | Junk local parts (test, noreply, all-numeric, `_test.*`) | Free | Yes |
+| 5 | `typo` | Common domain misspellings via prefix match (`gamil.`, `gmai.`, `aol.con`, `gmail.cok`, etc.) | Free | Yes |
+| 6 | `dns` | No MX record, null MX (RFC 7505), loopback MX, domain not found | Free | Opt-in |
+| 7 | `mailbox` | SMTP mailbox verification via NeverBounce or ZeroBounce | Paid | Opt-in |
+
+- **`DEFAULT_CHECKS`** = checks 1–5 (all free, run on every `mailer.add()`/`mailer.sync()` call)
+- **`ALL_CHECKS`** = checks 1–7 (used at signup to include paid mailbox verification)
+- The `dns` check is opt-in (not in DEFAULT_CHECKS) because it's async/slower — include it for bulk validation
+- The `typo` check uses prefix matching (`"gamil."` catches `gamil.com`, `gamil.con`, `gamil.co`) — see `data/typo-domains.js`
+- Custom disposable domains go in `data/custom-disposable-domains.json` (not the vendor list)
+- Run `node src/manager/libraries/email/validation.test.js` to verify all checks
+
 ## Data Contract
 
 The template receives one `data` object with a clear separation of concerns:
@@ -263,7 +284,7 @@ All email tests live under `test/email/`, mirroring the source at `src/manager/l
 |---|---|---|
 | `templates.js` | MJML rendering for all 4 email templates (11 tests) | No |
 | `transactional.js` | Transactional email building (assertions on output shape) | No |
-| `validation.js` | Email format/disposable/corporate/local-part checks (80+ tests) | No |
+| `validation.js` | Email format/disposable/corporate/local-part/typo/dns checks (60+ tests) | No |
 | `transactional-send.js` | Single transactional email send via SendGrid | Yes |
 | `campaign-send.js` | Marketing campaign send with title + CTA + discount code | Yes |
 | `feedback-and-plain-send.js` | Feedback + plain template visual test sends | Yes |
@@ -272,7 +293,7 @@ All email tests live under `test/email/`, mirroring the source at `src/manager/l
 | `marketing-lifecycle.js` | Contact lifecycle (add/sync/remove) | Yes |
 | `consent-lifecycle.js` | Consent webhook round-trip | Yes |
 
-Extended tests (`TEST_EXTENDED_MODE`) send real emails to `_test-*@{domain}` addresses. See [testing.md](testing.md) for the full test framework reference.
+Extended tests (`TEST_EXTENDED_MODE`) send real emails to `_test-*@{domain}` addresses. See [test-framework.md](test-framework.md) for the full test framework reference.
 
 ### Test recipient convention
 
@@ -295,5 +316,10 @@ All extended email tests send to `_test-<purpose>@{domain}` addresses (e.g. `_te
 | UTM link tagging | `src/manager/libraries/email/utm.js` |
 | Constants (senders, groups, fields, segments) | `src/manager/libraries/email/constants.js` |
 | Email validation | `src/manager/libraries/email/validation.js` |
+| Typo domain prefixes | `src/manager/libraries/email/data/typo-domains.js` |
+| Custom disposable domains | `src/manager/libraries/email/data/custom-disposable-domains.json` |
+| NeverBounce provider | `src/manager/libraries/email/validation-provider-neverbounce.js` |
+| ZeroBounce provider | `src/manager/libraries/email/validation-provider-zerobounce.js` |
+| Validation test | `src/manager/libraries/email/validation.test.js` |
 | Seed campaigns | `src/cli/commands/setup-tests/helpers/seed-campaigns.js` |
 | Transition email dispatcher | `src/manager/events/firestore/payments-webhooks/transitions/send-email.js` |

package/docs/environment-detection.md

@@ -88,4 +88,4 @@ If you need a new environment-derived helper, add it next to the others on the M
 
 ## See also
 
-- [testing.md](testing.md) — `BEM_TESTING` is set automatically by the test runner; `TEST_EXTENDED_MODE` gates real external APIs.
+- [test-framework.md](test-framework.md) — `BEM_TESTING` is set automatically by the test runner; `TEST_EXTENDED_MODE` gates real external APIs.

package/docs/firestore.md

@@ -0,0 +1,134 @@
+# Firestore Conventions
+
+## Path Style
+
+Use `.doc('collectionId/documentId')` instead of `.collection('collectionId').doc('documentId')`.
+
+## No Subcollections
+
+**NEVER use subcollections.** All collections MUST be top-level. Use an `owner` field to associate documents with a user instead of nesting under `/users/{uid}/`.
+
+```javascript
+// CORRECT — top-level collection with owner field
+await admin.firestore().doc(`items/${id}`).set({ owner: user.auth.uid, ... });
+
+// WRONG — subcollection under users
+await admin.firestore().doc(`users/${uid}/items/${id}`).set({ ... });
+```
+
+## Batch Collection Reads
+
+**NEVER** dump an entire collection with a bare `.get()`. Always read and process in batches of ~500 using `.limit()` with `.startAfter()` cursor pagination. This applies to routes, cron jobs, standalone scripts — anywhere we query Firestore collections that could have many documents.
+
+```javascript
+const BATCH_SIZE = 500;
+let lastDoc = null;
+let processed = 0;
+
+while (true) {
+  let query = db.collection('items').limit(BATCH_SIZE);
+
+  if (lastDoc) {
+    query = query.startAfter(lastDoc);
+  }
+
+  const snapshot = await query.get();
+
+  if (snapshot.empty) {
+    break;
+  }
+
+  for (const doc of snapshot.docs) {
+    processed++;
+    // process doc...
+  }
+
+  lastDoc = snapshot.docs[snapshot.docs.length - 1];
+}
+```
+
+If you need a `.where()` filter, apply it before `.limit()`:
+
+```javascript
+let query = db.collection('items')
+  .where('status', '==', 'active')
+  .limit(BATCH_SIZE);
+```
+
+## Document Metadata
+
+All Firestore documents must nest `created` and `updated` timestamps under a `metadata` parent field — never as top-level fields:
+
+```javascript
+// ✅ CORRECT — timestamps under metadata
+const itemData = {
+  id: settings.id,
+  owner: user.auth.uid,
+  metadata: {
+    created: {
+      timestamp: assistant.meta.startTime.timestamp,
+      timestampUNIX: assistant.meta.startTime.timestampUNIX,
+    },
+    updated: {
+      timestamp: assistant.meta.startTime.timestamp,
+      timestampUNIX: assistant.meta.startTime.timestampUNIX,
+    },
+  },
+};
+
+// On update — preserve created, refresh updated
+const updated = _.merge({}, existing, settings, {
+  metadata: {
+    created: existing.metadata.created,
+    updated: {
+      timestamp: assistant.meta.startTime.timestamp,
+      timestampUNIX: assistant.meta.startTime.timestampUNIX,
+    },
+  },
+});
+
+// ❌ WRONG — top-level timestamps
+const itemData = {
+  created: { ... },
+  updated: { ... },
+};
+```
+
+In schemas, use `assistant.Manager.Settings().constant('timestampFULL')`:
+
+```javascript
+metadata: {
+  created: assistant.Manager.Settings().constant('timestampFULL', { date: undefined }),
+  updated: assistant.Manager.Settings().constant('timestampFULL', { date: undefined }),
+},
+```
+
+## Response Format
+
+- **Return data in the same structure as Firestore.** Do NOT reshape, rename, or restructure fields (e.g., don't convert `metadata.created.timestamp` to `createdAt`).
+- For single document responses: `{ id: docId, ...docData }`
+- For collection responses: `[{ id: docId, ...docData }, ...]`
+- **Redaction:** Delete sensitive fields entirely from the response object. Do NOT replace them with `'[REDACTED]'`.
+
+```javascript
+// ✅ CORRECT — mirror the Firestore doc
+const doc = await admin.firestore().doc(`items/${id}`).get();
+return assistant.respond({ item: { id: doc.id, ...doc.data() } });
+
+// ❌ WRONG — reshaping into a different structure
+return assistant.respond({ item: { id, name: doc.data().name, createdAt: doc.data().metadata.created.timestamp } });
+
+// ✅ CORRECT — redact by deleting
+const data = doc.data();
+delete data.api?.privateKey;
+return assistant.respond({ item: { id: doc.id, ...data } });
+
+// ❌ WRONG — redact by replacing
+data.api.privateKey = '[REDACTED]';
+```
+
+## See also
+
+- [routes.md](routes.md) — the route handlers doing these reads/writes
+- [usage-rate-limiting.md](usage-rate-limiting.md) — the `usage` helper owns `{doc}.usage.*` fields (never write them manually)
+- [cli-firestore-auth.md](cli-firestore-auth.md) — reading/writing Firestore from the terminal