backend-manager 5.1.2 → 5.2.0

package/docs/testing.md

@@ -11,6 +11,87 @@ npx mgr test      # Terminal 2 - runs tests
 npx mgr test
 ```
 
+## Test Data Cleanup — at the START of every run
+
+**Hard rule: all LOCAL cleanup happens BEFORE the suite runs, never after.** If a previous run was killed mid-execution (Ctrl-C, OOM, emulator crash), end-of-run cleanup would never fire and the next run would inherit polluted state — broken trial-eligibility checks, leftover dispute alerts, stale webhook docs, polluted marketing-provider lists. Pre-test cleanup makes every run idempotent regardless of how the last one died.
+
+What the runner wipes pre-test (in [src/test/test-accounts.js](../src/test/test-accounts.js) `deleteTestUsers()` and [src/test/runner.js](../src/test/runner.js) `setupAccounts()`):
+
+1. **`meta/stats`** doc ensured (required for on-create batch writes).
+2. **`users/_test-*`** Firebase Auth users + Firestore docs (delete).
+3. **Marketing providers** (SendGrid + Beehiiv) — leftover test contacts removed. Runs only under `TEST_EXTENDED_MODE`. Runs **before** test users are recreated so a killed run can't leave a contact pinned to an about-to-be-recreated uid.
+4. **Mixed Firestore collections** — `payments-orders`, `payments-webhooks`, `payments-intents`, `payments-disputes`, `marketing-webhooks`. Two-pass cleanup per collection:
+   - Pass 1 — owner-keyed: `where('owner', 'in', [...testUids])` (batched at 30 uids per `in` query).
+   - Pass 2 — id-keyed: any doc whose ID starts with `_test-` (catches ownerless test docs like dispute alerts and raw test webhooks).
+5. **Test-only Firestore collections** — `_test`, `_test_query` — wiped in full.
+6. **Realtime Database** — the `_test` namespace removed in full (`admin.database().ref('_test').remove()`).
+
+### The one exception: third-party provider cleanup runs at start AND end
+
+The `cleanupMarketingProviders` call also fires **after** the suite completes (extended mode only). Reason: pre-run cleanup catches what a crashed previous run left behind, but during a normal-completion run we'd still leak real SendGrid/Beehiiv contacts until the NEXT run cleaned them up. By running cleanup post-suite too, each extended-mode run leaves the provider lists in the same state it found them.
+
+This is **specifically** for third-party state that lives outside the local emulator. **Do not use trailing cleanup for Firestore or Auth data** — those follow the start-only rule because we control the local state and pre-run cleanup is enough.
+
+### When adding a new test that writes data
+
+| If your test writes to... | Then... |
+|---|---|
+| A new Firestore collection (mixed test + prod data) | Add the collection name to `testDataCollections` in `src/test/test-accounts.js`. |
+| A new Firestore collection (test-only data) | Add it to `testOnlyCollections` in `src/test/test-accounts.js`. |
+| Realtime Database | Use a path under `_test/...` — already wiped pre-test. |
+| Anywhere else | Use the `_test-` doc-id prefix so id-keyed pass-2 catches it. |
+
+### Within-run state isolation is different
+
+Per-test cleanup is still appropriate when a test sets up DB state that would pollute a **later test in the same run** — e.g. trial-eligibility's `try/finally` that removes the fake `payments-orders/_test-trial-eligibility-*` doc so the next sibling test sees a clean slate. Those stay in the test. They are *intra-run* state management, not *next-run* cleanup, and the distinction matters.
+
+The rule: **never put cleanup at the END of a test file or suite for the purpose of preparing the next run** — for LOCAL state. If a test cleans up local Firestore/Auth data only to "leave no trace," that cleanup belongs in the runner's pre-test phase instead. Add the collection/namespace to the runner's wipe list and remove the trailing cleanup step. The third-party provider exception (see above) lives in the runner's post-suite hook, not in individual tests.
+
+## Extended Mode (`TEST_EXTENDED_MODE`)
+
+Several routes/handlers skip external API calls (SendGrid, Beehiiv, Stripe webhooks, dispute handlers, marketing libraries) when `process.env.TEST_EXTENDED_MODE` is unset, so unit tests don't fire real emails or webhook side effects. Set the flag to opt **in** to those side effects for a full end-to-end run.
+
+**Live sync — no env coordination across terminals.** The flag flows automatically from the test command to the running emulator via a small shared state file at `<projectRoot>/.temp/test-mode.json`. The test command writes the file pre-flight; the emulator's function workers watch it via `fs.watch` and mutate their own `process.env.TEST_EXTENDED_MODE` in place. Effect: you only need to set the flag on **the test command**. The emulator follows.
+
+```bash
+# Terminal 1 — start once, leave running. NO flag needed.
+npx mgr emulator
+
+# Terminal 2 — toggle freely between runs:
+TEST_EXTENDED_MODE=true npx mgr test ...   # runs in extended mode
+npx mgr test ...                            # runs in normal mode
+TEST_EXTENDED_MODE=true npx mgr test ...   # back to extended
+```
+
+The emulator log shows each flip, e.g.:
+
+```
+[test-mode] resolved TEST_EXTENDED_MODE=false (file present)   ← worker boot
+[test-mode] flip TEST_EXTENDED_MODE: (unset) → true            ← test --extended fired
+[test-mode] flip TEST_EXTENDED_MODE: true → (unset)            ← test (no flag) fired
+```
+
+The test command also confirms the mode in its own output (`Test mode: EXTENDED (real APIs)` pre-flight, `Mode: EXTENDED (real APIs)` after the health check). The runner's old "TEST_EXTENDED_MODE mismatch" warning is gone — mismatch is impossible by construction.
+
+**Allowlist.** Only env vars listed in `SYNCED_ENV_KEYS` (`src/test/utils/test-mode-file.js`) flow through. Today: `TEST_EXTENDED_MODE`. Add a key there to make a new env var live-syncable across terminals. The allowlist exists to prevent accidentally syncing process-specific vars (`FIRESTORE_EMULATOR_HOST`) or sensitive ones (API keys).
+
+**Preferred flow: set the flag on the test command.** Every `npx mgr test` invocation overwrites the shared state file with whatever flags it was called with, and the emulator follows live. This is the recommended pattern — start the emulator once with no flag, leave it running, control the mode from your test invocations.
+
+```bash
+# Recommended
+npx mgr emulator                                # boots in normal mode
+TEST_EXTENDED_MODE=true npx mgr test ...        # flips emulator to extended
+npx mgr test ...                                 # flips emulator back to normal
+```
+
+**Also supported: set the flag on the emulator command.** This still works as a boot default — the emulator command writes the file with whatever it was started with, so the very first test run (before any `npx mgr test` overrides it) sees that mode. Useful if you want to inspect the emulator in a particular mode before firing any tests, or if you script the emulator boot from CI. Just know that the next test command overrides whatever you set here.
+
+```bash
+# Also works (boot default — overridden by next test command)
+TEST_EXTENDED_MODE=true npx mgr emulator        # boots in extended mode
+npx mgr test ...                                 # ← this still flips it back to normal
+```
+
 ## Log Files
 
 BEM CLI commands automatically save all output to log files in `functions/` while still streaming to the console:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "backend-manager",
-  "version": "5.1.2",
+  "version": "5.2.0",
   "description": "Quick tools for developing Firebase functions",
   "main": "src/manager/index.js",
   "bin": {

package/src/cli/commands/emulator.js

@@ -8,16 +8,33 @@ const powertools = require('node-powertools');
 const WatchCommand = require('./watch');
 const { DEFAULT_EMULATOR_PORTS } = require('./setup-tests/emulator-config');
 const { EXTENDED_MODE_WARNING } = require('../../test/utils/extended-mode-warning');
+const { writeTestMode, captureSyncedEnv } = require('../../test/utils/test-mode-file');
 
 class EmulatorCommand extends BaseCommand {
   async execute() {
     this.log(chalk.cyan('\n  Starting Firebase emulator (keep-alive mode)...\n'));
     this.log(chalk.gray('  Emulator will stay running until you press Ctrl+C\n'));
 
-    // Warn if TEST_EXTENDED_MODE is enabled
+    // Boot-time: seed the shared state file with whatever this emulator was
+    // started with. Two flows are supported:
+    //   - Recommended: start emulator without the flag, set TEST_EXTENDED_MODE
+    //     on `npx mgr test` instead. The test command writes the file; the
+    //     emulator's function workers watch it and flip live.
+    //   - Also supported: start emulator with TEST_EXTENDED_MODE=true. We
+    //     write the file here as a boot default. Useful for inspecting the
+    //     emulator before any tests fire. Note: the next `npx mgr test`
+    //     overwrites the file regardless of how the emulator booted.
+    {
+      const projectDir = this.main.firebaseProjectPath;
+      const envSubset = captureSyncedEnv(process.env);
+      writeTestMode(projectDir, envSubset);
+    }
+
+    // Show the standard warning if the emulator boots in extended mode.
     if (process.env.TEST_EXTENDED_MODE) {
       this.log(chalk.yellow.bold(`\n  ${EXTENDED_MODE_WARNING[0]}`));
       EXTENDED_MODE_WARNING.slice(1).forEach((line) => this.log(chalk.yellow(`  ${line}`)));
+      this.log(chalk.gray(`  (Tip: you can also flip mode per-run by setting TEST_EXTENDED_MODE on \`npx mgr test\`.)`));
       this.log('');
     }
 
@@ -65,15 +82,47 @@ class EmulatorCommand extends BaseCommand {
       : 'BEM_TESTING=true';
     const emulatorCommand = `${envPrefix} firebase emulators:exec --only functions,firestore,auth,database,hosting,pubsub --ui "${command}"`;
 
-    // Set up log file in the project directory
+    // Set up log file in the project directory.
+    // We use a mutable `currentStream` so the test command can request a fresh log
+    // by touching emulator.log.reset — the watcher below detects it, closes the
+    // current stream, reopens with flags: 'w' (truncating cleanly from our process'
+    // perspective, no sparse-file issue), and deletes the sentinel.
     const logPath = path.join(projectDir, 'functions', 'emulator.log');
-    const logStream = fs.createWriteStream(logPath, { flags: 'w' });
+    const resetSentinelPath = `${logPath}.reset`;
     const stripAnsi = (str) => str.replace(/\x1B\[[0-9;]*[a-zA-Z]/g, '');
 
+    let currentStream = fs.createWriteStream(logPath, { flags: 'w' });
+
+    function writeToLog(data) {
+      if (currentStream && !currentStream.destroyed) {
+        currentStream.write(stripAnsi(data.toString()));
+      }
+    }
+
+    // Clean up any stale sentinel from a prior crashed emulator run
+    try { fs.unlinkSync(resetSentinelPath); } catch (e) { /* not present, ok */ }
+
+    // Watch for the test command's request to roll the log.
+    // Poll every 500ms — cheap, no fs.watch quirks across platforms.
+    const resetWatcher = setInterval(() => {
+      if (!fs.existsSync(resetSentinelPath)) {
+        return;
+      }
+
+      try {
+        const oldStream = currentStream;
+        currentStream = fs.createWriteStream(logPath, { flags: 'w' });
+        oldStream.end();
+        fs.unlinkSync(resetSentinelPath);
+      } catch (e) {
+        // Best-effort. If reset fails the test still runs, the log just won't be fresh.
+      }
+    }, 500);
+
     // Write pre-emulator info to log file
     if (process.env.TEST_EXTENDED_MODE) {
-      EXTENDED_MODE_WARNING.forEach((line) => logStream.write(`${line}\n`));
-      logStream.write('\n');
+      EXTENDED_MODE_WARNING.forEach((line) => writeToLog(`${line}\n`));
+      writeToLog('\n');
     }
 
     this.log(chalk.gray(`  Logs saving to: ${logPath}\n`));
@@ -89,18 +138,22 @@ class EmulatorCommand extends BaseCommand {
       // Tee stdout to both console and log file (strip ANSI codes for clean log)
       child.stdout.on('data', (data) => {
         process.stdout.write(data);
-        logStream.write(stripAnsi(data.toString()));
+        writeToLog(data);
       });
 
       // Tee stderr to both console and log file (strip ANSI codes for clean log)
       child.stderr.on('data', (data) => {
         process.stderr.write(data);
-        logStream.write(stripAnsi(data.toString()));
+        writeToLog(data);
       });
 
-      // Clean up log stream when child exits
+      // Clean up log stream + watcher when child exits
       child.on('close', () => {
-        logStream.end();
+        clearInterval(resetWatcher);
+        if (currentStream && !currentStream.destroyed) {
+          currentStream.end();
+        }
+        try { fs.unlinkSync(resetSentinelPath); } catch (e) { /* ok */ }
       });
     });
   }

package/src/cli/commands/serve.js

@@ -25,11 +25,63 @@ class ServeCommand extends BaseCommand {
     // Start Stripe webhook forwarding in background
     this.startStripeWebhookForwarding();
 
-    // Set up log file in the project directory
+    // Set up log file in the project directory.
+    // Mirrors the emulator.js pattern: the file is truncated on boot and on every
+    // hot reload. Two reset signals are honored:
+    //   1. Sentinel file (serve.log.reset) — used by the BEM watcher when source
+    //      changes in backend-manager itself trigger a reload.
+    //   2. Reload marker on stdout (`Using node@22 from host.`) — catches reloads
+    //      triggered by firebase serve's own internal watcher (any change inside
+    //      the consumer's functions/ directory). MUST be the line firebase-tools
+    //      prints at the START of each reload cycle, not somewhere in the middle.
+    //      If we rolled mid-cycle (e.g. on "Loaded functions definitions"), the
+    //      tail of the reload sequence still gets captured into the fresh log;
+    //      but firebase serve sometimes only emits the trailing function-initialized
+    //      lines on the first cycle (subsequent cycles route those elsewhere), so
+    //      we'd end up with a near-empty log. Rolling at the START of the cycle
+    //      lets us capture whatever firebase-tools does emit, complete-or-not.
     const logPath = path.join(projectDir, 'functions', 'serve.log');
-    const logStream = fs.createWriteStream(logPath, { flags: 'w' });
+    const resetSentinelPath = `${logPath}.reset`;
+    // Match any node version: "Using node@22 from host.", "Using node@20 from host.", etc.
+    const RELOAD_MARKER = /Using node@\d+ from host\./;
     const stripAnsi = (str) => str.replace(/\x1B\[[0-9;]*[a-zA-Z]/g, '');
 
+    let currentStream = fs.createWriteStream(logPath, { flags: 'w' });
+    let reloadCount = 0; // skip rolling on the first marker (initial boot, not a reload)
+
+    function rollLog() {
+      try {
+        const oldStream = currentStream;
+        currentStream = fs.createWriteStream(logPath, { flags: 'w' });
+        oldStream.end();
+      } catch (e) {
+        // Best-effort. If roll fails, serve keeps running with the existing stream.
+      }
+    }
+
+    function writeToLog(data) {
+      if (currentStream && !currentStream.destroyed) {
+        currentStream.write(stripAnsi(data.toString()));
+      }
+    }
+
+    // Clean up any stale sentinel from a prior crashed serve run
+    try { fs.unlinkSync(resetSentinelPath); } catch (e) { /* not present, ok */ }
+
+    // Poll every 500ms for the reset sentinel — cheap, no fs.watch quirks
+    const resetWatcher = setInterval(() => {
+      if (!fs.existsSync(resetSentinelPath)) {
+        return;
+      }
+
+      try {
+        rollLog();
+        fs.unlinkSync(resetSentinelPath);
+      } catch (e) {
+        // Best-effort.
+      }
+    }, 500);
+
     this.log(chalk.gray(`  Logs saving to: ${logPath}\n`));
 
     // Execute with tee to log file
@@ -42,21 +94,35 @@ class ServeCommand extends BaseCommand {
           env: { ...process.env, FORCE_COLOR: '1' },
         },
       }, (child) => {
-        // Tee stdout to both console and log file (strip ANSI codes for clean log)
+        // Tee stdout to both console and log file (strip ANSI codes for clean log).
+        // Watch each chunk for the reload marker — when seen (after the initial boot),
+        // roll the log BEFORE writing this chunk so the marker becomes the first
+        // line of the fresh file.
         child.stdout.on('data', (data) => {
           process.stdout.write(data);
-          logStream.write(stripAnsi(data.toString()));
+          const text = data.toString();
+          if (RELOAD_MARKER.test(text)) {
+            reloadCount++;
+            if (reloadCount > 1) {
+              rollLog();
+            }
+          }
+          writeToLog(data);
         });
 
         // Tee stderr to both console and log file (strip ANSI codes for clean log)
         child.stderr.on('data', (data) => {
           process.stderr.write(data);
-          logStream.write(stripAnsi(data.toString()));
+          writeToLog(data);
         });
 
-        // Clean up log stream when child exits
+        // Clean up log stream + watcher when child exits
         child.on('close', () => {
-          logStream.end();
+          clearInterval(resetWatcher);
+          if (currentStream && !currentStream.destroyed) {
+            currentStream.end();
+          }
+          try { fs.unlinkSync(resetSentinelPath); } catch (e) { /* ok */ }
         });
       });
     } catch (error) {

package/src/cli/commands/test.js

@@ -6,6 +6,7 @@ const jetpack = require('fs-jetpack');
 const JSON5 = require('json5');
 const powertools = require('node-powertools');
 const { DEFAULT_EMULATOR_PORTS } = require('./setup-tests/emulator-config');
+const { writeTestMode, captureSyncedEnv, SYNCED_ENV_KEYS } = require('../../test/utils/test-mode-file');
 const EmulatorCommand = require('./emulator');
 
 class TestCommand extends BaseCommand {
@@ -20,6 +21,23 @@ class TestCommand extends BaseCommand {
     const projectDir = self.firebaseProjectPath;
     const functionsDir = path.join(projectDir, 'functions');
 
+    // Pre-flight: write the allowlisted env subset to a shared state file
+    // (`<projectDir>/.temp/test-mode.json`). The running emulator watches this
+    // file and mutates its own `process.env` to match, eliminating the need
+    // to coordinate env vars across both terminals. The test command is the
+    // authoritative writer — whatever you pass here becomes the live mode
+    // within ~50ms.
+    //
+    // Allowlist lives in src/test/utils/test-mode-file.js (SYNCED_ENV_KEYS).
+    // Today: just TEST_EXTENDED_MODE. Add more keys there to make them
+    // live-syncable.
+    {
+      const envSubset = captureSyncedEnv(process.env);
+      writeTestMode(projectDir, envSubset);
+      const extended = !!process.env.TEST_EXTENDED_MODE;
+      this.log(chalk.gray(`  Test mode: ${extended ? 'EXTENDED (real APIs)' : 'normal (mocked)'}`));
+    }
+
     // Load emulator ports from firebase.json
     const emulatorPorts = this.loadEmulatorPorts(projectDir);
 
@@ -33,7 +51,7 @@ class TestCommand extends BaseCommand {
     // Use hosting URL for all API requests (rewrites to bm_api function)
     const testConfig = {
       ...projectConfig,
-      hostingUrl: `http://127.0.0.1:${emulatorPorts.hosting}`,
+      apiUrl: `http://127.0.0.1:${emulatorPorts.hosting}`,
       projectDir,
       testPaths,
       emulatorPorts,
@@ -167,12 +185,58 @@ class TestCommand extends BaseCommand {
     return this.isPortInUse(emulatorPorts.functions);
   }
 
+  /**
+   * Signal the running emulator process to roll emulator.log.
+   *
+   * Mechanism: write a sentinel file at emulator.log.reset. The emulator command
+   * (src/cli/commands/emulator.js) polls for it and, on detection, closes its
+   * current write stream and reopens with flags: 'w' (truncating cleanly from its
+   * own perspective — avoids the sparse-file problem caused by external truncation).
+   *
+   * Waits up to 2s for the sentinel to be consumed. If it's still there after 2s
+   * the emulator isn't watching (probably running an older BEM or started outside
+   * `npx mgr emulator`); we delete the sentinel and proceed — tests still run, the
+   * log just won't be reset for this run.
+   */
+  async requestEmulatorLogReset(projectDir) {
+    const logPath = path.join(projectDir, 'functions', 'emulator.log');
+    const sentinelPath = `${logPath}.reset`;
+
+    try {
+      fs.writeFileSync(sentinelPath, '');
+    } catch (e) {
+      return; // Can't write — skip, not fatal
+    }
+
+    // Poll for the emulator to consume the sentinel (it deletes the file when done)
+    const maxWaitMs = 2000;
+    const pollIntervalMs = 100;
+    const start = Date.now();
+
+    while (Date.now() - start < maxWaitMs) {
+      if (!fs.existsSync(sentinelPath)) {
+        return; // Emulator picked it up and rolled the log
+      }
+      await new Promise((resolve) => setTimeout(resolve, pollIntervalMs));
+    }
+
+    // Timed out — emulator didn't see the sentinel. Clean up so we don't leave it behind.
+    try { fs.unlinkSync(sentinelPath); } catch (e) { /* ok */ }
+  }
+
   /**
    * Run tests directly (emulator already running)
    */
   async runTestsDirectly(testCommand, functionsDir, emulatorPorts) {
     const projectDir = this.main.firebaseProjectPath;
 
+    // Ask the running emulator process to roll emulator.log so this test run gets a
+    // clean slate. We touch a sentinel file the emulator polls for (every ~500ms) and
+    // wait briefly for it to be consumed. If the emulator isn't watching (older BEM
+    // version, or not started via `npx mgr emulator`), we time out silently — the log
+    // just won't be fresh, tests still run normally.
+    await this.requestEmulatorLogReset(projectDir);
+
     this.log(chalk.gray(`  Hosting: http://127.0.0.1:${emulatorPorts.hosting}`));
     this.log(chalk.gray(`  Firestore: 127.0.0.1:${emulatorPorts.firestore}`));
     this.log(chalk.gray(`  Auth: 127.0.0.1:${emulatorPorts.auth}`));

package/src/cli/commands/watch.js

@@ -53,13 +53,21 @@ class WatchCommand extends BaseCommand {
     // So we must: 1) ensure file exists, 2) wait for FS to settle, 3) write new content
     // --on-change-only: only run exec when files change, not on initial startup
     // --delay 1: debounce multiple rapid changes into one trigger
+    //
+    // The exec also drops <log>.reset sentinels so the parent serve/emulator command
+    // rolls its log file cleanly on every hot reload (mirrors the emulator log-roll
+    // pattern used by the test runner). Sentinels are best-effort — if a parent
+    // command isn't watching, the file is harmless and gets cleaned up by the next
+    // boot's stale-sentinel sweep.
     const triggerFile = config.triggerFile;
+    const serveLogResetPath = path.join(config.functionsDir, 'serve.log.reset');
+    const emulatorLogResetPath = path.join(config.functionsDir, 'emulator.log.reset');
     const nodemon = spawn(nodemonPath, [
       '--on-change-only',
       '--delay', '1',
       '--watch', config.bemSrcDir,
       '--ext', 'js,json',
-      '--exec', `node -e "var f='${triggerFile}',fs=require('fs');if(!fs.existsSync(f)){fs.writeFileSync(f,'// init');require('child_process').execSync('sleep 0.1');}fs.writeFileSync(f,'// '+Date.now())" && echo "  [BEM] Triggered hot reload"`,
+      '--exec', `node -e "var f='${triggerFile}',fs=require('fs');if(!fs.existsSync(f)){fs.writeFileSync(f,'// init');require('child_process').execSync('sleep 0.1');}fs.writeFileSync(f,'// '+Date.now());try{fs.writeFileSync('${serveLogResetPath}','');}catch(e){}try{fs.writeFileSync('${emulatorLogResetPath}','');}catch(e){}" && echo "  [BEM] Triggered hot reload"`,
     ], {
       stdio: 'inherit',
       detached: false,
@@ -93,11 +101,15 @@ class WatchCommand extends BaseCommand {
       jetpack.write(config.triggerFile, `// BEM reload trigger\n`);
     }
 
-    // Use nodemon to watch the BEM src directory and touch the trigger file on changes
+    // Use nodemon to watch the BEM src directory and touch the trigger file on changes.
+    // Also drop <log>.reset sentinels so any sibling serve/emulator command rolls its
+    // log file on each reload (mirrors the test runner's log-roll pattern).
+    const serveLogResetPath = path.join(config.functionsDir, 'serve.log.reset');
+    const emulatorLogResetPath = path.join(config.functionsDir, 'emulator.log.reset');
     const nodemon = spawn(nodemonPath, [
       '--watch', config.bemSrcDir,
       '--ext', 'js,json',
-      '--exec', `touch "${config.triggerFile}" && echo "  → Triggered hot reload"`,
+      '--exec', `touch "${config.triggerFile}" "${serveLogResetPath}" "${emulatorLogResetPath}" && echo "  → Triggered hot reload"`,
     ], {
       stdio: 'inherit',
       cwd: config.bemDir,

package/src/defaults/CLAUDE.md

@@ -16,11 +16,13 @@ This project consumes **Backend Manager** (BEM) — a comprehensive framework fo
 
 ```bash
 cd functions
-npx mgr setup       # validate config + scaffold defaults + run checks
-npx mgr emulator    # start Firebase emulators (auth/firestore/functions/database/storage)
-npx mgr watch       # auto-reload functions on file change
-npx mgr deploy      # deploy to Firebase
-npx mgr logs        # tail Cloud Functions logs
+npx mgr setup             # validate config + scaffold defaults + run checks
+npx mgr emulator          # start Firebase emulators (auth/firestore/functions/database/storage)
+npx mgr watch             # auto-reload functions on file change
+npx mgr deploy            # deploy to Firebase
+npx mgr logs:read         # read Cloud Functions logs (also: logs:tail to stream)
+npx mgr firestore:get     # read a doc from Firestore (also: firestore:set / :query / :delete)
+npx mgr auth:get          # read an Auth user (also: auth:list / :delete / :set-claims)
 ```
 
 All `npx mgr <cmd>` aliases — `npx bm <cmd>`, `npx bem <cmd>`, `npx backend-manager <cmd>` work too.

package/src/manager/events/cron/daily/marketing-newsletter-generate.js

@@ -94,14 +94,25 @@ module.exports = async ({ Manager, assistant, libraries }) => {
     const nowISO = new Date().toISOString();
     const nowUNIX = Math.round(Date.now() / 1000);
 
-    // Strip non-serializable fields out of the generator's return before
-    // writing to Firestore. `images` is an array of PNG Buffers (not safe
-    // to persist) and `mjml` is a raw template string that pollutes the doc.
-    const { images: _images, mjml: _mjml, assets, meta, ...campaignSettings } = generated;
+    // Strip non-serializable / oversized fields out of the generator's return
+    // before writing to Firestore.
+    //   images: Buffer[] — not safe to persist
+    //   mjml:   raw template string — pollutes the doc, available in the GH archive
+    //   structure: full JSON dump (5-10kb) — available via assets.markdownUrl + assets.htmlUrl
+    //   contentMarkdown: large markdown blob — available via assets.markdownUrl
+    const {
+      images: _images,
+      mjml: _mjml,
+      structure: _structure,
+      contentMarkdown: _contentMarkdown,
+      assets,
+      meta,
+      ...campaignSettings
+    } = generated;
 
     await admin.firestore().doc(`marketing-campaigns/${newId}`).set({
       settings: campaignSettings,
-      assets: assets || null,   // { folderUrl, htmlUrl, imageUrls, campaignId } or null
+      assets: assets || null,   // { folderUrl, htmlUrl, markdownUrl, summaryUrl, imageUrls, beehiivPostId, tags, campaignId }
       meta:   meta   || null,   // tokens, cost, durations, source scores
       type,
       sendAt: data.sendAt,
@@ -115,11 +126,16 @@ module.exports = async ({ Manager, assistant, libraries }) => {
 
     assistant.log(`Created campaign ${newId} from generator ${campaignId}: "${generated.subject}"`);
     if (assets?.htmlUrl) {
-      assistant.log(`  HTML:   ${assets.htmlUrl}`);
-      assistant.log(`  Folder: ${assets.folderUrl}`);
+      assistant.log(`  HTML:     ${assets.htmlUrl}`);
+      assistant.log(`  Markdown: ${assets.markdownUrl || '(none)'}`);
+      assistant.log(`  Summary:  ${assets.summaryUrl || '(none)'}`);
+      assistant.log(`  Folder:   ${assets.folderUrl}`);
     }
     if (assets?.beehiivPostId) {
-      assistant.log(`  Beehiiv: draft post ${assets.beehiivPostId}`);
+      assistant.log(`  Beehiiv:  draft post ${assets.beehiivPostId}`);
+    }
+    if (assets?.tags?.length) {
+      assistant.log(`  Tags:     ${assets.tags.join(', ')}`);
     }
 
     // Advance the recurring doc's sendAt to the next occurrence

package/src/manager/helpers/user.js

@@ -147,6 +147,35 @@ const SCHEMA = {
       page: { type: 'string', default: null, nullable: true },
     },
   },
+  consent: {
+    legal: {
+      status: { type: 'string', default: 'revoked' },
+      grantedAt: {
+        timestamp: { type: 'string', default: null, nullable: true },
+        timestampUNIX: { type: 'number', default: null, nullable: true },
+        source: { type: 'string', default: null, nullable: true },
+        ip: { type: 'string', default: null, nullable: true },
+        text: { type: 'string', default: null, nullable: true },
+      },
+    },
+    marketing: {
+      status: { type: 'string', default: 'revoked' },
+      grantedAt: {
+        timestamp: { type: 'string', default: null, nullable: true },
+        timestampUNIX: { type: 'number', default: null, nullable: true },
+        source: { type: 'string', default: null, nullable: true },
+        ip: { type: 'string', default: null, nullable: true },
+        text: { type: 'string', default: null, nullable: true },
+      },
+      revokedAt: {
+        timestamp: { type: 'string', default: null, nullable: true },
+        timestampUNIX: { type: 'number', default: null, nullable: true },
+        source: { type: 'string', default: null, nullable: true },
+        ip: { type: 'string', default: null, nullable: true },
+        text: { type: 'string', default: null, nullable: true },
+      },
+    },
+  },
 };
 
 /**