backend-manager 5.0.203 → 5.1.0

package/docs/payment-system.md

@@ -0,0 +1,325 @@
+# Payment System
+
+This document covers the full payment system: pipeline architecture, subscription model + statuses, transition handlers, processor interface, product configuration, and the test processor.
+
+## Pipeline
+
+The payment system follows a linear pipeline: **Intent → Webhook → On-Write → Transition**.
+
+1. **Intent** (`POST /payments/intent`): Client requests a payment session. BEM validates the product, generates an order ID (`XXXX-XXXX-XXXX`), and delegates to the processor module (e.g., Stripe creates a Checkout Session). Saves to `payments-intents/{orderId}`.
+
+2. **Webhook** (`POST /payments/webhook?processor=X&key=Y`): Processor sends event data. BEM parses and categorizes the event (`subscription` or `one-time`), extracts the UID, and saves to `payments-webhooks/{eventId}` with `status: 'pending'`.
+
+3. **On-Write** (Firestore trigger on `payments-webhooks/{eventId}`): Fetches the latest resource from the processor API (not stale webhook data), transforms it into a unified object, detects state transitions, dispatches handlers, tracks analytics, and writes to `users/{uid}.subscription` (subscriptions) and `payments-orders/{orderId}`.
+
+4. **Transitions** (fire-and-forget): Handler files run asynchronously after detection. Failures never block webhook processing. Skipped during tests unless `TEST_EXTENDED_MODE` is set.
+
+## 3-Layer Architecture
+
+The payment system is cleanly separated into three independent layers:
+
+| Layer | Purpose | Tests |
+|-------|---------|-------|
+| **Processor input** (Stripe, PayPal, Test) | Parse raw webhooks + transform to unified shape | Helper tests per processor (`payment/stripe/to-unified-subscription.js`, `payment/paypal/to-unified-one-time.js`, etc.) |
+| **Unified pipeline** (processor-agnostic) | Transition detection, Firestore writes, analytics | Journey tests (`journey-payments-*.js`) |
+| **Transition handlers** (fire-and-forget) | Emails, notifications, side effects | Skipped during tests unless `TEST_EXTENDED_MODE` |
+
+Each processor transforms its raw data into the **same unified shape**. Once data enters the pipeline, the code doesn't know or care which processor it came from. This means:
+- Adding a new processor = implement the processor interface (below). The pipeline handles the rest.
+- Journey tests use the `test` processor but exercise the full unified pipeline end-to-end.
+- Processor-specific tests only need to verify correct transformation to the unified shape.
+
+## Subscription Statuses
+
+| Status | Meaning | User can delete account? |
+|--------|---------|--------------------------|
+| `active` | Subscription is current and valid (includes trialing) | No (unless `product.id === 'basic'`) |
+| `suspended` | Payment failed (Stripe: `past_due`, `unpaid`) | No |
+| `cancelled` | Subscription terminated (Stripe: `canceled`, `incomplete`, `incomplete_expired`) | Yes |
+
+### Stripe Status Mapping
+
+| Stripe Status | `subscription.status` | Notes |
+|---|---|---|
+| `active` | `active` | Normal active subscription |
+| `trialing` | `active` | `trial.claimed = true` |
+| `past_due` | `suspended` | Payment failed, retrying |
+| `unpaid` | `suspended` | Payment failed |
+| `canceled` | `cancelled` | Subscription terminated |
+| `incomplete` | `cancelled` | Never completed initial payment |
+| `incomplete_expired` | `cancelled` | Expired before completion |
+| `active` + `cancel_at_period_end` | `active` | `cancellation.pending = true` |
+
+## Unified Subscription Object (`users/{uid}.subscription`)
+
+```javascript
+subscription: {
+  product: {
+    id: 'basic',                   // product ID from config ('basic', 'premium', etc.)
+    name: 'Basic',                 // display name from config
+  },
+  status: 'active',                // 'active' | 'suspended' | 'cancelled'
+  expires: { timestamp, timestampUNIX },
+  trial: {
+    claimed: false,                // has user EVER used a trial
+    expires: { timestamp, timestampUNIX },
+  },
+  cancellation: {
+    pending: false,                // true = cancel at period end
+    date: { timestamp, timestampUNIX },
+  },
+  payment: {
+    processor: null,               // 'stripe' | 'paypal' | etc.
+    orderId: null,                 // BEM order ID (e.g., '1234-5678-9012')
+    resourceId: null,              // provider subscription ID (e.g., 'sub_xxx')
+    frequency: null,               // 'monthly' | 'annually' | 'weekly' | 'daily'
+    price: 0,                      // resolved from config (number, e.g., 4.99)
+    startDate: { timestamp, timestampUNIX },
+    updatedBy: {
+      event: { name: null, id: null },
+      date: { timestamp, timestampUNIX },
+    },
+  },
+}
+```
+
+## Access Check Patterns
+
+```javascript
+// Is premium (paid)?
+user.subscription.status === 'active' && user.subscription.product.id !== 'basic'
+
+// Is on trial?
+user.subscription.trial.claimed && user.subscription.status === 'active'
+
+// Has pending cancellation?
+user.subscription.cancellation.pending === true
+
+// Payment failed?
+user.subscription.status === 'suspended'
+```
+
+## resolveSubscription(account)
+
+`User.resolveSubscription(account)` is a static method on the User helper that derives calculated subscription fields from raw account data. It returns only fields that require derivation logic — raw data (product.id, status, trial, cancellation) lives on the account object directly.
+
+```javascript
+const User = require('backend-manager/src/manager/helpers/user');
+
+const resolved = User.resolveSubscription(account);
+// Returns: { plan, active, trialing, cancelling }
+```
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `plan` | `string` | Effective plan ID the user has access to RIGHT NOW (`'basic'` if cancelled/suspended) |
+| `active` | `boolean` | User has active access (active, trialing, or cancelling) |
+| `trialing` | `boolean` | In an active trial (status `'active'` + `trial.claimed` + unexpired `trial.expires`) |
+| `cancelling` | `boolean` | Cancellation pending (status `'active'` + `cancellation.pending` + NOT trialing) |
+
+Accepts either a raw Firestore account object or a resolved `User` instance (checks both `account.subscription` and `account.properties.subscription`).
+
+**Unified with web-manager**: The same function exists as `auth.resolveSubscription(account)` in web-manager (`modules/auth.js`) with identical logic and return shape.
+
+**Use this instead of manual access checks** — it centralizes all the derivation logic in one place:
+
+```javascript
+// ✅ PREFERRED — use resolveSubscription
+const resolved = User.resolveSubscription(user);
+if (resolved.active) { /* has access */ }
+
+// ❌ AVOID — manual checks that duplicate logic
+if (user.subscription.status === 'active' && user.subscription.product.id !== 'basic') { /* ... */ }
+```
+
+## Transition Handlers
+
+When a webhook changes a subscription or processes a one-time payment, BEM detects the state transition and dispatches to a handler file. Handlers are fire-and-forget (non-blocking) — they run after the transition is detected but before or during the Firestore writes. Handler failures never block webhook processing.
+
+Handlers are skipped during tests unless `TEST_EXTENDED_MODE` is set.
+
+### Transition Detection
+
+The `transitions/index.js` module compares the **before** state (current `users/{uid}.subscription`) with the **after** state (new unified subscription) to detect what changed.
+
+### Subscription Transitions
+
+| Transition | Before → After | File |
+|---|---|---|
+| `new-subscription` | basic/null → active paid | `transitions/subscription/new-subscription.js` |
+| `payment-failed` | active → suspended | `transitions/subscription/payment-failed.js` |
+| `payment-recovered` | suspended → active | `transitions/subscription/payment-recovered.js` |
+| `cancellation-requested` | pending=false → pending=true | `transitions/subscription/cancellation-requested.js` |
+| `subscription-cancelled` | non-cancelled → cancelled | `transitions/subscription/subscription-cancelled.js` |
+| `plan-changed` | active product A → active product B | `transitions/subscription/plan-changed.js` |
+
+Note: Trials are NOT a separate transition. The `new-subscription` handler checks `after.trial.claimed` to determine if the subscription started with a trial.
+
+### One-Time Transitions
+
+| Transition | Event Type | File |
+|---|---|---|
+| `purchase-completed` | `checkout.session.completed` | `transitions/one-time/purchase-completed.js` |
+| `purchase-failed` | `invoice.payment_failed` | `transitions/one-time/purchase-failed.js` |
+
+### Handler Interface
+
+All handlers are in `src/manager/events/firestore/payments-webhooks/transitions/` and export a single async function:
+
+```javascript
+module.exports = async function ({ before, after, uid, userDoc, admin, assistant, Manager, eventType, eventId }) {
+  // before: previous subscription state (null for new/one-time)
+  // after: new unified state (subscription or one-time)
+  // userDoc: full user document data
+  // eventType: original webhook event type (e.g., 'customer.subscription.updated')
+  // eventId: webhook event ID
+};
+```
+
+### Creating a New Transition Handler
+
+1. Add detection logic in `transitions/index.js` (in priority order)
+2. Create handler file in `transitions/{category}/{name}.js`
+3. Handler receives full context — use `assistant.log()` for logging, `Manager.project.apiUrl` for API calls
+
+## Processor Interface
+
+Each processor implements three modules:
+
+**Intent processor** (`routes/payments/intent/processors/{processor}.js`):
+
+```javascript
+module.exports = {
+  async createIntent({ uid, orderId, product, productId, frequency, trial, confirmationUrl, cancelUrl, Manager, assistant }) {
+    return { id, url, raw };
+  },
+};
+```
+
+**Webhook processor** (`routes/payments/webhook/processors/{processor}.js`):
+
+```javascript
+module.exports = {
+  isSupported(eventType) { return boolean; },
+  parseWebhook(req) { return { eventId, eventType, category, resourceType, resourceId, raw, uid }; },
+};
+```
+
+**Cancel processor** (`routes/payments/cancel/processors/{processor}.js`):
+
+```javascript
+module.exports = {
+  async cancelAtPeriodEnd({ resourceId, uid, subscription, assistant }) { /* cancel at end of period */ },
+};
+```
+
+**Refund processor** (`routes/payments/refund/processors/{processor}.js`):
+
+```javascript
+module.exports = {
+  async processRefund({ resourceId, uid, subscription, assistant }) {
+    return { amount, currency, full };
+  },
+};
+```
+
+**Portal processor** (`routes/payments/portal/processors/{processor}.js`):
+
+```javascript
+module.exports = {
+  async createPortalSession({ resourceId, uid, returnUrl, assistant }) {
+    return { url };
+  },
+};
+```
+
+**Shared library** (`libraries/payment/processors/{processor}.js`):
+
+```javascript
+module.exports = {
+  init() { /* return SDK instance */ },
+  async fetchResource(resourceType, resourceId, rawFallback, context) { /* return resource */ },
+  getOrderId(resource) { /* return orderId string or null */ },
+  toUnifiedSubscription(rawSubscription, options) { /* return unified object */ },
+  toUnifiedOneTime(rawResource, options) { /* return unified object */ },
+};
+```
+
+## Product Resolution
+
+Products are resolved differently per processor, but always end up matching a product in `config.payment.products`:
+
+| Processor | Resolution chain | Stable ID |
+|-----------|-----------------|-----------|
+| **Stripe** | `sub.items.data[0].price.product` or `raw.plan.product` → match `product.stripe.productId` or `legacyProductIds` | `prod_xxx` |
+| **PayPal** | `sub → plan_id → plan → product_id` → match `product.paypal.productId` | PayPal catalog product ID |
+| **Test** | Uses `product.stripe.productId` in Stripe-shaped data | Same as Stripe |
+
+Falls back to `{ id: 'basic' }` if no match found.
+
+## Processor-Specific Details
+
+**Stripe:** Uses `metadata.uid` and `metadata.orderId` on subscriptions for UID/order resolution.
+
+**PayPal:** Uses `custom_id` field on subscriptions with format `uid:{uid},orderId:{orderId}`. Product resolution fetches the plan from the subscription, then gets `product_id` from the plan. Plans are scoped by `product_id` query param to avoid cross-brand matches on shared PayPal accounts.
+
+## Product Configuration
+
+Products are defined in `backend-manager-config.json` under `payment.products`:
+
+```javascript
+payment: {
+  processors: {
+    stripe: { publishableKey: 'pk_live_...' },
+    paypal: { clientId: 'ARvf...' },
+  },
+  products: [
+    {
+      id: 'basic',           // Free tier (no prices, no processor keys)
+      name: 'Basic',
+      type: 'subscription',
+      limits: { requests: 100 },
+    },
+    {
+      id: 'premium',         // Paid subscription
+      name: 'Premium',
+      type: 'subscription',
+      limits: { requests: 1000 },
+      trial: { days: 14 },
+      prices: { monthly: 4.99, annually: 49.99 },       // Flat numbers; also supports 'weekly' and 'daily'
+      stripe: { productId: 'prod_xxx', legacyProductIds: ['prod_OLD'] },
+      paypal: { productId: 'PROD-abc123' },
+    },
+    {
+      id: 'credits-100',     // One-time purchase
+      name: '100 Credits',
+      type: 'one-time',
+      prices: { once: 9.99 },
+      stripe: { productId: 'prod_yyy' },
+      paypal: { productId: null },
+    },
+  ],
+}
+```
+
+Key rules:
+- `prices` contains **flat numbers only** — no processor-specific IDs
+- Processor IDs live at the product level: `stripe: { productId }`, `paypal: { productId }`
+- `stripe.productId` is stable — never changes even when prices change
+- `stripe.legacyProductIds` maps old pre-migration Stripe products to this product
+- Price IDs (Stripe `price_xxx`, PayPal plan IDs) are **resolved at runtime** by matching amount + interval against active prices on the processor's product
+- `basic` product has no `prices` and no processor keys — it's the free tier
+- `archived: true` stops offering a product to new subscribers while keeping it resolvable for existing ones
+
+## Firestore Collections
+
+| Collection | Key | Purpose |
+|---|---|---|
+| `payments-intents/{orderId}` | Order ID | Intent metadata (processor, product, status) |
+| `payments-webhooks/{eventId}` | Processor event ID | Webhook processing state + transition result |
+| `payments-orders/{orderId}` | Order ID | Unified order data (single source of truth for orders) |
+| `users/{uid}.subscription` | User UID | Current subscription state (subscriptions only) |
+
+## Test Processor
+
+The `test` processor generates Stripe-shaped data and auto-fires webhooks to the local server. Only available in non-production environments. Use `processor: 'test'` in intent requests during testing. The test webhook processor delegates to Stripe's parser since it generates Stripe-shaped payloads.

package/docs/response-headers.md

@@ -0,0 +1,7 @@
+# Response Headers
+
+BEM automatically sets `bm-properties` header with:
+- `code`: HTTP status code
+- `tag`: Function name and execution ID
+- `usage`: Current usage stats
+- `schema`: Resolved schema info

package/docs/routes.md

@@ -0,0 +1,126 @@
+# Creating Routes, API Commands, Events, Cron Jobs
+
+Recipes for building consumer-side routes plus BEM-side API commands, event handlers, and cron jobs. See also [docs/schemas.md](schemas.md) for schema definitions, [docs/auth-hooks.md](auth-hooks.md) for auth lifecycle hooks, and [docs/common-operations.md](common-operations.md) for inside-the-handler patterns.
+
+## New API Command
+
+Create `src/manager/functions/core/actions/api/{category}/{action}.js`:
+
+```javascript
+function Module() {}
+
+Module.prototype.main = function () {
+  const self = this;
+  const Manager = self.Manager;
+  const Api = self.Api;
+  const assistant = self.assistant;
+  const payload = self.payload;
+
+  return new Promise(async function(resolve, reject) {
+    // Validate input
+    if (!payload.data.payload.requiredField) {
+      return reject(assistant.errorify('Missing required field', { code: 400 }));
+    }
+
+    // Business logic here
+    const result = { success: true };
+
+    // Log and return
+    assistant.log('Action completed', result);
+    return resolve({ data: result });
+  });
+};
+
+module.exports = Module;
+```
+
+## New Route (Consumer Project)
+
+Create `routes/{name}/index.js`:
+
+```javascript
+function Route() {}
+
+Route.prototype.main = async function (assistant) {
+  const Manager = assistant.Manager;
+  const usage = assistant.usage;
+  const user = assistant.usage.user;
+  const analytics = assistant.analytics;
+  const settings = assistant.settings;
+
+  // Check authentication if needed
+  if (!user.authenticated) {
+    return assistant.respond('Authentication required', { code: 401 });
+  }
+
+  // Track usage
+  await usage.validate('requests');
+  usage.increment('requests');
+  await usage.update();
+
+  // Send response
+  assistant.respond({ success: true, data: settings });
+};
+
+module.exports = Route;
+```
+
+## New Event Handler
+
+Create `src/manager/functions/core/events/{type}/{event}.js`:
+
+```javascript
+function Module() {}
+
+Module.prototype.init = function (Manager, payload) {
+  const self = this;
+  self.Manager = Manager;
+  self.assistant = Manager.Assistant();
+  self.libraries = Manager.libraries;
+  self.user = payload.user;
+  self.context = payload.context;
+  return self;
+};
+
+Module.prototype.main = function () {
+  const self = this;
+  const Manager = self.Manager;
+  const assistant = self.assistant;
+
+  return new Promise(async function(resolve, reject) {
+    const { admin } = self.libraries;
+
+    assistant.log('Event triggered', self.user);
+
+    // Event logic here
+
+    return resolve(self);
+  });
+};
+
+module.exports = Module;
+```
+
+## New Cron Job (Consumer Project)
+
+Create `hooks/cron/daily/{job}.js`:
+
+```javascript
+function Job() {}
+
+Job.prototype.main = function () {
+  const self = this;
+  const Manager = self.Manager;
+  const assistant = self.assistant;
+
+  return new Promise(async function(resolve, reject) {
+    assistant.log('Running daily job...');
+
+    // Job logic here
+
+    return resolve();
+  });
+};
+
+module.exports = Job;
+```

package/docs/sanitization.md

@@ -0,0 +1,61 @@
+# Sanitization (XSS Prevention)
+
+BEM automatically sanitizes all incoming request data — stripping HTML tags and trimming whitespace from every string field. This happens in the middleware pipeline before route handlers execute, so **routes receive clean data by default**.
+
+## How It Works
+
+1. **Schema fields**: Sanitized per-field during the middleware pipeline. Fields can opt out with `sanitize: false` in the schema.
+2. **Non-schema fields** (when `setupSettings: false` or `includeNonSchemaSettings: true`): All strings are sanitized with no opt-out.
+3. The middleware uses `Manager.Utilities().sanitize()` under the hood.
+
+## Schema Opt-Out
+
+For fields that legitimately need HTML (rich text, email templates, etc.), set `sanitize: false` in the schema:
+
+```javascript
+// This field will NOT be sanitized — raw HTML is preserved
+htmlContent: {
+  types: ['string'],
+  default: '',
+  sanitize: false,
+},
+// This field IS sanitized (default behavior, no flag needed)
+name: {
+  types: ['string'],
+  default: '',
+},
+```
+
+## Route-Level Opt-Out
+
+Disable sanitization entirely for a route (rare — only for routes that handle raw HTML everywhere):
+
+```javascript
+// In functions/index.js
+Manager.Middleware(req, res).run('my-route', { sanitize: false });
+```
+
+## Manual Sanitization (Outside Middleware)
+
+For cron jobs, event handlers, or anywhere outside the request pipeline, use `utilities.sanitize()` directly:
+
+```javascript
+// Available in route context
+const clean = utilities.sanitize(untrustedData);
+
+// Or via Manager
+const clean = Manager.Utilities().sanitize(untrustedData);
+```
+
+Accepts any data type — strings, objects, arrays, primitives. Walks objects/arrays recursively, strips HTML from strings, passes everything else through unchanged.
+
+## Route Handler Context
+
+The middleware injects these into every route handler:
+
+```javascript
+module.exports = async ({ Manager, assistant, analytics, usage, user, settings, libraries, utilities }) => {
+  // settings    — already sanitized by middleware
+  // utilities   — Manager.Utilities() instance for manual sanitization
+};
+```

package/docs/schemas.md

@@ -0,0 +1,39 @@
+# Schemas
+
+Schemas define and validate the payload your routes accept. Schema names should match the route name (e.g. route `myEndpoint` ↔ schema `myEndpoint`).
+
+## New Schema (Consumer Project)
+
+Create `schemas/{name}/index.js`:
+
+```javascript
+module.exports = function (assistant, settings, options) {
+  const user = options.user;
+
+  return {
+    defaults: {
+      fieldName: {
+        types: ['string'],
+        default: 'default value',
+        required: false,
+      },
+      numericField: {
+        types: ['number'],
+        default: 10,
+        min: 1,
+        max: 100,
+      },
+    },
+    // Override for premium users
+    premium: {
+      numericField: {
+        max: 1000,
+      },
+    },
+  };
+};
+```
+
+## Field Sanitization
+
+By default, every string field in a schema is sanitized (HTML tags stripped, whitespace trimmed) during the middleware pipeline. To preserve raw HTML (rich text, email templates), set `sanitize: false` on the field. See [docs/sanitization.md](sanitization.md).

package/docs/stripe-webhook-forwarding.md

@@ -0,0 +1,18 @@
+# Stripe Webhook Forwarding
+
+BEM auto-starts Stripe CLI webhook forwarding when running `npx mgr serve` or `npx mgr emulator`. This forwards Stripe test webhooks to the local server so the full payment pipeline works end-to-end during development.
+
+**Requirements:**
+- `STRIPE_SECRET_KEY` set in `functions/.env`
+- `BACKEND_MANAGER_KEY` set in `functions/.env`
+- [Stripe CLI](https://stripe.com/docs/stripe-cli) installed
+
+**Standalone usage:**
+
+```bash
+npx mgr stripe
+```
+
+If any prerequisite is missing, webhook forwarding is silently skipped with an info message.
+
+The forwarding URL is: `http://localhost:{hostingPort}/backend-manager/payments/webhook?processor=stripe&key={BACKEND_MANAGER_KEY}`