@terreno/mcp 57.2.0 → 57.4.0

package/dist/docs/versioned/57.2.0/implementationPlans/comms-adapter-twilio-sms.md

@@ -0,0 +1,155 @@
+# Implementation Plan: Comms adapter — Twilio SMS
+
+**Status:** Draft
+**Roadmap issue:** https://github.com/FlourishHealth/terreno/issues/1020
+**Priority:** Medium
+**Effort:** Small batch
+**Owner:** unassigned
+**Created:** 2026-08-09
+**Program:** [B2B platform](b2b-platform-program.md)
+**Depends on:** [comms-abstraction](comms-abstraction.md); inbound-webhooks (Phase 2 only)
+**RTK deprecation flag:** None — backend only
+
+## Goal
+
+Implement `SmsProvider` on Twilio Programmable Messaging so Terreno apps can send
+transactional SMS (OTP fallback, alerts, invitation nudges) through the `@terreno/comms`
+facade with delivery logging.
+
+## Non-Goals
+
+- OTP verification (that is `comms-adapter-twilio-verify`, on Twilio Verify).
+- Push via Twilio (dropped by decision D5 — Expo push is the push adapter).
+- Bulk/campaign messaging, MMS/RCS.
+- Phone-number provisioning automation.
+
+## Decisions
+
+| Question | Decision |
+|----------|----------|
+| SDK | `twilio` as optional peer, subpath `@terreno/comms/adapters/twilioSms` |
+| Sender config | Prefer `TWILIO_MESSAGING_SERVICE_SID`; fall back to `TWILIO_FROM_NUMBER`; constructor options override env |
+| Number validation | Validate/normalize to E.164 with `libphonenumber-js` (already in catalog) before hitting the API |
+| Delivery status | Phase 2: status callback endpoint via the inbound-webhooks framework mapping Twilio statuses to `DeliveryEvent`s |
+| Secrets | `TWILIO_ACCOUNT_SID` + `TWILIO_AUTH_TOKEN` via env or the existing secret-provider layer |
+| Error classification | Adapter owns a Twilio error-code → `errorClass` map (table below) so the facade retries only transient failures and the admin dashboard gates manual retries correctly |
+| SDK failure handling | Twilio SDK calls wrapped with `withApiErrorHandling` from `@terreno/api` (per [call-external-apis](../how-to/call-external-apis.md)) — the SDK owns transport, the wrapper standardizes logging/normalization |
+| Opt-outs | Phase 2: inbound STOP/START (and 21610 send failures) surface as `OptOutEvent` → the consumer's `onOptOut` hook; enforcement stays consumer-side via `beforeSend` (Twilio blocks opted-out sends itself) |
+| Admin dashboard support | Every `CommsMessage` row stores the Twilio error code/class and a `metadata.consoleUrl` deep link to the message in the Twilio console for log digging |
+
+## Architecture
+
+```typescript
+// @terreno/comms/adapters/twilioSms
+export class TwilioSmsProvider implements SmsProvider {
+  readonly id = "twilio";
+  constructor(options?: {
+    accountSid?: string;      // default env TWILIO_ACCOUNT_SID
+    authToken?: string;       // default env TWILIO_AUTH_TOKEN
+    messagingServiceSid?: string;
+    fromNumber?: string;
+    statusCallbackUrl?: string; // Phase 2
+  });
+  sendSms(message: {to: string; body: string}): Promise<SendResult>;
+}
+```
+
+Errors map Twilio exception codes to `SendResult.error`/`errorCode`/`errorClass`, with
+the full Twilio error payload kept in `CommsMessage.metadata` for triage. Invalid
+destination numbers fail fast with a `BadRequestError` before any API call.
+
+### Error classification
+
+The adapter classifies Twilio error codes into the abstraction's `errorClass` taxonomy.
+Permanent failures must never be auto-retried (retrying an unsubscribed number is a
+compliance problem, not a delivery problem); transient failures are retried once inline
+by the facade and stay manually retryable from the admin dashboard.
+
+| Twilio code(s) | Meaning | `errorClass` |
+|---|---|---|
+| 21211, 21214, 21217 | Invalid / unroutable destination number | `permanent` |
+| 21408 | SMS not enabled for destination region | `permanent` |
+| 21610 | Recipient has opted out (STOP) | `permanent` (also emits `OptOutEvent`) |
+| 30003, 30005, 30006 | Unreachable handset / unknown device / landline | `permanent` |
+| 30007 | Carrier content filtering | `permanent` (surface loudly — usually a sender-reputation problem) |
+| 20429, 429 responses | Rate limited / concurrency limit | `transient` |
+| 30001, 30002 | Queue overflow / account suspended-ish delivery interruption | `transient` |
+| 5xx / network errors | Twilio-side or transport failure | `transient` |
+| 20003, 20404 (auth/SID misconfig) | Bad credentials or missing resource | `config` |
+
+Unknown codes default to `transient` (safe for one retry) and are logged with the raw
+code so the map can be extended. `config` failures additionally `logger.error` at send
+time — they mean the app is misconfigured, not that the message was undeliverable.
+
+### Status callback mapping (Phase 2)
+
+| Twilio `MessageStatus` | `DeliveryEvent.status` | `CommsMessage.status` |
+|---|---|---|
+| `queued`, `accepted`, `sending`, `sent` | — (no event; already `sent`) | `sent` |
+| `delivered` | `delivered` | `delivered` |
+| `undelivered`, `failed` | `failed` (+ `errorCode` from `ErrorCode` param) | `failed` |
+
+Inbound `STOP`/`START` keyword webhooks (same inbound-webhooks registration) map to
+`OptOutEvent {channel: "sms", reason: "sms-stop"}` / opt-in, feeding the consumer's
+`onOptOut` hook and `CommsMessage.metadata` on subsequent 21610 failures.
+
+## Models / APIs / Notifications / UI
+
+None new — Phase 2 adds one webhook route via the inbound-webhooks framework.
+
+## Phases
+
+1. **Send path:** provider, config resolution, E.164 validation, error-code
+   classification map, `withApiErrorHandling` wrapping, console deep-link metadata, tests
+   with mocked Twilio client; example-backend env-gated registration; docs.
+2. **Delivery callbacks + opt-outs** *(gated on inbound-webhooks IP)*: status callback
+   route with Twilio signature verification, statuses → `DeliveryEvent` →
+   `CommsMessage.status` updates (with `errorCode`), STOP/START keyword handling →
+   `OptOutEvent` → `onOptOut`.
+
+## Feature Flags & Migrations
+
+None.
+
+## Activity Log & User Updates
+
+All sends logged to `CommsMessage` with provider `twilio`.
+
+## Not Included / Future Work
+
+- Inbound SMS (replies) handling beyond STOP/START keywords.
+- Alphanumeric sender IDs, international compliance tooling.
+- Terreno-managed opt-out suppression list — the adapter emits `OptOutEvent`; storing and
+  enforcing preferences is consumer-side (`beforeSend`) until `notification-center` lands.
+
+## Files to Create / Modify
+
+- `comms/src/adapters/twilioSms.ts` + tests
+- `comms/package.json` — optional peer `twilio`
+- `example-backend/src/server.ts` — env-gated registration
+- `docs/reference/comms.md`, `docs/reference/environment-variables.md`
+
+## Task List
+
+See [docs/tasks/comms-adapter-twilio-sms.md](../tasks/comms-adapter-twilio-sms.md).
+
+## Acceptance Criteria
+
+- [ ] `sendSms` delivers via messaging service or from-number config, preferring the
+      messaging service when both are present.
+- [ ] A non-E.164 destination fails with a 400 `APIError` before any Twilio call.
+- [ ] Twilio API errors produce `accepted: false` results with `errorCode` and a correct
+      `errorClass` per the classification table; the full Twilio error payload lands in
+      `CommsMessage.metadata`; nothing throws through the facade.
+- [ ] Permanent-class failures (21610, 30003, …) are never retried inline and appear as
+      non-retryable in the admin dashboard; transient-class failures retry once and stay
+      manually retryable.
+- [ ] Each send stores `metadata.consoleUrl` linking the message SID to the Twilio
+      console.
+- [ ] Consumer `onError` fires with the classified `SendResult` on every failed send
+      (mocked-client test).
+- [ ] Apps not using the adapter do not install the `twilio` SDK.
+- [ ] (Phase 2) Status callbacks flip `CommsMessage.status` to `delivered`/`failed` (with
+      `errorCode`) under verified Twilio signatures.
+- [ ] (Phase 2) An inbound STOP fires `onOptOut` with `reason: "sms-stop"`; a subsequent
+      send to that number records 21610 as `permanent`.

package/dist/docs/versioned/57.2.0/implementationPlans/comms-adapter-twilio-verify.md

@@ -0,0 +1,99 @@
+# Implementation Plan: Comms adapter — Twilio Verify (OTP)
+
+**Status:** Draft
+**Roadmap issue:** https://github.com/FlourishHealth/terreno/issues/1021
+**Priority:** Medium
+**Effort:** Small batch
+**Owner:** unassigned
+**Created:** 2026-08-09
+**Program:** [B2B platform](b2b-platform-program.md)
+**Depends on:** [comms-abstraction](comms-abstraction.md)
+**RTK deprecation flag:** None — backend only
+
+## Goal
+
+Implement `VerificationProvider` on Twilio Verify so Terreno apps get managed one-time-code
+flows (SMS and email channels) without storing, expiring, or rate-limiting codes
+themselves. This is the delivery channel the future `mfa-step-up-auth` work builds on, and
+it is immediately usable for phone-number verification.
+
+## Non-Goals
+
+- The MFA enrollment/step-up product itself (`mfa-step-up-auth` item).
+- TOTP (app-authenticator) support — that is code-local (`otpauth`), not a Verify concern.
+- Custom code storage or self-managed rate limiting (Verify owns both).
+
+## Decisions
+
+| Question | Decision |
+|----------|----------|
+| SDK | `twilio` optional peer (shared with the SMS adapter), subpath `@terreno/comms/adapters/twilioVerify` |
+| Service config | `TWILIO_VERIFY_SERVICE_SID` env or constructor option |
+| Channels | `sms` and `email` (Verify's email channel requires a configured Verify email integration; documented, not automated) |
+| Abuse control | Rely on Verify's built-in rate limits; add Terreno-side per-user attempt caps only when `rate-limiting` IP lands |
+| Logging | Start/check attempts logged to `CommsMessage` with channel `verification`, never logging the code |
+| Error classification | Verify errors map to the abstraction taxonomy: 20429/429 and 5xx → `transient`; max-attempts, expired, and invalid-destination → `permanent`; missing/invalid service SID → `config` |
+| Admin dashboard support | Verification rows appear in the [comms-admin-dashboard](comms-admin-dashboard.md) with `errorCode`/`errorClass` and a `metadata.consoleUrl` deep link to the Verify service logs — but the `verification` channel is **never retryable** from admin (codes expire; the user restarts the flow) |
+| Hooks | `onError` fires on failed starts/checks like any other channel; `beforeSend` can gate verification starts (e.g. per-user cool-downs) |
+
+## Architecture
+
+```typescript
+// @terreno/comms/adapters/twilioVerify
+export class TwilioVerifyProvider implements VerificationProvider {
+  readonly id = "twilio-verify";
+  constructor(options?: {accountSid?: string; authToken?: string; verifyServiceSid?: string});
+  startVerification(options: {to: string; channel: "sms" | "email"}): Promise<SendResult>;
+  checkVerification(options: {to: string; code: string}): Promise<{valid: boolean; error?: string}>;
+}
+```
+
+`checkVerification` maps Verify's `approved` status to `valid: true`; `pending`, `expired`,
+and max-attempt states return `valid: false` with reason in the result error, letting
+callers throw the appropriate `APIError`.
+
+## Models / APIs / Notifications / UI
+
+None new. Consumer routes (e.g. `POST /auth/verifyPhone`) arrive with the features that
+need them; this IP ships the provider plus example-backend registration only.
+
+## Phases
+
+Single phase.
+
+## Feature Flags & Migrations
+
+None.
+
+## Activity Log & User Updates
+
+Verification attempts logged to `CommsMessage` (no codes; destination fully redacted by core,
+with the verification channel stored in metadata).
+
+## Not Included / Future Work
+
+- MFA enrollment, recovery codes, biometric step-up (`mfa-step-up-auth`).
+- WhatsApp/voice channels.
+- Android SMS autofill hash configuration (`react-native-otp-verify`, decision D7).
+
+## Files to Create / Modify
+
+- `comms/src/adapters/twilioVerify.ts` + tests (mocked Twilio client)
+- `example-backend/src/server.ts` — env-gated registration
+- `docs/reference/comms.md`, `docs/reference/environment-variables.md`
+
+## Task List
+
+See [docs/tasks/comms-adapter-twilio-verify.md](../tasks/comms-adapter-twilio-verify.md).
+
+## Acceptance Criteria
+
+- [ ] `startVerification` creates a Verify verification on the configured service for sms
+      and email channels.
+- [ ] `checkVerification` returns `valid: true` only for `approved`; expired and
+      max-attempt states return `valid: false` with a reason.
+- [ ] Failed starts/checks record `errorCode`/`errorClass` per the classification row and
+      fire the consumer `onError` hook; verification rows are marked non-retryable for
+      the admin dashboard.
+- [ ] No OTP code ever appears in logs or `CommsMessage` rows; destinations are redacted.
+- [ ] Missing `TWILIO_VERIFY_SERVICE_SID` fails fast at registration with a clear error.

package/dist/docs/versioned/57.2.0/implementationPlans/comms-admin-dashboard.md

@@ -0,0 +1,180 @@
+# Implementation Plan: Comms admin dashboard (errors, retries, log digging)
+
+**Status:** Draft
+**Roadmap issue:** https://github.com/FlourishHealth/terreno/issues/1091
+**Priority:** High
+**Effort:** Medium batch
+**Owner:** unassigned
+**Created:** 2026-08-11
+**Program:** [B2B platform](b2b-platform-program.md)
+**Depends on:** [comms-abstraction](comms-abstraction.md) (Phases 1–2, including the hook + error-taxonomy additions); adapters optional but the dashboard is most useful once [comms-adapter-twilio-sms](comms-adapter-twilio-sms.md) and [comms-adapter-sendgrid](comms-adapter-sendgrid.md) land
+**RTK deprecation flag:** Partial — screens use the generated SDK; migrate with syncdb like other admin screens
+
+## Goal
+
+Make the admin panel the single place to **operate** the comms layer: see errors and
+failure rates at a glance, filter and dig into individual delivery logs (attempt history,
+provider error codes, raw provider metadata), and retry failed sends with one click —
+across every channel and provider, without touching Twilio/SendGrid consoles or raw Mongo.
+
+The comms-abstraction IP ships a minimal read-only `GET /comms/messages` explorer. This IP
+upgrades that into a real operations dashboard, following the same pattern the AI package
+uses (`addAiRequestsExplorerRoutes`) but with mutation (retry) and much richer filtering.
+
+## Non-Goals
+
+- Building the messages themselves (comms-abstraction) or provider integrations (adapter
+  IPs).
+- Charts beyond simple count cards — themed chart components arrive with
+  `charts-and-dashboards`; the stats endpoint is designed so charts can be added then.
+- Broadcast/bulk *composition* from admin (sending new messages, campaign tooling).
+- User-facing notification history (`notification-center` item).
+- Cross-service log aggregation (Sentry/Cloud Logging stay the deep-debug tools; the
+  dashboard links out via `providerMessageId`).
+
+## Decisions
+
+| Question | Decision |
+|----------|----------|
+| Where do the routes live? | `@terreno/comms` (`src/routes/commsDashboard.ts`) — they operate on `CommsMessage`, which the comms package owns. `@terreno/admin-backend` stays model-generic |
+| Where does the UI live? | `@terreno/admin-frontend` (`CommsDashboard*` components), rendered inside `AdminShellLayout` like the rest of the admin panel, and wired into example-frontend + admin-spa |
+| Retry semantics | Retry creates a **new** `CommsMessage` row linked back via `retriedFromId`; the original row is never mutated except `retriedById`. Sends go through the normal `commsService` facade so hooks, logging, and error taxonomy all apply |
+| What is retryable? | `status: "failed" \| "bounced"` **and** `errorClass !== "permanent"` (permanent failures — invalid number, unsubscribed, suppressed address — get a disabled button with the reason) **and** a retained payload exists. `verification` channel is never retryable (codes expire; users restart the flow) |
+| Payload retention for retry | `CommsApp` option `retainPayloadDays` (default 30) keeps the rendered message body on the `CommsMessage` row, passed through the `redactPayload` hook before storage; without a retained payload the retry button is disabled with the reason |
+| Filtering | Server-side query params, not client-side table filtering — delivery logs get large. Free-text `q` matches `subject`, `error`, and `to` (last-4 for redacted destinations) |
+| Bulk retry | `POST /comms/messages/retryMany` takes the same filter object as the list route plus a hard cap (default 100 per call) so an admin can re-drive an outage window without scripting |
+| Stats | One aggregation endpoint with day buckets per channel/provider/status; rendered as count cards now, charts later |
+
+## Architecture
+
+```
+@terreno/comms
+  src/routes/commsDashboard.ts   # list/detail/retry/retryMany/stats (replaces commsExplorer.ts)
+  src/commsService.ts            # retryMessage({messageId, req}) added to the facade
+
+@terreno/admin-frontend
+  src/comms/CommsDashboardScreen.tsx   # cards + filter bar + DataTable
+  src/comms/CommsMessageDetail.tsx     # attempt timeline + raw metadata + retry
+  src/comms/CommsStatusBadge.tsx       # status → Badge mapping
+  src/comms/useCommsDashboardApi.tsx   # RTK Query hooks (pattern: useAdminApi)
+```
+
+### Retry flow
+
+```
+Admin clicks Retry
+  → POST /comms/messages/:id/retry (IsAdmin)
+  → commsService.retryMessage: validates retryability, loads retained payload
+  → normal send path (beforeSend/onSend/onError hooks fire with isRetry: true)
+  → new CommsMessage row {retriedFromId: original._id}
+  → original row gets {retriedById: new._id}
+  → response returns the new row; UI navigates to it
+```
+
+## Models
+
+No new models. Uses the `CommsMessage` fields added by the comms-abstraction IP
+(`errorCode`, `errorClass`, `attempts[]`, `attemptCount`, `lastAttemptAt`,
+`retriedFromId`, `retriedById`, `templateId`, retained `payload`).
+
+## APIs
+
+All routes `Permissions.IsAdmin`, built with `createOpenApiBuilder`, in `/openapi.json`.
+
+| Method | Path | Notes |
+|---|---|---|
+| GET | `/comms/messages` | Paginated list. Filters: `channel`, `provider`, `status`, `errorClass`, `errorCode`, `userId`, `to`, `templateId`, `retriedFromId`, `startDate`, `endDate`, free-text `q`; sort `-created` default |
+| GET | `/comms/messages/:id` | Full detail: attempts array, raw provider `metadata`, retained payload (post-redaction), linked retry rows |
+| POST | `/comms/messages/:id/retry` | Re-send; 400 with a stable `code` (`comms-retry-not-retryable`, `comms-retry-payload-expired`, `comms-retry-channel-unconfigured`) when not retryable |
+| POST | `/comms/messages/retryMany` | Body: same filter object as list + `limit` (cap 100); returns `{retried, skipped: [{id, reason}]}` |
+| GET | `/comms/stats` | Aggregation: counts by `channel` × `provider` × `status` with day buckets over `startDate`/`endDate` (default 7d); includes failure rate per provider |
+
+## Notifications
+
+None. (A failure-rate alert notifier is Future Work — see below.)
+
+## UI
+
+All screens live in `@terreno/admin-frontend`, use `@terreno/ui` components only, and
+render on the `AdminShellLayout` canvas (`Page` with `color="transparent"`, `padding={0}`).
+
+- **`CommsDashboardScreen`**
+  - Summary `Card` row from `/comms/stats`: sent / delivered / failed / bounced for the
+    selected range, failure rate per provider, with an error-colored card when the failure
+    rate exceeds 5%.
+  - Filter bar: `SelectField`s for channel/provider/status/errorClass, `TextField`
+    (search icon) for `q`, `DateTimeField` pair for the range. Filters map 1:1 to list
+    query params and persist in the URL so views are shareable.
+  - `DataTable`: created, channel, provider, to (redacted), subject/preview, status
+    (`CommsStatusBadge`), errorCode, attemptCount; row click → detail; inline Retry
+    `IconButton` on retryable rows.
+  - Loading, error, and empty states per admin-frontend conventions.
+- **`CommsMessageDetail`**
+  - Header: status badge, channel/provider, recipient, timestamps, linked user
+    (`AdminRefField` → User), `retriedFromId`/`retriedById` links.
+  - **Attempt timeline**: one row per `attempts[]` entry — timestamp, provider,
+    `providerMessageId` (rendered as an external link to the provider console when the
+    adapter supplies `consoleUrl` in metadata), errorCode + error message.
+  - **Log digging**: collapsible JSON viewer (`Accordion` + `MarkdownView` code block) for
+    retained payload and raw provider `metadata` — everything the adapter recorded, post
+    redaction.
+  - **Retry** `Button` (`withConfirmation`) — disabled with a tooltip reason when not
+    retryable.
+
+## Phases
+
+1. **Backend routes:** upgrade the explorer to `commsDashboard.ts` (filters, detail,
+   stats), `retryMessage` on the facade, retry + retryMany routes, supertest coverage for
+   every filter and every non-retryable reason.
+2. **Admin screens:** `CommsDashboardScreen`, `CommsMessageDetail`, badge + hooks; wire
+   into example-frontend admin routes and admin-spa nav; regenerate SDK; frontend
+   verification with screenshots.
+3. **Bulk retry + docs:** `retryMany` UI (filtered "Retry all failed" with count
+   confirmation), `docs/reference/comms.md` dashboard section, rulesync.
+
+## Feature Flags & Migrations
+
+None. New routes and screens only; `CommsMessage` fields ship in comms-abstraction.
+
+## Activity Log & User Updates
+
+Every retry is itself a `CommsMessage` row (with `retriedFromId` and the acting admin in
+`metadata.retriedByUserId`), so the audit trail is the same surface the dashboard shows.
+
+## Not Included / Future Work
+
+- Failure-rate alerting (Slack/Google Chat notifier when a provider's failure rate spikes)
+  — small follow-up once `/comms/stats` exists.
+- Charts on the stats endpoint (`charts-and-dashboards`).
+- Editing a message before retry ("retry with edits").
+- Automatic scheduled retries with backoff — arrives with `job-queues`; this IP is
+  explicitly *manual, admin-driven* retry.
+
+## Files to Create / Modify
+
+- `comms/src/routes/commsDashboard.ts` (+ tests) — replaces `commsExplorer.ts`
+- `comms/src/commsService.ts` — `retryMessage`
+- `admin-frontend/src/comms/*` (+ tests)
+- `example-frontend/app/admin/comms/*` — routes; `store/openApiSdk.ts` via `bun run sdk`
+- `admin-spa` — nav entry
+- `docs/reference/comms.md`, `docs/reference/admin.md`
+
+## Task List
+
+See [docs/tasks/comms-admin-dashboard.md](../tasks/comms-admin-dashboard.md).
+
+## Acceptance Criteria
+
+- [ ] An admin can filter delivery logs by channel, provider, status, error class, error
+      code, user, recipient, template, date range, and free text — server-side, paginated.
+- [ ] The detail view shows every attempt with timestamp, provider message id, error code
+      and message, plus the raw provider metadata and retained payload in a JSON viewer.
+- [ ] Retry re-sends through the configured provider, creates a linked `CommsMessage` row,
+      fires the normal lifecycle hooks with `isRetry: true`, and records the acting admin.
+- [ ] Non-retryable messages (permanent failure, expired payload, unconfigured channel,
+      verification channel) show a disabled retry with the specific reason; the route
+      returns 400 with the matching stable error `code`.
+- [ ] `retryMany` re-drives at most the cap per call and reports skipped ids with reasons.
+- [ ] `/comms/stats` counts match the filtered list totals for the same range.
+- [ ] All routes are `IsAdmin`; non-admin requests get 403 (supertest coverage).
+- [ ] Screens verified in the running example app with screenshots attached to the PR.