@terreno/mcp 58.0.0-beta.3 → 58.0.0

package/src/docs/versioned/57.22.0/how-to/observe-llm-calls.md

@@ -0,0 +1,345 @@
+# Observe LLM calls
+
+Register observability on the backend. Registration **is** the feature flag. Then run product iteration in [Develop an AI feature](ai-feature-development.md).
+
+## Register
+
+```typescript
+import {access} from "./access";
+
+const priceMap = parseObservabilityPriceMap(process.env.AI_OBS_PRICE_MAP_JSON);
+
+server.register(new ObservabilityApp({
+  accessControl: access,
+  aiService,
+  plugins: [localPlugin, otelSink].filter(Boolean),
+  control: {
+    datasets: process.env.AI_OBS_DATASETS_PRIMARY ?? "local",
+    experiments: process.env.AI_OBS_EXPERIMENTS_PRIMARY ?? "local",
+    prompts: process.env.AI_OBS_PROMPTS_PRIMARY ?? "local",
+    reviewQueue: "local",
+  },
+  priceMap,
+  sampleRate: Number(process.env.AI_OBS_SAMPLE_RATE ?? 0),
+}));
+```
+
+For an admin playground that accepts a user-supplied provider key, add
+`requestAiServiceFactory: ({apiKey, modelId}) => ...`. The playground route passes
+`x-ai-api-key` to that factory only when the server-wide `aiService` is absent. Never persist or
+log the key. `GET /ai/observability/status` exposes `playgroundAi.source` so admin UIs can block
+**Run once** with a host-specific hint (`request-key`) instead of surfacing the backend **503**
+missing-key title as a server misconfiguration.
+
+Validate the parsed object before registration: each model needs non-negative numeric
+`inputPerMTok` and `outputPerMTok`. The example implementation is
+`example-backend/src/utils/observabilityConfig.ts`; malformed values fail startup with the
+variable name.
+
+`local` is the only control primary. Boot **fails** if `experiments.primary !== datasets.primary`, if `reviewQueue` is not `local`, or if a primary’s plugin is missing.
+
+Pass the same `access` object you use for `TerrenoApp` and `AdminApp` so
+`GET /ai/observability/status` permissions match `/admin/config` screen filtering and every
+observability route enforces RBAC.
+
+### Example consumer roles
+
+The example backend seeds two optional roles in `example-backend/src/rbacRoles.ts`. They are
+**not** assigned automatically — attach them in the admin **Roles** UI or on `User.roles`.
+
+| Role | Grants |
+| --- | --- |
+| `aiObservabilityViewer` | `admin:access` plus `list` and `read` on all six observability resources |
+| `aiObservabilityOperator` | `admin:access` plus every approved action on those resources |
+
+Compose your own roles the same way: grant `admin: ["access"]` for the admin shell, then add
+only the `aiPrompt` / `aiTrace` / `aiReview` / `aiDataset` / `aiEvaluator` / `aiExperiment`
+actions each persona needs. Prompt `folder` and `description` stay consumer-defined metadata —
+Terreno does not ship domain-specific folder names or an in-app approval workflow.
+
+## Env
+
+| Variable | Role |
+| --- | --- |
+| `AI_OBS_PROMPTS_PRIMARY` | default `local` (the only primary) |
+| `AI_OBS_DATASETS_PRIMARY` | default `local` |
+| `AI_OBS_EXPERIMENTS_PRIMARY` | must equal datasets |
+| `AI_OBS_SAMPLE_RATE` | `0`–`1`, default `0` (live evals off) |
+| `AI_OBS_PRICE_MAP_JSON` | `{ "gemini-2.5-flash": { "inputPerMTok": 0.1, "outputPerMTok": 0.4 } }` |
+| `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` | optional OTLP trace sink |
+
+## App generate calls
+
+Pass identity and the production prompt ref so traces feed the SOP loop:
+
+```typescript
+await aiService.generateText({
+  prompt: text,
+  promptLabel: "production",
+  promptName: "example-summarize",
+  sessionId: req.header("x-ai-session-id"),
+  userId: req.user?.id,
+});
+```
+
+Opt out of tracing with `skipTrace: true`. Sink failures are logged and never fail the generate. `AIRequest` is still written.
+
+## Admin
+
+Phase 1 admin ships **Prompts**, **Traces**, and **Review**. Datasets, Evaluators, and
+Experiments screens arrive in phase 2. Review is hidden when the local plugin is off.
+
+On **Traces**, the header **Run a test trace** action (local sink only) runs two schema-backed
+`generateJsonObject` calls, a deterministic tool stage, and a final schema-backed combine
+call, then opens the nested trace. Each LLM span input includes `outputSchema`. It uses the
+server `aiService` when one is configured; otherwise it sends the admin's saved key as
+`x-ai-api-key` to `requestAiServiceFactory`, and the button stays disabled with a hint until
+that key exists.
+
+## Run the example locally
+
+1. Set `MONGO_URI` to a replica set and the auth secrets from
+   `example-backend/.env.example`.
+2. Run `bun run backend:seed`. The idempotent seed creates `examples/example-summarize`
+   v1 with `production` pointing to v1 and candidate v2, plus the human
+   `correctness-human` evaluator, automatic `schema-assert` evaluator, and
+   two-item `example-gold` dataset. It also creates `chat-safety-synthetic` (12 proofread
+   two-person chats), `chat-safety-screen` v1 and v2, and the `chat-safety-agreement`
+   evaluator. On a database that already booted, run Admin → Scripts →
+   `seedChatSafetyDataset` as a wet run, or
+   `cd example-backend && bun run script seedChatSafetyDataset --wet`.
+
+   The same seed also fills every observability screen, so you do not have to build data
+   by hand:
+
+   | Screen | Seeded data |
+   | --- | --- |
+   | Prompts | `example-summarize` v3 (staging, output capped at 120 characters); `joke-generator` and `chat-assistant` v1 (production) and v2 (staging); judge prompts `eval-judge-correctness` and `eval-judge-helpfulness`; `eval-judge-summary-questions` v1 (short paraphrased wording) and v2 (careful wording, production) |
+   | Evaluators | `correctness-human`, `helpfulness-human`, `schema-assert`, the `correctness` and `helpfulness` LLM judges, and the review-queue question sets `output-quality-questions` and `helpfulness-questions` (three yes/no questions each) |
+   | Datasets | `example-summarize-gold` (8 proofread rows, 2 unproofread), `joke-generator-topics` (6), `chat-assistant-gold` (6), and `summary-review-set` (the 10 newest seeded summaries, every item answered by humans with `summary-review-questions`) |
+   | Experiments | `Summarize v1 vs v2 (gold)`: v1 fails correctness, v2 passes. `Summarize v2 vs v3 (length cap)`: v3 fails `schema-assert` on 2 rows. `Jokes v1 vs v2 (one-liners)` and `Chat assistant v1 vs v2 (brevity)`: v1 fails helpfulness, v2 passes. `Summary judge v1 vs v2 (calibration)`: a judge calibration experiment on `summary-review-set`; v1 agrees with humans 77.5% (fails the 90% gate), v2 agrees 95% (passes) |
+   | Traces | 18 Todos → Summarize traces over the past week (2 errors, 1 sensitive, 2 flagged for the dataset), 6 `joke-generator` traces, and 6 `chat-assistant` turns (4 with a tool span); user-feedback, LLM-judge, and code scores |
+   | Review | 34 items: 11 pending, 1 in progress, 21 done, 1 skipped, including the batches `Recent summaries — batch 1` (8 of 10 answered) and `summary-review-set — human answers` (10 of 10) |
+
+   Seeded experiments run through the real experiment runner with a deterministic
+   `seed-fixture` client, so they need no API key. Seeded traces use session IDs that
+   start with `seed-obs-`. Production stays on v1 for every product prompt, so promoting v2 from
+   an experiment changes what the app sends.
+
+   **New experiment** (pipeline **Compare prompt versions**) keeps **Continue** disabled and
+   shows "Select a dataset." until you choose a dataset; the field reads "Choose a dataset"
+   until then. **Experiment results** shows one verdict banner, then one **Gates** table: a
+   row per gate (evaluator and dimension in plain words), **Needs** with the threshold as a
+   symbol (for example "≥ 0.8"), and one column per version with the measured value and Pass
+   or Fail. The last **Result** row says whether each version passes (for example "Fails 1 of
+   3 gates"). When no version passes, the screen explains why, for example "v1 fails 1 gate:
+   correctness: correct ≥ 0.8 (got 0.75) — can't promote v1.", and the header **Promote vN to
+   production** stays disabled.
+
+   On a database that already booted, run
+   Admin → Scripts → `seedAiObservabilityDemo`, or
+   `cd example-backend && bun run script seedAiObservabilityDemo --wet`.
+3. Start `bun run backend:dev` and `bun run frontend:web`, then sign in as the seeded
+   admin.
+4. Open **AI Observability → Prompts** to inspect or save a new immutable version. Move
+   `production` explicitly; the seed never overwrites an existing label.
+5. Configure `GOOGLE_VERTEX_PROJECT` or `GEMINI_API_KEY`, then sign in with Better Auth
+   and call the seeded route:
+
+   ```bash
+   API=http://localhost:4000
+   COOKIE_JAR="$(mktemp)"
+   curl -sS -c "$COOKIE_JAR" \
+     -H "Content-Type: application/json" \
+     -d '{"email":"admin@example.com","password":"testpassword123"}' \
+     "$API/api/auth/sign-in/email" >/dev/null
+
+   curl -X POST "$API/ai/example-summarize" \
+     -b "$COOKIE_JAR" \
+     -H "Content-Type: application/json" \
+     -H "x-ai-session-id: docs-walkthrough" \
+     -d '{"text":"Terreno records local nested traces for every AI call."}'
+   ```
+
+6. Open its `example-summarize` trace, send it to Review, score **correct**, and submit
+   until the queue reports **Queue clear**.
+
+The example chat reads two more registry prompts. `addGptRoutes` takes
+`systemPromptName: "chat-assistant"`, so each chat turn prepends that prompt's `production`
+version to the system prompt and its trace records the version. The chat's `generate_joke`
+tool calls `generateText` with `promptName: "joke-generator"` and falls back to a built-in
+prompt when the registry has no `production` version. Move either label in **Prompts** and
+the next chat message uses the new version.
+
+The example app also calls this route from the UI: **Todos → Summarize** sends the user's todo
+list, so each press produces a trace with the production prompt reference, `userId`, and a
+per-app-run `x-ai-session-id`. Without server credentials the screen forwards the Gemini key
+saved in **Profile** as `x-ai-api-key`; otherwise the route answers **503**.
+
+The route sends `text` as the user prompt. The labelled registry version supplies the
+system prompt and the route uses `TemperaturePresets.LOW`.
+
+The example always registers the local plugin. Set
+`AI_OBS_PRICE_MAP_JSON` to calculate USD cost; unlisted models show tokens with no cost,
+never `$0`.
+
+## Create a prompt and pin production
+
+Admin-only routes live under `/ai/observability`. Create the prompt in a folder, save later edits as `vN+1` (v1 stays immutable), then move `production` — the response names the outgoing version for the confirm modal.
+
+```bash
+# 1. Create v1 in a folder
+curl -X POST "$API/ai/observability/prompts" \
+  -H "Authorization: Bearer $ADMIN_TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{"folder":"examples","name":"example-summarize","system":"Summarize {{text}}","template":"{{text}}","type":"text"}'
+
+# 2. Pin production (first pin has no outgoing version)
+curl -X POST "$API/ai/observability/prompts/example-summarize/labels" \
+  -H "Authorization: Bearer $ADMIN_TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{"label":"production","version":1}'
+
+# 3. Save as v2, then move production (response.outgoingVersion is 1)
+curl -X POST "$API/ai/observability/prompts/example-summarize/versions" \
+  -H "Authorization: Bearer $ADMIN_TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{"system":"Summarize {{text}} in one sentence","template":"{{text}}","type":"text"}'
+curl -X POST "$API/ai/observability/prompts/example-summarize/labels" \
+  -H "Authorization: Bearer $ADMIN_TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{"label":"production","version":2}'
+```
+
+`GET /ai/observability/prompts?folder=examples&search=sum&include=usage7d` lists folder matches with 7-day call/cost rollups. A prompt with no `production` label returns `production: "—"`. Playground `POST /ai/observability/prompts/:name/playground` compiles `{{var}}`, runs one `AIService` call, and does not create a version.
+
+The example backend registers `requestAiServiceFactory` but no server `aiService`, so status reports
+`playgroundAi.source: "request-key"`. In **AI Observability → Prompts → Playground**, save a Gemini
+API key on **Profile** first; the admin UI blocks **Run once** with that hint until the key loads
+from storage and is sent as `x-ai-api-key`. **Traces → Run a test trace** uses the same
+saved key and is blocked the same way.
+
+## Share instructions across prompts
+
+Put the instructions that a set of prompts repeats in one prompt, then reference it from each
+prompt with `{{> name}}`.
+
+1. Create the shared prompt and pin `production`:
+
+   ```bash
+   curl -X POST "$API/ai/observability/prompts" \
+     -H "Authorization: Bearer $ADMIN_TOKEN" -H "Content-Type: application/json" \
+     -d '{"folder":"shared","name":"base-rules","template":"Answer in plain language for {{audience}}.","type":"text"}'
+   curl -X POST "$API/ai/observability/prompts/base-rules/labels" \
+     -H "Authorization: Bearer $ADMIN_TOKEN" -H "Content-Type: application/json" \
+     -d '{"label":"production","version":1}'
+   ```
+
+2. Reference it from each prompt's `system` or `template`. In the prompt editor, open the version and choose **Insert prompt reference**. That opens a modal where you pick the shared prompt. That appends:
+
+   ```text
+   {{> base-rules}}
+   Triage the intake form.
+   ```
+
+   - `{{> base-rules}}` follows the `production` label.
+   - `{{> base-rules label=staging}}` follows another label.
+   - `{{> base-rules version=3}}` pins one version. Use `label=` or `version=`, not both.
+   - A shared prompt can reference another one, up to 5 levels deep.
+
+3. Promote a new `base-rules` version. Every prompt that references it by label uses the new text
+   at once, with no new version of its own.
+
+Every reference must resolve when you save, so a saved prompt never fails at call time. Creating a
+prompt, saving a version, and moving a label return 400:
+
+- `Invalid prompt reference` when a tag is malformed, names an unknown prompt or version, or names
+  a label the shared prompt does not have yet. Promote the shared prompt before you reference it.
+- `Prompt reference cycle` when the save or label move would make prompts include each other. A
+  new version counts as `latest` for this check.
+- `Prompt references nested too deeply` when the change would nest references more than 5 levels,
+  in the saved prompt or in any prompt that includes it.
+
+The prompt editor keeps the tag; `PromptRegistry.get`, the playground, experiments, and judges use
+the resolved text. `{{variables}}` in a shared prompt are filled like the prompt's own.
+
+The trace stays named after the prompt you called. Its `prompts` list holds that prompt first,
+then one entry per shared prompt with `includedBy` set to the prompt whose tag pulled it in:
+
+```json
+[
+  {"label": "production", "name": "triage-intake", "version": 3},
+  {"includedBy": "triage-intake", "label": "production", "name": "base-rules", "version": 1}
+]
+```
+
+So `?prompt=base-rules` on the trace list finds every call that used the shared text, and
+`?prompt=triage-intake` still finds only that prompt's calls.
+
+## Review a trace
+
+1. Use the seeded human `correctness-human` evaluator. If you skipped `bun run backend:seed`,
+   install it once with `POST /ai/observability/evaluators/templates/correctness-human`; a
+   duplicate install returns 409.
+2. Enqueue traces: `POST /ai/observability/traces/review` with `{evaluatorId, traceIds, reason: "manual"}`.
+3. List oldest-first: `GET /ai/observability/review?status=pending` (response includes per-status counts).
+4. Open an item: `GET /ai/observability/review/:id` returns evaluator dimensions plus `given` / `wrote` panels (variable labels when present, otherwise raw keys).
+5. `POST /ai/observability/review/:id` with `action: "submit"` writes dimension-keyed scores through every ScoreSink and marks the item `done` (it leaves the pending list). `skip` and `assign` move status to `skipped` / `in_progress`.
+
+The seeded queue uses `output-quality-questions` and `helpfulness-questions`: three required yes/no questions on one screen. Re-running `bun run backend:seed` updates those question sets in place and points any queue item still using `correctness-human` or `helpfulness-human` at the matching three-question set. On the item screen, **What the AI was given** and **What the AI wrote** are headings above equal columns. Each column's card holds the field labels (for example source text) and the values. The output card uses the `ai` surface. Segmented Yes / No (and a numeric or categorical control) sit in a fixed left column. On web, `Y` / `N` answer the highlighted question as soon as the item opens, including when a Yes/No segment is focused, `Enter` submits, and `S` skips.
+
+## Check that an LLM judge agrees with people
+
+Use a review batch: a fixed set of traces (one input and one output each) that reviewers answer
+the same list of questions for. Then run LLM judges over the same traces and compare.
+
+1. Create a **question set**: a `human` evaluator with one `boolean` dimension per question. Put
+   the reviewer-facing wording in each dimension's `question`, for example
+   `{key: "covers_every_item", dataType: "boolean", question: "Does it mention every todo?"}`.
+   The seed installs `summary-review-questions` with four questions.
+2. Open **AI Observability → Review batches** (or **Review batches** in the Review queue header),
+   pick the question set and (optionally) a prompt, and press **Save batch** on the New batch
+   card. That grabs the 10 newest successful traces. API: `POST /ai/observability/review/batches` with
+   `{evaluatorId, promptName?, limit?: 10, name?}`. It takes the newest successful traces with an
+   output that are not already in a batch for that question set, so pressing it again gives the
+   next 10. Pass `traceIds` instead to pick traces by hand.
+3. Reviewers press **Answer N remaining**. Each item shows **What the AI was given** and **What
+   the AI wrote** as headings above equal columns, and every question as a segmented **Yes** /
+   **No** in a fixed left column. On web, `Y` / `N` answer the highlighted question as soon as the
+   item opens, including when a Yes/No segment is focused, `Enter` submits and
+   opens the next item in the batch (**Submit and open next**), and `S` skips. Items someone
+   assigned to themselves (in progress) stay in the batch's next/previous walk until they are
+   submitted or skipped.
+4. Create an `llm-judge` evaluator whose dimension keys match the question keys and whose judge
+   prompt returns those keys. On the batch, choose the judge and press **Run on N items**
+   (`POST /ai/observability/review/batches/:id/judgeRuns` with `{evaluatorId}`). The judge runs
+   on every batch trace through `ObservabilityApp.aiService` (or the caller's `x-ai-api-key`).
+5. Read **Which judge matches people?**: overall and per-question agreement for each judge,
+   counted only on items people answered and questions the judge also answers. 90% or more reads
+   **Matches**, 75–89% **Close**, and under 75% **Off**. Click a judge to list where it disagrees, then open the item to see why.
+   Re-running a judge after editing its prompt replaces that judge's previous result.
+
+The seed adds the batch `Recent summaries — batch 1` (8 of 10 answered) with two judges run:
+`summary-questions-judge` agrees 97% and `summary-questions-judge-quick` agrees 78%.
+
+## Calibrate a judge prompt against a human-judged dataset
+
+A batch answers "does this judge match people on these traces?". To improve a judge prompt
+version by version, judge a dataset once and reuse those answers:
+
+1. Build a dataset of traces (**Add to dataset** on traces, or
+   `POST /ai/observability/traces/add-to-dataset`).
+2. Create a review batch from it: `POST /ai/observability/review/batches` with
+   `{evaluatorId: <question set>, datasetId}`. Only items without a current answer are queued.
+   Every submitted answer is saved on its dataset item, and the dataset's `judgments` show
+   answered, current, and stale counts per question set.
+3. Save 2–3 versions of the judge prompt, then create a judge experiment:
+   `POST /ai/observability/experiments` with
+   `{kind: "judge", name, datasetId, questionSetId, judgeEvaluatorId, versions: [1, 2], targetAgreement: 0.9}`.
+   Each version answers every human-answered item; `results.agreement` ranks the versions and
+   each version gets an `agreement` gate.
+4. Promote the version that passes (`POST /ai/observability/experiments/:id/promote`); the
+   judge evaluator then shows its `calibration`. Use that judge in prompt experiments.
+
+Rewording a question makes earlier answers stale: re-run step 2 to re-queue those items.
+

package/src/docs/versioned/57.22.0/how-to/password-reset.md

@@ -0,0 +1,82 @@
+# Password reset and email verification
+
+Wire JWT recovery routes and Better Auth to `@terreno/comms` mail, then add a reset screen.
+
+## 1. User schema
+
+1. Apply `emailVerificationPlugin` so `emailVerified` defaults to false.
+2. Add `tokenEpoch` (number, default 0) so password reset can invalidate refresh tokens.
+
+```typescript
+import {emailVerificationPlugin} from "@terreno/api";
+
+userSchema.plugin(emailVerificationPlugin);
+```
+
+## 2. JWT `authOptions`
+
+Set `publicAppUrl` and `sendMail`. Forgot-password always returns 202.
+
+```typescript
+authOptions: {
+  publicAppUrl: process.env.FRONTEND_URL || "http://localhost:8082",
+  // Set false if the host already owns POST /resetPassword.
+  legacyResetPasswordRoute: false,
+  sendMail: async (message) => {
+    await getCommsService().sendMail(message);
+  },
+  requireEmailVerification: false,
+}
+```
+
+Pass the `message` through unchanged. Recovery mail carries `sensitive: true`, and `@terreno/comms` stores no payload for a sensitive message, so the reset or verify link never reaches the `CommsMessage` log or the admin Comms Dashboard. If you send through another transport, do not log or store the content of a message with `sensitive: true`.
+
+| Action | Path | Body |
+| --- | --- | --- |
+| Forgot | `POST /auth/forgotPassword` | `{email}` → 202 |
+| Reset | `POST /auth/resetPassword` or `POST /resetPassword` | `{token, password}` or `{token, newPassword}` |
+| Send verify | `POST /auth/sendVerification` | authenticated → 202 after delivery; 501 if `publicAppUrl` is missing |
+| Verify | `POST /auth/verifyEmail` | `{token}` |
+
+Recovery routes default on for compatibility. Set `authOptions.passwordReset: false` to omit
+both password-reset routes and forgot-password. Set `authOptions.emailVerification: false` to
+omit both verification routes. Set `authOptions.legacyResetPasswordRoute: false` when the host
+owns `POST /resetPassword`; `POST /auth/resetPassword` remains registered.
+
+Reset links: `${publicAppUrl}/resetPassword?token=...`. Verify links: `${publicAppUrl}/verifyEmail?token=...`.
+
+Set `requireEmailVerification: true` to return 403 `email-not-verified` on JWT login until verify succeeds. Signup still returns JWTs and sends the verify mail.
+
+## 3. Comms templates
+
+```typescript
+import {renderAuthMail} from "@terreno/comms";
+
+const mail = renderAuthMail({
+  publicAppUrl: "https://app.example.com",
+  templateId: "resetPassword",
+  token,
+});
+```
+
+Pass `templates` to override subject/text/html. Variables: `resetUrl`, `verifyUrl`, `publicAppUrl`, `token`.
+
+## 4. Better Auth
+
+On `BetterAuthConfig` set the same `publicAppUrl`, `sendMail`, and `renderAuthMail` from `@terreno/comms`. That wires `sendResetPassword` and `sendVerificationEmail` to those templates. Those hooks do not send (they throw 501) when `publicAppUrl` is empty.
+
+## 5. Frontend
+
+1. Pass `onForgotPassword` (or `onForgotPasswordPress`) to `LoginScreen`.
+2. Add `/forgotPassword` (email form) and `/resetPassword` that reads `token` from the query string.
+3. On Better Auth apps, request reset with `authClient.requestPasswordReset` and submit with `authClient.resetPassword({newPassword, token})`. JWT apps use `POST /auth/forgotPassword` and the RTK `resetPassword` mutation (`POST /resetPassword`). The example app falls back to the JWT request or reset route when Better Auth rejects the operation, so either auth mode can complete.
+4. Show a profile banner when `emailVerified` is false, with Resend calling `POST /auth/sendVerification`.
+5. Add `/verifyEmail` that reads `token` from the query string. JWT apps call `POST /auth/verifyEmail`. Better Auth apps try `GET /verify-email?token=...` first, then the JWT route so either token type can complete.
+
+A new `issueFor` for the same user and type invalidates earlier unused tokens of that type. Changing the mailbox through `PATCH /auth/me` invalidates unused `passwordReset` tokens even when the User schema has no `emailVerified` field. When `emailVerificationPlugin` is applied, mailbox change also sets `emailVerified` to false and invalidates unused `emailVerification` tokens. Letter casing alone does not count as a mailbox change. `emailVerified` and `tokenEpoch` are privileged user fields: signup and `PATCH /auth/me` drop them, along with the password fields `hash` and `salt`.
+
+To set a password from your own server code (for example an admin action), call `changeUserPassword(user, password, {adminId, app: req.app})` from `@terreno/api` so the change revokes sessions the same way a reset does. The example backend's `POST /users/:id/password` uses it.
+
+Either reset path also ends the user's MCP credentials: `mcp_` service tokens and MCP OAuth grants issued before the reset stop working, because they are bound to `tokenEpoch`. With `CommsApp` registered, push also stops going to devices the user registered before the reset until each device registers again.
+
+Better Auth password reset revokes that user's Better Auth sessions (`revokeSessionsOnPasswordReset`) and, when an app User exists, updates the JWT password and `tokenEpoch` so `POST /auth/login` cannot keep the old passport hash. JWT `POST /auth/resetPassword` does the reverse for Better Auth when `BetterAuthApp` is registered: it updates the Better Auth password and deletes Better Auth sessions. With `BetterAuthApp` registered, the JWT routes only replace an existing JWT password, because `POST /auth/login` has no second factor: `POST /auth/forgotPassword` mails no link to an app User without one (the response is still 202), and `POST /auth/resetPassword` refuses that User's token with 400 `legacy-password-reset-unavailable`, setting no password. That account (for example a social-only one) resets through Better Auth. Without Better Auth, the JWT routes still let any user set a password. Console mail in development prints length only; check `CommsMessage` / console adapter logs for delivery, not the raw token in logger output. Request logs redact password, token, secret, and code fields (any key name, any depth) in bodies and in URL query strings.

package/src/docs/versioned/57.22.0/how-to/recover-locked-out-user.md

@@ -0,0 +1,112 @@
+# Recover a locked-out user
+
+Help a user who lost their authenticator, passkey, or social account get back in. Platform
+staff use `/users/:id/security`; org-admins use `/orgs/:orgId/members/:userId/security`. Both
+run the same handlers, and every change writes a security `AuditEvent`.
+
+## Before you start
+
+1. Register `AuditApp` and `AccountSecurityApp` on the `TerrenoApp`:
+
+   ```typescript
+   import {AccountSecurityApp, AuditApp, BetterAuthApp, TerrenoApp} from "@terreno/api";
+
+   new TerrenoApp({accessControl: access, organizations: true, userModel: User})
+     .register(new BetterAuthApp({config: betterAuthConfig}))
+     .register(new AuditApp())
+     .register(new AccountSecurityApp())
+     .start();
+   ```
+
+2. For `disable` and `enable`, apply `isDisabledPlugin` to the User schema.
+3. For `send-password-reset`, set `sendMail` and `publicAppUrl` in `BetterAuthConfig`.
+4. For the org path, turn organizations on (`organizations: true`). `TerrenoApp` hands the
+   registered `AccountSecurityApp` to `OrgsApp`, which mounts the org routes.
+
+## Who can do what
+
+| Action | Route suffix | Platform staff (`user:manageSecurity`) | Org-admin of the user's org |
+| --- | --- | --- | --- |
+| Read the summary | `GET /security` | Yes | Yes |
+| Reset two-factor | `POST /security/reset-two-factor` | Yes | Yes |
+| Remove a passkey | `DELETE /security/passkeys/:passkeyId` | Yes | Yes |
+| Unlink a provider | `DELETE /security/accounts/:providerId` | Yes | Yes |
+| Sign out everywhere | `POST /security/revoke-sessions` | Yes | Yes |
+| Send a password reset | `POST /security/send-password-reset` | Yes | Yes |
+| Disable or enable the account | `POST /security/disable`, `/enable` | Yes | No |
+| Suspend the membership | `POST /security/suspend-membership` | Through the org path | Yes |
+| Issue a sign-in link | `POST /security/issue-sign-in-link` | `superadmin` only (`user:issueSignInLink`) | No |
+
+`superadmin` and `operator` hold `user:manageSecurity`. An org-admin is a user whose active
+`Membership` in that org has `roleName: "org-admin"`.
+
+Nobody can act on themselves, or on someone who holds a permission they lack. An org-admin can't
+reach platform staff or `admin: true` users, members of other orgs, or suspended members. Platform
+staff on the org path are judged the same way as on the platform path, so an org-only permission
+does not block them.
+
+## Platform path
+
+Prefix every route with `/users/<userId>`.
+
+1. Read the summary to see the user's sign-in methods and the actions that apply:
+
+   ```bash
+   curl -H "Authorization: Bearer $TOKEN" https://api.example.com/users/<userId>/security
+   ```
+
+   `data.actions` lists only the actions that apply to this user and that you may run.
+2. Run the action that fixes the problem:
+   - Lost authenticator: `POST .../reset-two-factor`. The user signs in with their password and
+     can enroll again.
+   - Lost device with a passkey: `DELETE .../passkeys/<passkeyId>` (ids are in the summary).
+   - Lost social account: `DELETE .../accounts/<providerId>`.
+   - Stolen device: `POST .../revoke-sessions`.
+   - Forgotten password: `POST .../send-password-reset`.
+   - Compromised account: `POST .../disable`. Later, `POST .../enable`.
+   - Legacy user with no password and no inbox: `POST .../issue-sign-in-link`, then give the
+     user the returned link yourself. Only a superadmin can do this.
+3. Removing the user's last sign-in method answers `409` unless their email is verified. With a
+   verified email, the route also mails a password reset.
+
+## Org path
+
+Prefix every route with `/orgs/<orgId>/members/<userId>`. The user must be an active member of
+`<orgId>`. They may belong to other orgs too.
+
+1. Read the summary: `GET /orgs/<orgId>/members/<userId>/security`.
+2. Run the same recovery actions as the platform path (2FA, passkey, provider, sessions, reset).
+3. To cut the user off from your org, run `POST .../suspend-membership`. It suspends only their
+   membership in `<orgId>`. They still sign in and keep their other orgs.
+4. To bring a suspended member back, use the membership route:
+   `PATCH /orgs/<orgId>/members/<memberId>` with `{"status": "active"}`. A suspended member is out
+   of reach of the security routes until then.
+
+To stop an org-admin from attaching an existing user by email and acting on them right away, set
+a minimum membership age:
+
+```typescript
+new TerrenoApp({
+  accessControl: access,
+  organizations: {orgRecovery: {minMembershipDays: 7}},
+  userModel: User,
+});
+```
+
+## Check the audit trail
+
+Every change is one `AuditEvent` with `source: "security"`. Org actions carry the org's id in
+`organizationId`. When `sendMail` is configured, each change also emails the user (except
+`send-password-reset`, whose reset email is the notice). For `issue-sign-in-link` the email says a
+link was created, never the link itself. The email says "an administrator" and never names you;
+see
+[Notifications](../reference/account-security.md#notifications).
+
+```bash
+curl -H "Authorization: Bearer $TOKEN" \
+  "https://api.example.com/audit-events?source=security&recordId=<userId>"
+```
+
+Related: [Account security reference](../reference/account-security.md),
+[How organization tenancy works](../explanation/organizations.md#member-recovery),
+[Enable the framework audit log](audit-log.md).

package/src/docs/versioned/57.22.0/how-to/release-automation.md

@@ -0,0 +1,83 @@
+# How to let agents merge release notes and roadmap updates
+
+An agent (or the scheduled roadmap workflow) opens a pull request. The **Bot merge** workflow
+([`.github/workflows/bot-merge.yml`](https://github.com/TerrenoLabs/terreno/blob/master/.github/workflows/bot-merge.yml))
+merges it after CI passes. No human review is needed. For a release it then tags the merged commit,
+and that tag starts CircleCI `publish-release`. The packages are built from the tagged commit, so the
+release notes are already in each npm tarball.
+
+```text
+release/X.Y.Z PR ──► Bot merge waits for CI ──► squash-merge (bot bypass)
+                                                   │
+                                                   ▼
+                     tag X.Y.Z on the squash commit + draft GitHub release
+                                                   │
+                                                   ▼
+            CircleCI publish-release: require merged notes ──► npm publish
+                                                    (every tarball ships CHANGELOG.md)
+```
+
+## What the bot merges
+
+[`scripts/ci/release/lib.ts`](https://github.com/TerrenoLabs/terreno/blob/master/scripts/ci/release/lib.ts)
+(`planBotMerge`) decides eligibility. A pull request qualifies only if all of these are true:
+
+- It comes from a branch in this repository. Drafts and forks are skipped.
+- Its author is listed in the `BOT_MERGE_AUTHORS` repository variable. Logins are compared without regard to case.
+- Its branch name and changed files match one of the rows below.
+
+| Branch | Files it may change |
+|--------|---------------------|
+| `release/X.Y.Z` or `cursor/release-X.Y.Z` (stable versions only) | `CHANGELOG.md`, `changelog/unreleased/*`, `example-frontend/constants/whatsNew.generated.json`, `mcp-server/src/docs/upgrades/*.md` |
+| `automation/roadmap-*` | `ROADMAP.md`, `docs/explanation/roadmap-seed-issues.md` |
+
+A release branch also needs a non-empty `## [X.Y.Z]` section in its `CHANGELOG.md`. The workflow
+uses `pull_request_target`, so both the workflow file and the eligibility script come from
+`master`. A pull request cannot change its own eligibility rules. The job never runs pull request
+code; it reads file names and `CHANGELOG.md` through the API. Any pull request that fails a rule keeps a green **Bot merge** check and waits
+for a human. Examples are a release pull request that also edits a skill, or a prerelease branch.
+
+The `master` ruleset requires no status checks, so the workflow does that job itself. It waits
+until every other check on the head commit has passed, including GitHub Actions and CircleCI
+statuses. It merges with the head SHA pinned, so a push made after the checks finished blocks the
+merge.
+
+## The publish gate
+
+`publish-release` runs
+`bun run scripts/ci/release/cli.ts check-tag <tag> <on-master>` before it compiles anything. A
+stable tag fails if any of these is true:
+
+- The tagged commit is not on `master`.
+- `CHANGELOG.md` has no non-empty `## [X.Y.Z]` section.
+- Fragments are left in `changelog/unreleased/`.
+
+Prerelease tags (`-beta.N`) skip the gate; they are often cut from a feature branch.
+
+The manual fallback `.github/workflows/publish-on-tag.yml` runs the same `check-tag` gate. It also
+fails unless it was dispatched from the tag itself, because its publish jobs build the dispatched
+ref, not the tag by name. Use **Run workflow** with the tag selected as the ref.
+
+`scripts/ci/prepare-package-publish.mjs` copies the root `CHANGELOG.md` into each package before
+`npm publish`. Packages that declare a `files` whitelist list `CHANGELOG.md` in it.
+
+## One-time GitHub setup
+
+1. Create a GitHub App under **TerrenoLabs → Settings → Developer settings → GitHub Apps**. Turn
+   off webhooks. Give it these repository permissions:
+   - Contents: read and write
+   - Pull requests: read and write
+   - Metadata: read
+
+   Install it on `TerrenoLabs/terreno` only.
+2. Generate a private key. Add it as the repository secret `BOT_MERGE_PRIVATE_KEY`. Add the
+   App ID as the repository variable `BOT_MERGE_APP_ID`.
+3. Set the repository variable `BOT_MERGE_AUTHORS` to the logins allowed to open bot-merged pull
+   requests, separated by commas. Include the account that owns `ROADMAP_PROJECT_TOKEN`, which
+   opens the roadmap pull requests.
+4. Add the App as an **Always** bypass actor on both `master` rulesets (**Branch** and
+   **master protection**). Its token can then squash-merge without an approving review.
+
+Once these are set, the agent's job ends at opening the pull request. Steps 5b–5c of the
+[`release` skill](https://github.com/TerrenoLabs/terreno/blob/master/.rulesync/skills/release/SKILL.md)
+describe that hand-off.