@terreno/mcp 57.20.0 → 57.22.0

package/src/docs/versioned/next/explanation/ai-observability.md

@@ -1,26 +1,26 @@
 # AI observability
 
-Terreno ships Langfuse-like prompt versioning, nested traces, evaluators, datasets, experiments, and a local human review queue **inside `@terreno/ai`**, with operator UI in **`admin-frontend` only**.
+Terreno ships prompt versioning, nested traces, evaluators, datasets, experiments, and a local human review queue **inside `@terreno/ai`**, with operator UI in **`admin-frontend` only**.
 
-## Why not Langfuse-only
+## Why built in
 
-Apps must iterate prompts without a required vendor account. `LangfuseApp` stays a second plugin. Traces and scores **fan out** to every registered sink (local Mongo, Langfuse, OTLP/OpenInference). Prompts, datasets, experiments, and the review queue have **one writer** each (a primary). The review queue is **local-only**.
+Apps must iterate prompts without a required vendor account, so the prompt registry, traces, and review queue live in the app's own Mongo. Traces and scores **fan out** to every registered sink (local Mongo plus any extra `TraceSink` / `ScoreSink` plugin an app registers). Prompts, datasets, experiments, and the review queue have **one writer** each (a primary). The review queue is **local-only**.
 
 That split is the two planes:
 
 | Plane | Write | Admin read |
 | --- | --- | --- |
-| Telemetry (traces, scores) | All sinks, best-effort | Local if the local plugin is on, else Langfuse |
+| Telemetry (traces, scores) | All sinks, best-effort | Local |
 | Control (prompts, datasets, experiments, queue) | One primary per capability | That primary |
 
-`experiments.primary` must equal `datasets.primary`. Defaults are all `local`.
+`local` is the only control primary, and the default for all four. `experiments.primary` must equal `datasets.primary`.
 
 `ObservabilityApp` fails boot (constructor throws) when:
 
 | Error | Cause |
 | --- | --- |
 | `experiments.primary must equal datasets.primary` | Mixed dataset/experiment writers |
-| `reviewQueue.primary must be local` | `reviewQueue` set to `langfuse` |
+| `reviewQueue.primary must be local` | `reviewQueue` set to anything but `local` |
 | `<capability> primary "<id>" has no plugin` | No registered plugin with that `id`, capability, and store |
 
 Telemetry sinks still fan out even when a control primary is local-only.
@@ -28,8 +28,10 @@ Telemetry sinks still fan out even when a control primary is local-only.
 Admin chrome lives in `admin-frontend`. One sidebar group **AI Observability** holds Prompts,
 Traces, Evaluators, Datasets, Experiments, and Review (Review is omitted when the local plugin is
 off). Existing **AI Requests** stays under **Screens**. Every observability screen shows
-breadcrumbs `Admin / AI Observability / <Section> / <leaf>` and a status chip from
-`GET /ai/observability/status`. Prompt detail is a hub (Overview, Versions, Traces, Experiments);
+breadcrumbs (`Admin / AI Observability / <Section>` on lists, plus `/ <leaf>` on detail
+screens), a quiet status chip from `GET /ai/observability/status`, and one left-aligned page
+header with the screen's single primary action. Prompt detail is a hub (Versions, Traces, Experiments).
+Versions shows the description, folder, tags, and the version editor on one screen;
 versions are immutable in admin (`Save as vN+1`); apps never inline the string.
 
 ## Why this shape for product work
@@ -58,7 +60,7 @@ customized `admin` and `auditor` roles gain only missing observability actions o
 
 ## Prompt hub relationships (phase 4)
 
-Prompts may carry an optional domain-neutral `description` (folder semantics stay consumer-defined). `GET /ai/observability/prompts/:name` composes bounded recent **traces** and **experiments** for that prompt name so the admin hub does not scan full trace or experiment lists on the client. Trace evidence is version-aware: list filters accept `prompt` plus optional `promptVersion`, and relationship trace rows include the matching `promptVersion` from `ObsTrace.prompts[]`. The same optional positive integer `promptVersion` on prompt detail filters hub traces with `$elemMatch` on `prompts` (name and version). Omit it to keep the unfiltered prompt-name match. Invalid or non-positive versions return **400**. Experiment relationships filter server-side by `promptName` on `GET /ai/observability/experiments?promptName=…` (omit or pass an empty value for no filter). The experiment list endpoint returns summary rows without per-item hydration; use experiment detail for full item results. Reading relationships uses the same `aiPrompt:read` grant as prompt detail.
+Prompts may carry an optional domain-neutral `description` (folder semantics stay consumer-defined). `GET /ai/observability/prompts/:name` composes bounded recent **traces** and **experiments** for that prompt name so the admin hub does not scan full trace or experiment lists on the client. Trace evidence is version-aware: list filters accept `prompt` plus optional `promptVersion`, and relationship trace rows include the matching `promptVersion` from `ObsTrace.prompts[]`. The same optional positive integer `promptVersion` on prompt detail filters hub traces with `$elemMatch` on `prompts` (name and version). Omit it to keep the unfiltered prompt-name match. Invalid or non-positive versions return **400**. Experiment relationships filter server-side by `promptName` on `GET /ai/observability/experiments?promptName=…` (omit or pass an empty value for no filter). The experiment list endpoint returns summary rows without per-item hydration; use experiment detail for full item results. Reading relationships uses the same `aiPrompt:read` grant as prompt detail. Detail also returns `usedBy` (`items`, `limit`, `total`): other prompts whose saved text contains `{{> this name}}`, capped at 20 like the other relationships. The editor inserts that tag from the **Insert prompt reference** modal.
 
 ## Phase 1 reference loop
 
@@ -67,7 +69,7 @@ The example backend always registers the local plugin. Its idempotent seed creat
 `correctness-human` evaluator, the automatic `schema-assert` evaluator, and a two-item
 `example-gold` dataset. It also creates `chat-safety-synthetic`, a proofread set of
 two-person chats for routing, toxicity, privacy, and dismissive-reply experiments.
-This makes the review and experiment loops walkable without Langfuse:
+This makes the review and experiment loops walkable with no external service:
 resolve the production prompt → emit a trace → inspect spans and sensitive I/O → send the
 trace to Review → record a human score.
 
@@ -87,11 +89,35 @@ With the local plugin registered and primaries set to `local`, the control plane
 
 - **`llm-judge` and `json-assert` evaluators** — judges call `AIService.generateJsonObject` through a named registry prompt (`judgePromptName`); create/update rejects a judge when the prompt `outputSchema` omits a required dimension (the error names that key). `json-assert` supports path/constraint checks and a built-in mode that validates output against the prompt version `outputSchema`. Parse or generation failures record an error outcome instead of throwing to the caller. Custom creates score a full trace; generation span and dataset item targets are reserved. Seeded templates install `correctness` / `hallucination` / `helpfulness` / `toxicity` as `llm-judge`, `schema-assert` as `json-assert`, and `*-human` variants for the review queue.
 - **Human review evaluators** — sending traces to review requires a `human` evaluator. Its dimensions become the review item's score fields and its instructions tell the reviewer how to apply them. `llm-judge` and `json-assert` evaluators run automatically and are rejected from the human queue.
+- **Review batches and judge agreement** — a human evaluator with several yes/no dimensions (each with a reviewer `question`) acts as a question set. A batch fixes the traces people answer, so an operator can run any number of LLM judges over the same traces and see, per judge and per question, how often the judge gives the human answer. That number is what tells you a new judge can replace or extend human review; disagreements point at the prompt rule to fix.
 - **Datasets** — CRUD plus item provenance (`origin`, `proofread`, `sourceTraceId`, tags, outcome class). Import accepts **JSON** (bare objects or structured rows) and **CSV** (quoted fields; `input.*` / `expectedOutput.*` column prefixes). Rows validate against the bound prompt input schema when configured. Adding from a trace copies I/O; sensitive traces always land `proofread: false`. Deleting an item never mutates the trace.
-- **Experiments** — compare 2–3 prompt versions on a dataset with optional `modelOverride` (requires `aiServiceFactory` on `ObservabilityApp`), evaluator thresholds (defaulting to the SOP gates), **per-version gate tiles** (`gates[].version`), outlier/low-confidence item ids, and version-scoped promote (**409** when the selected version's gates fail). Unproofread items are excluded unless `includeUnproofread` is true. Local runs compile exact prompt versions and call `AIService.generateText` with compiled `prompt` + `systemPrompt` (no `promptName`/`promptLabel`). `ObservabilityApp` wires the local experiment runner from `aiService` at register time.
+- **Experiments** — compare 2–3 prompt versions on a dataset with optional `modelOverride` (requires `aiServiceFactory` on `ObservabilityApp`), evaluator thresholds (defaulting to the SOP gates), **per-version gate tiles** (`gates[].version`; the results screen groups them in one card per version and shows operators as ≥ ≤ > < =, never `gte`/`lte`), outlier/low-confidence item ids, and version-scoped promote (**409** when the selected version's gates fail). Unproofread items are excluded unless `includeUnproofread` is true. Local runs compile exact prompt versions and call `AIService.generateText` with compiled `prompt` + `systemPrompt` (no `promptName`/`promptLabel`). `ObservabilityApp` wires the local experiment runner from `aiService` at register time.
 
 Operator UI for datasets and experiments ships in tasks 2.7–2.9; routes and stores are live for API clients and tests today.
 
+## The eval pipeline: humans first, then calibrated judges
+
+Terreno's evaluation flow has four steps, and each step reuses the one before it:
+
+1. **Build a dataset** of traces (or hand-written items).
+2. **Humans answer a question set** — a `human` evaluator whose dimensions are usually yes/no
+   `question`s — for every dataset item, through review batches built from the dataset. Each
+   submitted answer is written back onto the dataset item (`humanAnswers`).
+3. **Calibrate the judge.** A judge calibration experiment (`kind: "judge"`) runs 2–3 versions of
+   an LLM judge's prompt over the human-answered items and ranks the versions by agreement with
+   the humans. Iterate on the judge prompt until a version clears the target (default 90%), then
+   promote it.
+4. **Compare product prompts** with ordinary prompt experiments, scored by a judge whose
+   calibration (latest agreement, and whether it is stale) shows on the evaluator.
+
+Human answers are expensive, so they are reused: a dataset judged once feeds any number of judge
+experiments. The rule that keeps reuse honest is the **question fingerprint** — a hash of each
+question's key, type, wording, and range. Answers stay *current* while the fingerprint matches;
+rewording a question makes them *stale*, the dataset shows them as stale, new review batches
+re-queue those items, and judge calibrations based on the old wording are flagged stale.
+Judges always see what reviewers saw (the source trace, else the item's input and expected
+output), so agreement measures the judge, not a difference in evidence.
+
 ## GPT tool spans and nested traces
 
 `/gpt/prompt` streams tool rounds to the client **and** records them in observability traces. Each provider `tool-call` / `tool-result` pair becomes a `TOOL` span (input args, cleaned output, timing, `parentSpanId` on the chain root). When tools run, the exported trace uses a `CHAIN` root wrapping those `TOOL` children instead of a lone `LLM` span. Plain `AIService.generate*` calls without tool children still log a single `LLM` root.

package/src/docs/versioned/next/explanation/durable-agent-harness.md

@@ -69,7 +69,7 @@ transaction:
 
 Either all three land or none do. This is why `Harness.open` refuses to start without a
 replica set (transactions need one) or without the local observability models. Planned
-Langfuse and OTel exports will stay best-effort, after the transaction.
+exports to other trace sinks will stay best-effort, after the transaction.
 
 ```
 createTask ──tx──> HarnessTask(pending) + ObsTrace + root CHAIN span

package/src/docs/versioned/next/explanation/loop-engineering.md

@@ -119,9 +119,9 @@ include:
 | --- | --- |
 | Backend/API/data | `terreno-backend-api`, `mongoose-schema-safety`, `backend-test-env`, `generate-sdk`, `terreno-data-fetching` |
 | UI/app | `terreno-ui`, `building-terreno-apps`, `verify-ui-changes` |
-| Testing | `terreno-test-setup`, `terreno-backend-tests`, `terreno-frontend-tests`, `terreno-mobile-tests` |
+| Testing | `terreno-test-setup`, `terreno-backend-tests`, `terreno-frontend-tests`, `terreno-mobile-tests`, `backfill-regression` |
 | AI/prompts | `ai-prompt-governance` — new features also follow [Develop an AI feature](../how-to/ai-feature-development.md) |
-| Docs/submission | `update-docs`, Brew, `fix-conflicts` |
+| Docs/submission | `update-docs`, `backfill-user-guide`, Brew, `fix-conflicts` |
 | GitHub issues | `create-github-issue`, `work-github-issues`, `implement-ready-for-dev` (Pick plan comment is the Roast contract; unattended Grow may also write an IP) |
 | Deployment/runtime | `deploy-gcp`, Expo deployment/workflow skills |
 

package/src/docs/versioned/next/explanation/planning-plugins.md

@@ -5,7 +5,7 @@ the lifecycle plugin knows *how to build one approved thing correctly*, and the
 plugin knows *what is worth building next and whether it worked*.
 
 ```text
-terreno-scan                            terreno (lifecycle)
+terreno-loop                            terreno (lifecycle)
 ------------                            -------------------
 Aim     charter a measurable goal
 Sweep   shard + map the repository
@@ -42,10 +42,26 @@ Five bounded transitions plus three outer loops. Canonical source:
 Full contract: [lifecycle plugin reference](../reference/lifecycle-plugin.md) and
 [loop engineering](loop-engineering.md).
 
-## terreno-scan (scan plugin)
-
-Five bounded transitions plus two outer loops. Canonical source: `plugins/terreno-scan/`;
-Claude Code consumes the generated `plugins/terreno-scan-claude/`. It depends on the
+Every IP also carries a testing plan: an `## E2E tests` table and a `## Regression
+cases` table, either of which may be `None — <reason>` but never missing. Grow plans the
+rows, Pick writes each spec in the task that delivers it, Roast runs them, Brew merges
+the manual cases into one living suite file per area under `docs/regression/`, and
+Taste keeps both current after review fixes. The point is that no bot decides alone
+whether a feature deserves an end-to-end test, and the checks a human repeats before a
+release live in a known file instead of in someone's head. See
+[Testing plan](../reference/lifecycle-plugin.md#testing-plan).
+
+Every IP also says which pages of the end-user guide it adds, updates, or retires. The
+guide is one plain-text page per feature, written in the words users see, so a chat agent
+can answer "how do I…" questions from it. Writing the page in the same task as the
+feature keeps it true; a guide written once at the end drifts from the app the first time
+a label changes. A repository with no end users opts out in its agent instructions. See
+[User guide](../reference/lifecycle-plugin.md#user-guide).
+
+## terreno-loop (scan plugin)
+
+Five bounded transitions plus two outer loops. Canonical source: `plugins/terreno-loop/`;
+Claude Code consumes the generated `plugins/terreno-loop-claude/`. It depends on the
 lifecycle plugin — Plot hands every slice to Grow.
 
 | Stage | Owns | PASS next |
@@ -78,6 +94,8 @@ Full contract: [scan plugin reference](../reference/scan-plugin.md).
 | Artifact | Owner | Committed |
 | --- | --- | --- |
 | IP and task list | Grow | yes |
+| Regression suites (`docs/regression/<area>.md`) | Brew, Taste | yes |
+| User guide (`docs/user-guide/<area>/<feature>.md`) | Pick, Taste | yes |
 | Execution state (`.terreno/pipeline/<slug>.json`) | Lifecycle outer loops | no |
 | Charter (`docs/scans/<goal>/charter.md`) | Aim | yes |
 | Findings report (`docs/scans/<goal>/round-<n>-findings.md`) | Sift | yes |
@@ -104,9 +122,9 @@ Three properties fall out of keeping discovery and delivery in separate plugins:
 
 | Host | Lifecycle | Scan |
 | --- | --- | --- |
-| Cursor | `terreno-planning`, `/terreno-1-grow` | `terreno-scan`, `/terreno-scan-1-aim` |
-| Codex | `terreno-planning`, `$terreno-1-grow` | `terreno-scan`, `$terreno-scan-1-aim` |
-| Claude Code | `terreno`, `/terreno:1-grow` | `terreno-scan`, `/terreno-scan:1-aim` |
+| Cursor | `terreno-planning`, `/terreno-1-grow` | `terreno-loop`, `/terreno-loop-1-aim` |
+| Codex | `terreno-planning`, `$terreno-1-grow` | `terreno-loop`, `$terreno-loop-1-aim` |
+| Claude Code | `terreno`, `/terreno:1-grow` | `terreno-loop`, `/terreno-loop:1-aim` |
 
 Install both from the marketplaces described in
 [install agent skills](../how-to/install-agent-skills.md). Claude Code resolves a plugin

package/src/docs/versioned/next/explanation/roadmap-seed-issues.md

@@ -1157,18 +1157,46 @@ Delete the **legacy RTK cache-patching realtime path** in Terreno 58. Collection
 
 ## ai-observability
 
-**Title:** `AI observability (Langfuse-light, pluggable)`
+**Title:** `AI observability (built in, pluggable)`
 
 **Labels:** `area:ai`, `type:feature`  
 **Project fields:** Area=`ai`, Target=`Future`, Impact=`Feature`, IP=`ai-observability`, Status=`Planned`
 
-Ship **Langfuse-like** prompt versioning, nested traces (user / session / cost), multidimensional evaluators (LLM-as-judge + structured I/O), datasets, experiments, and a **local human review queue** — **inside Terreno**, as **plugins on `@terreno/ai`**, with an operator UI in **`admin-frontend` only**.
+Ship built-in prompt versioning, nested traces (user / session / cost), multidimensional evaluators (LLM-as-judge + structured I/O), datasets, experiments, and a **local human review queue** — **inside Terreno**, as **plugins on `@terreno/ai`**, with an operator UI in **`admin-frontend` only**.
 
 - **Implementation plan:** [ai-observability.md](https://github.com/TerrenoLabs/terreno/blob/master/docs/implementationPlans/ai-observability.md)
 - **Tasks:** [ai-observability.md](https://github.com/TerrenoLabs/terreno/blob/master/docs/tasks/ai-observability.md)
 
 ---
 
+## ai-observability-production-loop
+
+**Title:** `AI observability in production: live evals, alerts, threads, CI suites, prompt optimizer`
+
+**Labels:** `area:ai`, `type:feature`  
+**Project fields:** Area=`ai`, Target=`Next`, Impact=`Feature`, IP=`ai-observability-production-loop`, Status=`Planned`
+
+Terreno's AI observability covers building an AI feature: versioned prompts, traces, evaluators, datasets, experiments, and human review. It stops at production. The evaluator's live-sampling setting is never run. There's no view of quality, cost, or latency over time, nothing alerts on a regression, and a chat conversation is a loose set of traces. Prompt changes can't be gated in CI.
+
+This work adds:
+
+- Online evaluation of sampled production traces.
+- Named trace groups (conversations, or any id the app chooses) with a timeline view.
+- A monitoring dashboard.
+- Threshold alerts, delivered as signed, metadata-only webhooks to Slack or any URL.
+- Stored eval suites that `terreno ai test <suite>` runs from CI with a scoped service token.
+- A prompt optimizer that proposes better prompt versions from failing dataset items.
+
+Everything stays local to `@terreno/ai`, with screens in `admin-frontend`.
+
+Out of scope: conversation-level evaluators, eval backfill and budgets, email alerts, custom dashboards, and other optimizer algorithms.
+
+- **Implementation plan:** [ai-observability-production-loop.md](https://github.com/TerrenoLabs/terreno/blob/master/docs/implementationPlans/ai-observability-production-loop.md)
+- **Tasks:** [ai-observability-production-loop.md](https://github.com/TerrenoLabs/terreno/blob/master/docs/tasks/ai-observability-production-loop.md)
+- **Depends on:** `ai-observability` (phases 1–4)
+
+---
+
 ## better-auth-strict-oauth-provider
 
 **Title:** `Better Auth sync omits unset oauthProvider`
@@ -1342,6 +1370,42 @@ One easy-to-use harness inside `@terreno/ai` that runs long-lived agentic work d
 
 ---
 
+## e2e-and-regression-testing
+
+**Title:** `E2E tests and manual regression cases in every plan`
+
+**Labels:** `area:dx`, `type:feature`  
+**Project fields:** Area=`dx`, Target=`Next`, Impact=`Feature`, IP=`e2e-and-regression-testing`, Status=`In review`
+
+Today an implementation plan never says which end-to-end tests a feature needs, so bots decide alone whether a slice deserves a Playwright spec, and the manual checks a person should repeat before a release live in nobody's file. Most packages are tested only in-process.
+
+Every IP will carry `## E2E tests` and `## Regression cases` sections (either may be `None — <reason>`, neither may be missing). Grow plans the rows, Pick writes each spec with its feature, Roast runs them, Brew merges manual cases into one living suite per area under `docs/regression/`, and Taste keeps both current after fixes. Regression stays minimal: only what a developer or QA must check by hand, such as push notifications, real email delivery, or how the app feels. Terreno itself gains a real-process smoke test and an e2e guide page for every package, and brewery mirrors the contract.
+
+Out of scope: running manual cases automatically, and coverage targets.
+
+- **Implementation plan:** [e2e-and-regression-testing.md](https://github.com/TerrenoLabs/terreno/blob/master/docs/implementationPlans/e2e-and-regression-testing.md)
+- **Pull requests:** [#1628](https://github.com/TerrenoLabs/terreno/pull/1628), brewery [#15](https://github.com/joshgachnang/brewery/pull/15)
+
+---
+
+## lifecycle-user-guide
+
+**Title:** `An end-user guide that grows with every feature`
+
+**Labels:** `area:dx`, `type:feature`  
+**Project fields:** Area=`dx`, Target=`Next`, Impact=`Feature`, IP=`lifecycle-user-guide`, Status=`Shaping`
+
+Chat agents cannot answer "how do I…" questions about an app today, because no text describes its features in the words users see, and a guide written once drifts from the app the first time a label changes.
+
+Every IP will gain a `## User guide` section, and the task that ships a feature writes its page: one plain-text file per feature under `docs/user-guide/<area>/<feature>.md`, with `audience`, `platforms`, and the `questions` users would ask in its frontmatter, then what it does, how to use it, and what it is meant for. Roast checks the steps against the shipped screens, and Brew gates on the page. A repository with no end users opts out with `User guide: none`. A `backfill-user-guide` skill drafts pages for features that already shipped, for a human to approve.
+
+Open for discussion: how this relates to the support agent's `support/kb/` docs (one corpus or two), whether pages ship in the app or are served by the backend, and who reviews the wording.
+
+- **Implementation plan:** [lifecycle-user-guide.md](https://github.com/TerrenoLabs/terreno/blob/master/docs/implementationPlans/lifecycle-user-guide.md)
+- **Pull request:** [#1633](https://github.com/TerrenoLabs/terreno/pull/1633)
+
+---
+
 ## list-detail-layout
 
 **Title:** `ListDetailLayout: a route-driven list/detail layout for phone and tablet`
@@ -1353,6 +1417,17 @@ One easy-to-use harness inside `@terreno/ai` that runs long-lived agentic work d
 
 ---
 
+## e2e-and-regression-testing
+
+**Title:** `E2E tests and manual regression cases in every plan`
+
+**Labels:** `area:tooling`, `type:feature`  
+**Project fields:** Area=`tooling`, Target=`Next`, Impact=`Feature`, IP=`e2e-and-regression-testing`, Status=`Planned`
+
+- **Implementation plan:** [e2e-and-regression-testing.md](https://github.com/TerrenoLabs/terreno/blob/master/docs/implementationPlans/e2e-and-regression-testing.md)
+
+---
+
 # Shipped, umbrella, and declined IPs
 
 Tracking issues created for IPs that previously lacked a `**Roadmap issue:**` header.

package/src/docs/versioned/next/how-to/README.md

@@ -41,6 +41,8 @@ Problem-oriented, practical steps. Use these when you know what you want to do.
 - [Debug with MCP](debug-with-mcp.md) — Follow a failure from symptom to verified fix with the local MCP tools
 - [Connect an MCP client with a service token](connect-mcp-service-token.md) — Mint a personal `mcp_` key for Perplexity and other remote clients
 - [Enable MCP OAuth for a JWT app](enable-mcp-oauth.md) — Let Claude.ai and ChatGPT connect to `/mcp` through OAuth 2.1 with your own consent screen
+- [Manage MCP from the admin panel](manage-mcp-in-admin.md) — Service tokens, OAuth clients and connections, request log, tool on/off switches, and sign-out-everywhere
+- [Expose an MCP server safely](expose-mcp-server-safely.md) — Put `/mcp` on the internet with OAuth, service tokens, a named Cloudflare Tunnel, and Access on the console
 - [Install agent skills](install-agent-skills.md) — `npx skills`, the Cursor plugin, the Codex plugin, or the Claude Code plugin
 - [Develop an AI feature](ai-feature-development.md) — Dataset → prompt → evaluators → experiment → production label → live traces (SOP)
 - [Observe LLM calls](observe-llm-calls.md) — Register ObservabilityApp, primaries, price map, sampleRate

package/src/docs/versioned/next/how-to/ai-feature-development.md

@@ -1,6 +1,6 @@
 # Develop an AI feature
 
-Work this loop in **Terreno admin** when observability primaries are `local`. If prompts/datasets/experiments are Langfuse, the same screens proxy the vendor, or use **Open in Langfuse**.
+Work this loop in **Terreno admin**. Observability primaries are always `local`.
 
 Architecture: [AI observability](../explanation/ai-observability.md). Routes and models: [AI reference](../reference/ai.md). Register plugins: [Observe LLM calls](./observe-llm-calls.md).
 
@@ -43,9 +43,9 @@ Skip unlabeled items in experiments unless you only need generation smoke (still
 
 ## 3. Create the prompt
 
-1. Open **Admin → AI Observability → Prompts**. The folder rail shows counts; search filters name and folder. If your role lacks `aiPrompt:create`, **Create prompt** is hidden (API writes still 403).
-2. **Create prompt** (folder, name, system, user template with `{{variables}}`). That write is immutable **v1**. Open the row to the prompt hub (**Overview**, **Versions**, **Traces**, **Experiments**).
-3. On **Versions**, edit system/template/variables/temperature when `aiPrompt:update` is allowed, then **Save as vN+1**. There is no in-place save. **Playground → Run once** (when `aiPrompt:playground` is allowed) compiles and calls the model; it does **not** create a version. **Traces** and **Experiments** show recent items for this prompt from prompt detail relationships.
+1. Open **Admin → AI Observability → Prompts**. The folder filter shows counts; search filters name and folder. If your role lacks `aiPrompt:create`, the header **New prompt** is hidden (API writes still 403).
+2. **New prompt** (folder, name, system, user template with `{{variables}}`). That write is immutable **v1**. Open the row to the prompt hub (**Versions**, **Traces**, **Experiments**).
+3. **Versions** shows the description, folder, tags, and the editor together. Edit system/template/variables/temperature when `aiPrompt:update` is allowed, then **Save as vN+1**. There is no in-place save. System and template fields explain what each one is for. **Playground → Run once** (when `aiPrompt:playground` is allowed) compiles and calls the model; it does **not** create a version. **Traces** and **Experiments** show recent items for this prompt from prompt detail relationships.
 4. Do not paste that string into app routes. Apps call `AIService` with `promptName` + `promptLabel` (step 6).
 
 Read-only auditors with `aiPrompt:list` and `read` can review overview metadata and relationship tabs but do not see create, save, promote, or playground controls. Operators with the full `aiPrompt` action set see those controls. UI gating follows `permissions` on `GET /ai/observability/status`; assign roles in **Admin → Roles** per [API reference — AI observability RBAC](../reference/api.md#ai-observability-rbac).
@@ -126,15 +126,16 @@ Do this in parallel with steps 1–6 once the feature is reachable.
 4. In **Admin → AI Observability → Traces**, filter by time, prompt dropdown, status, user, session, score presence, or data sensitivity. Score and sensitivity dropdowns include both positive and negative states. Rows show a status dot, `sensitive` badge, error line, `N prompts`, tokens, cost, and latency. Open a row for the span tree (kind, indent, duration bar). Sensitive I/O disclosures start **collapsed**.
 5. Select one or more traces. The bulk bar warns when any selected row is `sensitive`. Choose **Send to human review**, read the queue explanation, and pick a human evaluator. The evaluator's dimensions become the score fields and its instructions guide the reviewer. Confirming posts `POST /ai/observability/traces/review` with `reason: "manual"`. Automatic (`llm-judge` / `json-assert`) evaluators cannot enter this queue.
 6. Open **Review queue**. Tabs show Pending / In progress / Done / Skipped counts, and pending
-   items are oldest-first. Select **Start reviewing — oldest first**.
+   items are oldest-first. Select the header **Review oldest pending**.
 7. On each item, compare **What the AI was given** with **What the AI wrote**. Long fields begin
    collapsed with a word count; structured outputs retain field labels and reviewer notes. Use
    **Raw JSON** only when the field view is insufficient.
-8. Score every required evaluator dimension: numeric dimensions use the declared range slider,
-   boolean dimensions use Pass / Fail, and categorical dimensions use pills. Add an optional
-   comment, then **Submit & next**. **Skip** records no score. For the second phase-1 intake path,
-   use **Assign to me** to manually claim an item and move it to In progress. Continue until the
-   toast says **Queue clear**.
+8. Score every required evaluator dimension on the same screen. The segmented **Yes** / **No**
+   control, the numeric slider, and categorical pills sit in a fixed column on the left of each
+   question. On web, `Y` / `N` answer the highlighted question as soon as the item opens, `Enter`
+   submits, and `S` skips. Add an optional note, then **Submit and open next**. **Skip** records
+   no score. For the second phase-1 intake path, use **Assign to me** to manually claim an item
+   and move it to In progress. Continue until the toast says **Queue clear**.
 
 Feedback is a score (`source: "user-feedback"`) fanned out to every ScoreSink. Non-owners who are not admin get 403.
 

package/src/docs/versioned/next/how-to/build-admin-screens.md

@@ -47,6 +47,9 @@ const syncConflicts = useConflicts();
 </AdminProvider>
 ```
 
+The nested `app/admin/[model]/_layout.tsx` stack sets `headerShown: false` too. Without that,
+Expo titles the screen `index` and draws a second back arrow above `AdminScreenPage`.
+
 Render this tree inside `SyncDbProvider client={adminSyncDb}`. If product screens
 also sync the same collection, make `adminSyncDb` a dedicated client with a unique
 store name and `windowCollections` set; do not mix admin rows into the owner/tenant

package/src/docs/versioned/next/how-to/circleci.md

@@ -112,7 +112,8 @@ which builds its own web bundle and backend (`playwright.config.ts` with
 policy checks share one `repo-policies` job so eight small checkouts do not
 sit in the concurrency queue. `repo-policies` uses Node 22.14 because Knip's
 oxc-parser throws `ERR_REQUIRE_ESM` on the shared 22.11 executor.
-Require `repo-policies` in branch protection. Require `e2e-auth` /
+The `typecheck` job runs in the same workflow: it compiles every package, then
+runs `bun run typecheck`. Require `repo-policies` and `typecheck` in branch protection. Require `e2e-auth` /
 `e2e-app` / … only as path-filtered checks; config-only PRs post `e2e-auth`
 as the smoke shard and do not run the other four. Do not require the old
 `no-barrel-imports` / `e2e-login` names.
@@ -296,6 +297,7 @@ the same commands through the parameterized `packages-ci` job, gated by
 |----------------------------|--------------|
 | Repository policies (barrels, source rules, explicit any, licenses, changelog, parity, lifecycle, static analysis) | `repo-policies` |
 | Verify rules are in sync | `rulesync-check` |
+| (none) — every package's `typecheck` (src + tests) | `typecheck` (runs with `run-repo-policies`) |
 | `dco` | `dco` |
 | (none) | `approve` |
 | Run all tests (API CI) | `api-ci` |
@@ -316,6 +318,7 @@ the same commands through the parameterized `packages-ci` job, gated by
 | Run admin script CLI (when backend CI did not already run) | `example-backend-script-runner` |
 | Build backend Docker image | `example-backend-docker` |
 | Admin SPA Build and E2E | `admin-spa-ci` |
+| (none) | `smoke` (every workspace `smoke` script; any package change) |
 | Lint, compile, and coverage (matrix package) | `packages-ci` (`admin-backend`, `admin-frontend`, `api-health`, `feature-flags`, `test`) |
 | E2E · `<shard>` | `e2e` (matrix `shard`: `auth`, `app`, `admin-core`, `admin-table`, `syncdb`) |
 | _(cross-site bearer specs, not required)_ | `e2e-cross-site` (after `e2e-prepare`; also the explicit-linking pass) |

package/src/docs/versioned/next/how-to/connect-mcp-service-token.md

@@ -110,3 +110,5 @@ The token acts as the owning user on **MCP only**. It cannot mint or list tokens
 Revoked and expired keys fail `tools/call` immediately. Disabled user accounts are refused the same way as over HTTP.
 
 A password reset, Better Auth password change, or "sign out everywhere" (`revoke-sessions` / `revoke-other-sessions`) bumps the user's `tokenEpoch` and ends every key minted before it. The keys show as revoked in the list; mint a new one for each client you still use.
+
+To put `/mcp` on the public internet behind a Cloudflare Tunnel, see [Expose an MCP server safely](expose-mcp-server-safely.md).

package/src/docs/versioned/next/how-to/enable-mcp-oauth.md

@@ -64,6 +64,10 @@ Booting with an issuer that has a path, a consent URL that is not a URL, or regi
 
 ## 2. Host the consent screen
 
+If the backend serves `@terreno/admin-spa` (`AdminSpaServeApp`), it already has one: set `consentUrl` to `${issuer}/console/oauth/consent` (use your `basePath` if it is not `/console`) and skip to step 3. Any signed-in user can approve there, and the page uses the Better Auth session cookie. See [admin-spa reference](../reference/admin-spa.md#mcp-oauth-consent-route).
+
+To build your own page instead:
+
 The client sends the user's browser to `/oauth/authorize`. Terreno checks the client, redirect URI, and PKCE, stores a pending request, and redirects to `consentUrl?request=<id>`. Your page:
 
 1. Makes sure the user is signed in (send them through login and back if not, keeping `request`).
@@ -183,3 +187,5 @@ Refresh tokens rotate on every use. If an old refresh token comes back, Terreno
 - Tokens, codes, and refresh tokens are stored as SHA-256 hashes.
 - Registration, authorize, token, and revoke count against the `auth` [rate-limit](rate-limiting.md) bucket. Turn rate limiting on in production, since registration is unauthenticated.
 - If a browser-based client (such as the MCP Inspector) runs discovery from another origin, allow that origin in `corsOrigin`.
+
+To put `/mcp` on the public internet behind a Cloudflare Tunnel, see [Expose an MCP server safely](expose-mcp-server-safely.md).

package/src/docs/versioned/next/how-to/expose-mcp-server-safely.md

@@ -0,0 +1,140 @@
+# Expose an MCP server safely
+
+Put a Terreno backend's `POST /mcp` on the public internet with OAuth 2.1 for interactive clients, service tokens for static-key clients, and a Cloudflare Tunnel in front, so no inbound port is open on the host. The admin console stays behind Cloudflare Access.
+
+This guide fits any Terreno backend with MCP tools. It is written for the single-binary MCP server from the `bootstrap-mcp-server` skill, which serves `/mcp` and the admin console (including its OAuth consent screen at `/console/oauth/consent`) from one origin.
+
+## Prerequisites
+
+- A Terreno backend with at least one MCP tool ([Expose MCP tools](expose-mcp-tools.md))
+- A domain on Cloudflare, for example `example.com`, and `cloudflared` on the host that runs the backend
+- A hostname for the server, for example `mcp.example.com`
+
+## What is public and what is not
+
+| Path | Who calls it | Exposure |
+| --- | --- | --- |
+| `POST /mcp` | MCP clients | Public. Every request needs a Bearer credential. |
+| `/.well-known/oauth-protected-resource/mcp`, `/.well-known/oauth-authorization-server` | MCP clients during discovery | Public |
+| `/oauth/authorize`, `/oauth/token`, `/oauth/register`, `/oauth/revoke` | MCP clients and the user's browser | Public, rate limited |
+| `/console` (including `/console/oauth/consent`), `/api/auth` | Operators, and any user approving an OAuth connection (they sign in and approve in the console) | Behind Cloudflare Access when every OAuth approver is in the Access policy; otherwise public, guarded by Better Auth and the `admin` check |
+| `/admin` | The console's admin API | Behind Cloudflare Access when `/console` is |
+| `/health` | Uptime checks | Public |
+
+MCP clients cannot complete a Cloudflare Access login, so the first three rows must bypass Access. Everything else does not need to.
+
+## 1. Pick the credential for each client
+
+| Client | Credential | Guide |
+| --- | --- | --- |
+| Claude.ai custom connectors, ChatGPT connectors | OAuth 2.1 access token (`mcpat_…`) | [Enable MCP OAuth](enable-mcp-oauth.md) |
+| Perplexity, Cursor, Claude Code, MCP Inspector | Service token (`mcp_…`) | [Connect an MCP client with a service token](connect-mcp-service-token.md) |
+| A headless service you run | OAuth client credentials | [Enable MCP OAuth](enable-mcp-oauth.md#headless-clients-client-credentials) |
+
+Every credential acts as one user and gets that user's MCP permissions. Mint service tokens for the person who uses them, not for an admin account.
+
+## 2. Configure the backend for a public origin
+
+```typescript
+const publicUrl = "https://mcp.example.com";
+
+new TerrenoApp({
+  userModel: User,
+  mcpServiceTokens: {enabled: true, publicMcpUrl: publicUrl},
+  mcpOAuth: {
+    allowedRedirectHosts: ["claude.ai", "chatgpt.com"],
+    consentUrl: `${publicUrl}/console/oauth/consent`,
+    dynamicClientRegistration: true,
+    enabled: true,
+    issuer: publicUrl,
+  },
+  rateLimit: {store: "memory", trustProxy: "loopback"}, // cloudflared on the same host
+});
+```
+
+- `issuer` must be the exact origin clients reach. Discovery documents, the `WWW-Authenticate` challenge, and token audiences are built from it. If it says `http://localhost:4000`, every remote client fails discovery.
+- Keep `allowedRedirectHosts` to the clients you support. With dynamic client registration on, anyone can register a client, and this list is what stops a registered client from sending authorization codes to a host you did not choose.
+- Turn rate limiting on. Registration and token requests are unauthenticated.
+- Set `trustProxy` for the proxy in front of the app. With `cloudflared` on the same host, every request arrives from `127.0.0.1`; `"loopback"` tells Express to read the client address from `X-Forwarded-For` instead. Confirm with one request that `req.ip` is your public address. Without it, every visitor shares one rate-limit bucket.
+- Set `BETTER_AUTH_URL` to the same public origin so console sign-in cookies are issued for it.
+
+## 3. Create a named tunnel
+
+Use a named tunnel. A quick tunnel (`cloudflared tunnel --url …`) gets a new `trycloudflare.com` hostname on every run, so the `issuer`, the discovery documents, and every client registered against the old hostname break. Cloudflare documents quick tunnels as a testing tool.
+
+```bash
+cloudflared tunnel login
+cloudflared tunnel create mcp
+cloudflared tunnel route dns mcp mcp.example.com
+```
+
+Write `~/.cloudflared/config.yml`:
+
+```yaml
+tunnel: mcp
+credentials-file: /home/<user>/.cloudflared/<tunnel-id>.json
+ingress:
+  - hostname: mcp.example.com
+    service: http://127.0.0.1:4000
+  - service: http_status:404
+```
+
+Run it as a service (`sudo cloudflared service install`), and check:
+
+```bash
+curl -s https://mcp.example.com/health
+curl -si -X POST https://mcp.example.com/mcp | grep -i www-authenticate
+```
+
+The second command prints `WWW-Authenticate: Bearer resource_metadata="https://mcp.example.com/.well-known/oauth-protected-resource/mcp"`. If the URL in it is not your public hostname, fix `issuer`.
+
+## 4. Close the direct path
+
+The tunnel makes an outbound connection, so the backend needs no inbound port. Block the backend port (4000 above) at the host firewall so the only way in is through Cloudflare, where Access and Cloudflare's own protections apply. Do not port-forward it from a router.
+
+## 5. Put the console behind Cloudflare Access
+
+In Cloudflare Zero Trust, add a **self-hosted** Access application for `mcp.example.com` with these paths, and an Allow policy for your operators' emails or identity-provider group:
+
+- `/console`
+- `/admin`
+- `/api/auth`
+
+That works only when every user who approves an OAuth connection is in the policy, because approving needs the console's consent screen, its sign-in page, and Better Auth. When people outside the policy connect clients too, leave `/console` and `/api/auth` out of the Access application and rely on Better Auth sign-in and the `admin` check. You can still keep `/admin` behind Access, since only operators (who are in the policy) call it.
+
+Do not add `/mcp`, `/.well-known`, or the other `/oauth/*` paths. An Access application on the bare hostname covers them too, so scope it to paths.
+
+Access is a second layer, not a replacement: the admin routes still require `admin: true`, and the console still requires a Better Auth session.
+
+## 6. Connect a client and check it
+
+1. Claude.ai: **Settings → Connectors → Add custom connector**, paste `https://mcp.example.com/mcp`, connect, approve on the consent page.
+2. If you registered `mcpAdminModels()` ([Manage MCP from the admin panel](manage-mcp-in-admin.md)), open **MCP OAuth clients** and **MCP OAuth connections** in the console and confirm the new client and connection.
+3. Static-key client: mint a token with `POST /mcp/service-tokens` (signed in as that user), then run `tools/list` with `Authorization: Bearer mcp_…`.
+
+## Revoking access
+
+- One service token: delete it on **MCP service tokens**, or the owner calls `DELETE /mcp/service-tokens/<id>`.
+- Every MCP credential one user holds: run the `mcp-sign-out-everywhere` admin script.
+- One OAuth connection: delete it on **MCP OAuth connections**, or have the client call `/oauth/revoke`.
+- Everything a user holds: disable the user, or reset their password or revoke their sessions (both bump `tokenEpoch`). Their OAuth grants and service tokens stop working on the next request. See [Enable MCP OAuth](enable-mcp-oauth.md#revoking-access).
+- Cut all traffic at once: stop the tunnel (`sudo systemctl stop cloudflared`).
+
+## Security checklist
+
+- [ ] `issuer` and `BETTER_AUTH_URL` equal the tunnel hostname
+- [ ] Named tunnel, not a quick tunnel
+- [ ] Backend port closed to inbound traffic
+- [ ] `rateLimit` on, with `trustProxy` set so `req.ip` is the visitor, not the proxy
+- [ ] `allowedRedirectHosts` lists only supported clients
+- [ ] Access covers `/console` and `/api/auth` only when every OAuth approver is in the policy (`/admin` can always be covered), and never `/mcp`, `/.well-known`, or `/oauth/*`
+- [ ] MCP tools expose only the methods each model needs (`mcp: {methods: ["list", "read"]}` unless writes are intended)
+- [ ] Credentials are minted per person, never shared from an admin account
+
+## Related
+
+- [Enable MCP OAuth for a JWT app](enable-mcp-oauth.md)
+- [Connect an MCP client with a service token](connect-mcp-service-token.md)
+- [Expose MCP tools](expose-mcp-tools.md)
+- [Manage MCP from the admin panel](manage-mcp-in-admin.md)
+- [Rate limiting](rate-limiting.md)