@terreno/mcp 57.9.0 → 57.10.1

package/src/docs/versioned/next/tasks/ai-observability.md

@@ -1,134 +1,263 @@
-# Tasks: AI observability (Langfuse-light)
+# Tasks: AI observability (Langfuse replacement)
 
 IP: [ai-observability](../implementationPlans/ai-observability.md)
+Design: [AI Observability Prototype](https://claude.ai/design/p/36eec001-f2f0-4161-b60d-20dace4d0a9c?file=AI+Observability+Prototype.dc.html)
 
 Supporting skills (every task as applicable): `mongoose-schema-safety`, `terreno-backend-api`, `backend-test-env`, `ai-prompt-governance`, `update-docs`, `generate-sdk`, `verify-ui-changes`, `terreno-ui`, `terreno-data-fetching`, `building-terreno-apps`.
 
-## Phase 1 — Contracts and tracer
+---
 
-- [ ] **Task 1.1**: Observability types + `ObservabilityApp` config validation
-  - Delivers: `ObservabilityPlugin` / sinks / `ObservabilityControlConfig`; boot throws on `experiments.primary !== datasets.primary` or `reviewQueue` ≠ `local`; throws if a primary’s plugin is missing
+# Phase 1 — MVP: prompt library, tracing, review queue
+
+Ships design screens: Prompts, Prompt editor (+ Playground), Traces, Trace detail, Review queue, Review item.
+
+## 1A — Contracts and tracer
+
+- [x] **Task 1.1**: Observability types + `ObservabilityApp` config validation
+  - Delivers: `ObservabilityPlugin` / `TraceSink` / `ScoreSink` / `PromptRegistry` / `ReviewQueue` interfaces; `ObservabilityControlConfig`; boot throws on `experiments.primary !== datasets.primary`, on `reviewQueue !== "local"`, and on a primary whose plugin is missing
   - Files: `ai/src/observability/types.ts`, `ai/src/observability/observabilityApp.ts`, `ai/src/observability/observabilityApp.test.ts`, `ai/src/index.ts`
   - Blocked by: none
-  - Docs: stub `docs/explanation/ai-observability.md` (two planes + config errors)
-  - Acceptance: unit tests for valid local-only, both plugins + defaults, illegal mix, langfuse reviewQueue, local-off + prompts primary local
+  - Docs: `docs/explanation/ai-observability.md` (two planes + config errors)
+  - Acceptance: unit tests for local-only valid config, illegal mix, langfuse review queue, missing plugin for a primary
 
-- [ ] **Task 1.2**: In-memory TraceSink/ScoreSink + `AIService` emit
-  - Delivers: every `generate*` exports a trace unless `skipTrace`; sink failures logged not thrown; `AIRequest` still written; `userId`/`sessionId`/`promptRef`/`priceMap` on options
+- [x] **Task 1.2**: In-memory sinks + `AIService` emit + production prompt resolve
+  - Delivers: `promptName` + `promptLabel` resolves via `PromptRegistry` **before the model call**, even when `skipTrace` is true or no trace sink is registered; missing registry/label/version returns 400 without a model call; resolved-version `sensitive` applies unless the generate option overrides it; every `generate*` exports a trace unless `skipTrace`; sink failures logged not thrown; `AIRequest` still written; `userId` / `sessionId` / `sensitive` / `promptRef` / `priceMap` on options; `costUsd` omitted for unpriced models
   - Files: `ai/src/observability/memorySinks.ts`, `ai/src/service/aiService.ts`, `ai/src/types/index.ts`, `ai/src/service/aiService.test.ts`
   - Blocked by: 1.1
-  - Docs: `docs/reference/ai.md` — `skipTrace`, `promptRef`, usage/cost fields
-  - Acceptance: tests: emit; skipTrace; throwing sink; costUsd from price map; missing model → no usd; multi-step still logs AIRequest
+  - Docs: `docs/reference/ai.md` — `skipTrace`, `promptName`, `promptLabel`, `sensitive`, usage/cost fields
+  - Acceptance: tests for emit, `skipTrace`, throwing sink, cost from price map, **no `costUsd` when the model is unpriced**, multi-step still logs `AIRequest`, resolved prompt body reaches the model with `skipTrace`, and missing registry/production label → 400 with no model call
 
-- [ ] **Task 1.3**: GPT routes pass identity
-  - Delivers: `/gpt/prompt` sets `userId` from `req.user` and `sessionId` from body/header `x-ai-session-id` into `AIService`
-  - Files: `ai/src/routes/gpt.ts`, `ai/src/routes/gpt.test.ts` (or existing route tests)
+- [x] **Task 1.3**: GPT routes pass identity and sensitivity
+  - Delivers: `/gpt/prompt` sets `userId` from `req.user`, `sessionId` from body/header `x-ai-session-id`, and may upgrade sensitivity with request `sensitive: true`; client `false` never downgrades a sensitive prompt version. Client `promptName` / `promptLabel` selection is admin-only; app routes select production prompts server-side
+  - Files: `ai/src/routes/gpt.ts`, route tests
   - Blocked by: 1.2
   - Docs: `docs/reference/ai.md` GPT section
-  - Acceptance: authenticated prompt produces a memory sink trace with that userId
+  - Acceptance: an authenticated prompt produces a memory-sink trace carrying that `userId` and `sessionId`
 
-## Phase 2 — Local telemetry + prompts
+## 1B — Local store: prompts, traces, review
 
-- [ ] **Task 2.1**: Local Mongo models for prompts + traces + spans + scores
-  - Delivers: schemas with descriptions, plugins, indexes from the IP; models registered only when local plugin constructs
-  - Files: `ai/src/observability/local/models/*.ts`, `ai/src/observability/local/types.ts`, model tests
+- [x] **Task 1.4**: Local Mongo models — prompts, versions, labels, traces, spans, scores
+  - Delivers: schemas with a description on every field, standard plugins, the indexes in the IP; models registered only when the local plugin constructs; `ObsPromptVersion` carries `variables[]` (`key`, `required`, `label?`, `reviewerNote?`), `inputSchema`, `outputSchema`, `outputFieldNotes`, **`sensitive` (default false)**, `config`; `ObsPrompt` carries `folder` + `tags[]`; `ObsTrace` carries `sensitive`, `errorSummary`, `prompts[]`; `ObsSpan` carries `status` and optional short `error`
+  - Files: `ai/src/observability/local/models/*.ts`, `ai/src/types/observability*.ts`, model tests
   - Blocked by: 1.1
   - Docs: `docs/reference/ai.md` model tables
-  - Acceptance: save/load version immutability; unique `(promptId, label)`; findExactlyOne/findOneOrNone only
+  - Acceptance: version immutability; unique `(promptId, label)`; `findExactlyOne` / `findOneOrNone` only; no unique index on scores
 
-- [ ] **Task 2.2**: Local `PromptRegistry` + prompt HTTP
-  - Delivers: create named prompt, immutable versions, move production/latest, compile `{{vars}}`, get by label
+- [x] **Task 1.5**: Local `PromptRegistry` + prompt HTTP + playground
+  - Delivers: create prompt in a folder; immutable `vN+1` create; move `production` / `staging` returning the outgoing version; `{{var}}` compile; `get({name, label})`; `POST /prompts/:name/playground` runs one `AIService` call and returns compiled messages, output, latency, tokens, cost without creating a version; list supports folder filter, search, and `include=usage7d`
   - Files: `ai/src/observability/local/promptStore.ts`, `ai/src/observability/routes/prompts.ts`, tests
-  - Blocked by: 2.1, 1.1
-  - Docs: `docs/how-to/observe-llm-calls.md` — create + pin production
-  - Acceptance: v1 unchanged after v2; get production returns v2; playground compile without new version
+  - Blocked by: 1.4, 1.1
+  - Docs: `docs/how-to/observe-llm-calls.md` — create a prompt and pin production
+  - Acceptance: v1 unchanged after v2; `get production` returns the labelled version; playground creates no version; usage rollup returns calls/cost for the last 7 days and `—` for a prompt with no production label
 
-- [ ] **Task 2.3**: Local `TraceSink`/`ScoreSink` + trace/session/user/cost HTTP
-  - Delivers: persist fan-in from ObservabilityApp; list/detail/filters; cost aggregates
+- [x] **Task 1.6**: Local `TraceSink` / `ScoreSink` + trace HTTP
+  - Delivers: persist the fan-in; list with time / prompt / status / user / session / `hasScore` / `sensitive` / `flaggedForDataset` filters and pagination; detail returns the span tree with kinds, offsets, durations, per-span I/O and cost; derive `errorSummary` from the first span where `status: "error"` using its `error`; `prompts[]` length drives the `N prompts` display; `POST /traces/:id/scores`
   - Files: `ai/src/observability/local/traceStore.ts`, `ai/src/observability/routes/traces.ts`, tests
-  - Blocked by: 2.1, 1.2
+  - Blocked by: 1.4, 1.2
   - Docs: `docs/reference/ai.md` routes
-  - Acceptance: filter user/session/prompt/minCost; nested spans round-trip; costs by model
-
-## Phase 3 — Evaluators, live sample, review
-
-- [ ] **Task 3.1**: Evaluator model + LLM-judge + code/json-assert
-  - Delivers: multidimensional scores; judge prompt from PromptRegistry production (or named constant in `prompts.ts` for the built-in judge wrapper); structured JSON via `generateJsonObject`
-  - Files: `ai/src/observability/local/evaluatorStore.ts`, `ai/src/observability/evaluate.ts`, `ai/src/service/prompts.ts` (judge wrapper constant), tests
-  - Blocked by: 2.2, 2.3
-  - Docs: explanation — dimensions; how-to — define evaluator
-  - Acceptance: numeric + boolean dimensions on one trace; judge parse failure records error score not throw to caller
-
-- [ ] **Task 3.2**: `sampleRate` live evals via BackgroundTask
-  - Delivers: default 0; >0 enqueues eval after trace export; still never throws from generate
-  - Files: `ai/src/observability/liveEval.ts`, `observabilityApp.ts`, tests
-  - Blocked by: 3.1
-  - Docs: how-to env `AI_OBS_SAMPLE_RATE`
-  - Acceptance: rate 0 → no task; rate 1 → BackgroundTask created (mock/stub runner ok)
-
-- [ ] **Task 3.3**: Local review queue API
-  - Delivers: enqueue from trace; list pending; submit scores; skip; assign
-  - Files: `ai/src/observability/local/reviewStore.ts`, `ai/src/observability/routes/review.ts`, tests
-  - Blocked by: 3.1
-  - Docs: how-to — review flow
-  - Acceptance: submit writes ScoreSink fan-out path; done item not in pending
-
-## Phase 4 — Datasets and local experiments
-
-- [ ] **Task 4.1**: Datasets + items (including from trace)
-  - Delivers: CRUD; add-from-trace copies input/output snapshot
-  - Files: dataset store + routes + tests
-  - Blocked by: 2.3
-  - Docs: how-to — gold set from traces
-  - Acceptance: item retains sourceTraceId; delete item does not delete trace
-
-- [ ] **Task 4.2**: Local ExperimentRunner always BackgroundTask
-  - Delivers: POST experiment compares 2–3 prompt versions on a dataset, runs evaluators, stores per-item results, aggregates; promote → production label
-  - Files: `ai/src/observability/local/experimentRunner.ts`, routes, tests
-  - Blocked by: 4.1, 3.1, 2.2
-  - Docs: how-to — run experiment + promote
-  - Acceptance: 1-item dataset still uses BackgroundTask; completed aggregates; promote moves label; failure sets experiment failed
-
-## Phase 5 — Langfuse adapter, OTel, admin, example
-
-- [ ] **Task 5.1**: Langfuse TraceSink + ScoreSink wrapping existing client
-  - Delivers: no second OTel SDK; best-effort; uses `getLangfuseClient()`
-  - Files: `ai/src/observability/langfuse/traceAdapter.ts`, tests (mock client)
+  - Acceptance: nested spans round-trip; every filter tested; a trace with two prompts reports both; unpriced model returns no `costUsd`
+
+- [x] **Task 1.7**: Human evaluator model + templates
+  - Delivers: `ObsEvaluator` with `type`, `target`, typed `dimensions[]`, `runModes`, `instructions`, `confidenceAlertBelow`; CRUD routes; `GET /evaluators/templates` returning the seeded set; `human` + `liveSampleRate > 0` rejected on save
+  - Files: `ai/src/observability/local/evaluatorStore.ts`, `ai/src/observability/evaluatorTemplates.ts`, `ai/src/observability/routes/evaluators.ts`, tests
+  - Blocked by: 1.4
+  - Docs: `docs/how-to/ai-feature-development.md` step 4 (human half)
+  - Acceptance: numeric / boolean / categorical dimensions persist; human evaluator with live sampling is a 400; a template installs by name
+
+- [x] **Task 1.8**: Local review queue API
+  - Delivers: `POST /traces/review` enqueues one or many traces against a human evaluator (`reason: "manual"`); `GET /review` by status with counts, oldest-first; `GET /review/:id` returns the item, its evaluator dimensions, and rendered `given` / `wrote` panels built from the prompt version's `variables[]` labels/notes and `outputSchema` + `outputFieldNotes`, falling back to raw keys; `POST /review/:id` submit / skip / assign
+  - Files: `ai/src/observability/local/reviewStore.ts`, `ai/src/observability/reviewPanels.ts`, `ai/src/observability/routes/review.ts`, tests
+  - Blocked by: 1.6, 1.7
+  - Docs: `docs/how-to/observe-llm-calls.md` — the review flow
+  - Acceptance: submit writes dimension-keyed scores through the ScoreSink fan-out and leaves pending; skip and assign move status; panels fall back cleanly for a prompt with no labels; a submitted item is absent from pending
+
+## 1C — Admin UI and example app
+
+- [x] **Task 1.9**: Admin chrome — nav group, breadcrumbs, status chip
+  - Delivers: one `AI Observability` sidebar group; `Admin / AI Observability / <Section> / <leaf>` breadcrumbs; status chip reading `GET /status` (local state, primaries); **Review queue hidden when the local plugin is off**; existing `AI Requests` untouched
+  - Files: `admin-frontend/src/widgets/aiObservability/shell/*.tsx`, `builtInWidgets.ts`
+  - Blocked by: 1.1
+  - Docs: `docs/reference/admin-frontend.md` screen names
+  - Acceptance: widget tests for chip states and for the hidden review entry
+
+- [x] **Task 1.10**: Prompts list + prompt editor + playground screens
+  - Delivers: folder rail with counts and search; table with type badge, latest, production, last used, calls 7d, cost 7d and the latest-vs-production tooltips; editor with full-width version history rows (`vN`, attached labels, creation time, selected highlight), Editor/Playground tabs, system + template + variables + schema summary, temperature preset and model hint, **Save as vN+1**, **Set vN as production…** confirm modal naming the outgoing version; playground variable inputs, Run once with a running state, compiled-messages disclosure, output, latency/tokens/cost, and a **Save this run to dataset** button disabled until phase 2
+  - Files: `admin-frontend/src/widgets/aiObservability/prompts/*.tsx`
+  - Blocked by: 1.5, 1.9
+  - Docs: `docs/how-to/ai-feature-development.md` steps 3 and 6
+  - Acceptance: widget tests empty/loading/loaded; no in-place save control exists; `verify-ui-changes` walks create → save v2 → set production
+
+- [x] **Task 1.11**: Traces list + trace detail screens
+  - Delivers: filter bar; row selection with the bulk bar (**Send to review queue**, sensitive-count warning, Clear; Add to dataset disabled until phase 2); status dot, `sensitive` badge, error line, `N prompts`, spans/tokens/cost/latency/scores columns; pagination footer; detail header with actions, left span list (kind badge, indent, duration bar), right span detail with **collapsed** sensitive I/O disclosures, and the scores panel with value + source
+  - Files: `admin-frontend/src/widgets/aiObservability/traces/*.tsx`
+  - Blocked by: 1.6, 1.8, 1.9
+  - Docs: `docs/how-to/ai-feature-development.md` steps 7–8 (observe half)
+  - Acceptance: widget tests for empty, error row, sensitive badge, and bulk bar; `verify-ui-changes` selects traces and sends them to the queue
+
+- [x] **Task 1.12**: Review queue + review item screens
+  - Delivers: tabs Pending / In progress / Done / Skipped with counts; oldest-first table (item, what the AI did, prompt, assignee, waiting, status); **Start reviewing — oldest first**; empty state naming both intake paths; item screen with "Item N of M pending" and prev/next, read-only given/wrote panels (long fields collapsed with word count, structured output as key/value rows, per-field reviewer notes), raw JSON disclosure, dimension-driven score form (slider / Pass-Fail / pills) headed by the evaluator instructions, comment, **Submit & next** / Skip / Assign to me, and a toast reporting the remaining count
+  - Files: `admin-frontend/src/widgets/aiObservability/review/*.tsx`
+  - Blocked by: 1.8, 1.9
+  - Docs: `docs/how-to/ai-feature-development.md` — the review loop
+  - Acceptance: widget tests for each dimension control type and the empty state; `verify-ui-changes` reviews an item end to end and reaches "Queue clear"
+
+- [x] **Task 1.13**: example-backend registration + seed + phase-1 docs
+  - Delivers: `ObservabilityApp` with the local plugin always on; `AI_OBS_PRICE_MAP_JSON` wiring; seed prompt `example-summarize` v1 labelled production in folder `examples`; seed the human evaluator template so the queue is walkable on a fresh DB; `.env.example`
+  - Files: `example-backend/src/server.ts`, seed script, `.env.example`, `docs/explanation/ai-observability.md`, `docs/how-to/observe-llm-calls.md`, `docs/how-to/ai-feature-development.md`, `docs/reference/ai.md`
+  - Blocked by: 1.10, 1.11, 1.12
+  - Docs: (this task)
+  - Acceptance: server boots local-only with no Langfuse keys; a stranger can follow the how-to to create a prompt, pin production, trace a call, and review it; `bun run --filter=@terreno/ai compile test lint` green
+
+---
+
+# Phase 2 — Evaluation loop: evaluators, datasets, experiments
+
+Ships design screens: Evaluators, Evaluator detail, New evaluator, Datasets, Dataset detail, Experiments, New experiment, Experiment results.
+
+- [x] **Task 2.1**: `llm-judge` + `json-assert` evaluator types with the schema contract
+  - Delivers: judge execution via `AIService.generateJsonObject` against a registry judge prompt; `json-assert` in-process assertion (`path`, `constraint`, plus the built-in "validate against the prompt version `outputSchema`" mode); **create/save rejects a judge whose prompt output schema omits a required dimension, naming that dimension**; parse failure records an error score rather than throwing to the caller
+  - Files: `ai/src/observability/evaluate.ts`, `ai/src/observability/local/evaluatorStore.ts`, `ai/src/observability/evaluatorTemplates.ts`, tests
+  - Blocked by: 1.7, 1.5
+  - Docs: explanation — dimensions and the schema contract; how-to step 4
+  - Acceptance: mismatch rejected with the dimension key in the message; a judge writes exactly its declared keys; `confidenceAlertBelow` defaults to 0.7; templates for correctness / hallucination / helpfulness / toxicity / schema-assert install (`*-human` variants preserved for the review queue)
+
+- [ ] **Task 2.2**: Live sampling execution
+  - Delivers: after trace export, roll each evaluator's `liveSampleRate` (capped by `AI_OBS_SAMPLE_RATE`) and enqueue the eval on the same `BackgroundTask` path experiments use; never throws from generate
+  - Files: `ai/src/observability/liveEval.ts`, `ai/src/observability/observabilityApp.ts`, tests
+  - Blocked by: 2.1, 1.2
+  - Docs: how-to — the sampling ceiling and its cost
+  - Acceptance: rate 0 → no task; rate 100 with ceiling 0 → no task; rate 100 with ceiling 100 → task created
+
+- [x] **Task 2.3**: Datasets and items with provenance
+  - Delivers: dataset CRUD with `inputSchemaPromptName` + `expectedOutputSchema`; item CRUD with `origin`, `proofread`, `annotatedBy`, `tags`, `outcomeClass`; `POST /datasets/:id/import` (**JSON** array of bare objects or structured rows, plus **CSV** via `Content-Type: text/csv` or `{format: "csv", content}`; validates against the bound input schema); `POST /traces/add-to-dataset` single and bulk copying the I/O snapshot with `sourceTraceId`; **a `sensitive` trace always lands `proofread: false`**
+  - Files: `ai/src/observability/local/datasetStore.ts`, `ai/src/observability/routes/datasets.ts`, tests
+  - Blocked by: 1.6
+  - Docs: how-to steps 1–2 and 8
+  - Acceptance: sensitive rule enforced; item keeps `sourceTraceId`; deleting an item does not touch the trace; import rejects schema-invalid rows with the failing path
+
+- [ ] **Task 2.4**: Synthetic dataset generation + playground save-to-dataset
+  - Delivers: `POST /datasets/:id/generate` drafting items via `AIService` and a named generator prompt, `origin: "synthetic"`, `proofread: false`; the phase-1 playground **Save this run to dataset** button wired
+  - Files: generate route, `ai/src/service/prompts.ts` generator constant, `admin-frontend/.../prompts/*`, tests
+  - Blocked by: 2.3, 1.5
+  - Docs: how-to step 1 (synthetic)
+  - Acceptance: generated items are excluded from a default experiment run; a playground run lands as an unproofread item with its input and output
+
+- [x] **Task 2.5**: Experiment runner, gates, and gate-blocked promote
+  - Delivers: `POST /experiments/estimate` (generations, USD, wall clock); `POST /experiments` comparing 2–3 versions on a dataset with optional `modelOverride`, always via `BackgroundTask` locally; per-item results per version; aggregates vs `thresholds[]` (defaulting to `SOP_DEFAULT_THRESHOLDS`); outlier and low-confidence item ids; failed items ordered first; `POST /experiments/:id/promote` returning **409 while any gate fails**
+  - Files: `ai/src/observability/local/experimentRunner.ts`, `ai/src/observability/routes/experiments.ts`, tests
+  - Blocked by: 2.3, 2.1, 1.5
+  - Docs: how-to steps 5–6
+  - Acceptance: a 1-item dataset still uses `BackgroundTask`; gate pass/fail computed for boolean (`trueRate`) and numeric (`mean`) dimensions; promote 409s on a failing gate and moves the label when gates pass; unproofread items excluded unless `includeUnproofread`
+
+- [ ] **Task 2.6**: Authenticated trace feedback
+  - Delivers: `POST /traces/:id/feedback` — thumbs / outcome class / flag-for-dataset; `source: "user-feedback"` score; owner (`trace.userId`) or admin; sets `flaggedForDataset`; optional review enqueue with `reason: "feedback"`
+  - Files: `ai/src/observability/routes/feedback.ts`, tests
+  - Blocked by: 1.6, 1.8
+  - Docs: how-to step 7; `docs/reference/ai.md`
+  - Acceptance: owner thumbs-down + flag succeeds; a stranger gets 403; an admin succeeds; the trace list `flaggedForDataset` filter finds it
+
+- [ ] **Task 2.7**: Evaluator screens (list, detail, new)
+  - Delivers: list with type badge, dimension summary, target, run-mode chips and the footnote; detail with the dimensions table, type-specific panel (judge prompt link + model + schema-match check, assertion path/constraint, reviewer instructions), run modes with the live-sampling slider and cost warning, and the **Used by** list with 30-day runs and cost; new-evaluator screen with type/target pickers, dimension builder, and the **inline schema mismatch error naming the undeclared dimension**
+  - Files: `admin-frontend/src/widgets/aiObservability/evaluators/*.tsx`
+  - Blocked by: 2.1, 1.9
+  - Docs: `docs/reference/admin-frontend.md`
+  - Acceptance: widget tests for each type's panel and for the mismatch error; `verify-ui-changes` creates a judge evaluator
+
+- [ ] **Task 2.8**: Dataset screens (list, detail)
+  - Delivers: list with the human/auto legend, provenance bar, source and updated columns; detail with human/auto counts, the input-schema binding line, Add item, Run experiment, tabs All / Human / Auto / **Needs review** (red count), and the items table with attribution lines and trace links
+  - Files: `admin-frontend/src/widgets/aiObservability/datasets/*.tsx`
+  - Blocked by: 2.3, 2.4, 1.9
+  - Docs: `docs/reference/admin-frontend.md`
+  - Acceptance: widget tests for each tab and the empty dataset; `verify-ui-changes` bulk-adds traces and sees them under Needs review
+
+- [ ] **Task 2.9**: Experiment screens (list, wizard, results)
+  - Delivers: list with status badge, running progress bar and cost, plus the BackgroundTask/Langfuse footnote; 4-step wizard with a completed-step rail, dataset picker showing counts and schema match, version checkboxes tagged latest/production/superseded, evaluator checkboxes with per-item cost, and a Review & run step showing the estimate; results with gate tiles, "N gates failing", the **Promote blocked** state naming the failing gate, the outliers list with reason chips, the side-by-side table with failed rows floated and highlighted, and the promote confirm modal
+  - Files: `admin-frontend/src/widgets/aiObservability/experiments/*.tsx`
+  - Blocked by: 2.5, 2.8, 1.9
+  - Docs: how-to steps 5–6
+  - Acceptance: widget tests for running progress, gate-fail results, and blocked promote; `verify-ui-changes` runs an experiment and promotes a passing version
+
+- [ ] **Task 2.10**: Phase-2 example seeds + docs
+  - Delivers: seed judge/assert evaluator templates and dataset `example-gold` with an input-schema binding; docs updated for evaluators, datasets, experiments, feedback, and `AI_OBS_SAMPLE_RATE`
+  - Files: `example-backend` seed, docs listed in the IP, `.rulesync/skills/ai-prompt-governance/SKILL.md` + `bun run skills:sync`
+  - Blocked by: 2.7, 2.8, 2.9, 2.6
+  - Docs: (this task)
+  - Acceptance: the full SOP loop is runnable from `docs/how-to/ai-feature-development.md` without Langfuse
+
+---
+
+# Phase 3 — Analytics and pluggable backends
+
+Ships design screens: Sessions, Session timeline, Users, User AI profile, Costs, plus sink health and Open in Langfuse.
+
+- [ ] **Task 3.1**: Session, user, and cost aggregates + screens
+  - Delivers: `GET /sessions`, `/sessions/:id`, `/users`, `/users/:id`, `/costs`; Sessions list with the Traces/Sessions toggle and the timeline detail; Users list subtitled "AI cost and usage per user — not the user CRUD table" and the profile with masked email, **no prompt content**, and 7d/30d/all-time rollups; Costs with the KPI row, by-model table showing **tokens only** for unpriced models plus the "no $0 lie" note, and the by-prompt table
+  - Files: `ai/src/observability/routes/analytics.ts`, `ai/src/observability/local/traceStore.ts`, `admin-frontend/src/widgets/aiObservability/analytics/*.tsx`, tests
+  - Blocked by: 1.6, 1.9
+  - Docs: `docs/reference/ai.md` routes; `docs/reference/admin-frontend.md`
+  - Acceptance: an unpriced model renders `tokens only` and never `$0`; no prompt or output text appears in any users response; widget tests for each screen
+
+- [ ] **Task 3.2**: Langfuse `TraceSink` + `ScoreSink`
+  - Delivers: adapters wrapping the existing `getLangfuseClient()`; **no second OTel SDK**; best-effort with logged failures
+  - Files: `ai/src/observability/langfuse/traceAdapter.ts`, tests with a mocked client
   - Blocked by: 1.2
-  - Docs: explanation — LangfuseApp vs ObservabilityApp
-  - Acceptance: generate with both sinks; langfuse mock throws → generate still ok; local row exists
+  - Docs: explanation — `LangfuseApp` vs `ObservabilityApp`
+  - Acceptance: one generate dual-writes; a throwing Langfuse mock leaves the local row intact and the caller unaffected
 
-- [ ] **Task 5.2**: Langfuse PromptRegistry / DatasetStore / ExperimentRunner adapters
-  - Delivers: primaries can be `langfuse`; Terreno HTTP facade delegates; Open in Langfuse URLs on status
+- [ ] **Task 3.3**: Langfuse `PromptRegistry` / `DatasetStore` / `ExperimentRunner` + primaries
+  - Delivers: `AI_OBS_*_PRIMARY` env wiring; Terreno HTTP facade delegating to the existing Langfuse admin routes; **Open in Langfuse** deep-link bases on `GET /status`
   - Files: `ai/src/observability/langfuse/*.ts`, tests
-  - Blocked by: 5.1, 2.2, 4.2
-  - Docs: how-to — switch primaries via env
-  - Acceptance: prompts primary langfuse list uses client; experiments primary langfuse does not start BackgroundTask
+  - Blocked by: 3.2, 1.5, 2.5
+  - Docs: how-to — switching primaries by env
+  - Acceptance: prompts primary `langfuse` reads through the client; experiments primary `langfuse` starts no `BackgroundTask`; the illegal mixes still fail boot
 
-- [ ] **Task 5.3**: `OtelTraceSink` OpenInference
-  - Delivers: OTLP HTTP JSON or SDK exporter; `openinference.span.kind=LLM`; optional endpoint
-  - Files: `ai/src/observability/otel/otelTraceSink.ts`, tests with mock exporter
+- [ ] **Task 3.4**: `OtelTraceSink` (OpenInference)
+  - Delivers: OTLP exporter behind `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`; `openinference.span.kind` set from the span kind; model and token attributes
+  - Files: `ai/src/observability/otel/otelTraceSink.ts`, tests with a mock exporter
   - Blocked by: 1.2
-  - Docs: how-to — Phoenix/collector endpoint
-  - Acceptance: mock exporter receives span with model + token attrs
+  - Docs: how-to — pointing at Phoenix or a generic collector
+  - Acceptance: the mock exporter receives an `LLM` span with model and token attributes
 
-- [ ] **Task 5.4**: `admin-frontend` screens
-  - Delivers: prompts, traces, costs, evaluators, datasets, experiments, review; status chip; hide review without local; wire `customScreens`
-  - Files: `admin-frontend/src/widgets/aiObservability/*.tsx`, `builtInWidgets.ts`, `ai/src` adminContribution
-  - Blocked by: 2.2, 2.3, 3.3, 4.2
-  - Docs: `docs/reference/admin-frontend.md` screen names
-  - Acceptance: widget tests empty/loading; `verify-ui-changes` on admin (login admin, open Prompts + Traces)
-
-- [ ] **Task 5.5**: example-backend register + seed + env
-  - Delivers: local plugin on; Langfuse sinks if keys; OTel if endpoint; seed prompt/evaluator; fail-fast config
-  - Files: `example-backend/src/server.ts`, seed script, `.env.example`
-  - Blocked by: 5.1, 5.3, 2.2, 3.1
-  - Docs: how-to uses example-backend snippet
-  - Acceptance: server boots with local-only; invalid primary mix prevents listen (test or documented script)
-
-- [ ] **Task 5.6**: Docs completeness + Langfuse IP pointer + prompt-governance skill note
-  - Delivers: explanation, how-to, reference complete; `docs/implementationPlans/terreno-langfuse-integration.md` see-also; skill: registry production prompts allowed
-  - Files: docs listed in IP; `.github/skills/ai-prompt-governance/SKILL.md` (source) + `bun run skills:sync` if required
-  - Blocked by: 5.5
+- [ ] **Task 3.5**: Sink health chip, example wiring, and phase-3 docs
+  - Delivers: last-error-per-sink on `GET /status`; the chip renders Langfuse health and active primaries and shows **Open in Langfuse** only when that plugin is registered; example-backend adds the Langfuse and OTel sinks conditionally; docs and the `terreno-langfuse-integration.md` pointer finalized
+  - Files: `ai/src/observability/routes/status.ts`, `admin-frontend/.../shell/*.tsx`, `example-backend/src/server.ts`, docs
+  - Blocked by: 3.1, 3.3, 3.4
+  - Docs: (this task)
+  - Acceptance: chip states covered by widget tests; a Langfuse-less boot shows no deep link; `verify-ui-changes` evidence attached
+
+---
+
+# Phase 4 — Domain-neutral RBAC and prompt relationship hub
+
+The user approved this phase on 2026-10-02. It adds no domain-specific vocabulary or
+in-app approval workflow. Supporting skills: `building-admin-interfaces`,
+`terreno-backend-api`, `terreno-ui`, `terreno-data-fetching`, `mongoose-schema-safety`,
+`update-docs`, and `verify-ui-changes`.
+
+- [x] **Task 4.1**: Observability RBAC vocabulary, compatibility grants, and route enforcement
+  - Delivers: resources `aiPrompt`, `aiTrace`, `aiReview`, `aiDataset`, `aiExperiment`, `aiEvaluator` with the actions approved in Q62/Q63; statement descriptions; full seeded Admin grants, read-only Auditor expansion, Super Admin wildcard; existing stored Admin roles gain only missing observability grants; optional `accessControl` on `ObservabilityApp`; every route maps to one resource/action while legacy admin fallback remains
+  - Files: `api/src/rbac/statements.ts`, `api/src/rbac/roleModel.ts`, `ai/src/observability/observabilityApp.ts`, `ai/src/observability/routes/*.ts`, route/RBAC tests, public exports as needed
+  - Blocked by: none
+  - Docs: `docs/reference/api.md`, `docs/reference/ai.md`, `docs/explanation/ai-observability.md`
+  - Acceptance: catalog and role tests prove exact actions; a read-only RBAC caller can GET but receives 403 for writes; denied mutations fail even when called directly; legacy admin tests remain green
+
+- [x] **Task 4.2**: Prompt descriptions and prompt-scoped relationship APIs
+  - Delivers: optional prompt `description` in Mongo/types/create/detail/list; trace list accepts prompt version in addition to name; experiment list accepts `promptName`; prompt detail API composes bounded recent traces and experiments for the selected prompt without client-side full-list scans
+  - Files: prompt/trace/experiment models, stores, routes, types, tests
+  - Blocked by: 4.1
+  - Docs: `docs/reference/ai.md`, `docs/explanation/ai-observability.md`
+  - Acceptance: description round-trips; prompt v2 filtering excludes v1-only traces; prompt detail relationships contain only the named prompt's traces and experiments; old prompts without descriptions still load
+
+- [x] **Task 4.3**: Caller-filtered navigation and read-only prompt hub
+  - Delivers: each custom screen declares its read/list permission; unauthorized screens disappear from `/admin/config`; prompt detail renders Overview, Versions, Traces, and Experiments with description/folder/tags; controls are hidden independently for denied `update`, `promote`, and `playground`; direct API calls remain protected by Task 4.1
+  - Files: `ai/src/observability/adminScreens.ts`, admin contribution tests, `admin-frontend/src/widgets/aiObservability/prompts/*`, permission helpers/tests
+  - Blocked by: 4.1, 4.2
+  - Docs: `docs/reference/admin-frontend.md`, `docs/how-to/ai-feature-development.md`
+  - Acceptance: admin-config tests cover mixed grants; widget tests prove a read-only prompt hub shows relationships and no write controls; an operator sees all controls; `verify-ui-changes` captures both roles
+
+- [x] **Task 4.4**: Phase-4 integration and documentation gate
+  - Delivers: `ObservabilityApp({accessControl: access})` in example-backend; example roles demonstrating read-only and operator composition; integration tests for admin vs viewer HTTP; all architecture/reference/how-to pages aligned; generated SDK and `verify-ui-changes` walkthrough when the parent closes UI gates
+  - Files: `example-backend` access/seed files, `observabilityRbac.integration.test.ts`, docs named above, generated SDK when required
+  - Blocked by: 4.3
   - Docs: (this task)
-  - Acceptance: a stranger can register ObservabilityApp from how-to without reading the IP
+  - Acceptance: example boots with existing admin access intact; a consumer can configure a domain-neutral read-only role from docs; example-backend RBAC unit + integration tests, compile, and lint pass; parent runs SDK regen and mandatory UI walkthrough before checking this task complete

package/src/docs/versioned/next/tasks/durable-agent-harness.md

@@ -0,0 +1,183 @@
+# Tasks: Durable agent harness
+
+IP: [durable-agent-harness.md](../implementationPlans/durable-agent-harness.md)  
+**Feature profile:** false (full IP)  
+**Base branch:** `cursor/ai-observability-ip-64ca` (rebase as it lands)  
+**First PR scope:** Phases 1–2. Phases 3–5 ship in follow-up PRs.
+
+## Resume here (paused 2026-10-03)
+
+The Pick ⇄ Roast loop is paused after Phase 1. Tasks 1.1–1.12 are Roast-verified, one commit each.
+
+1. **Sync the base.** `git fetch origin && git merge origin/cursor/ai-observability-ip-64ca`.
+   At pause it was 5+ commits ahead, including a master merge and "Preserve AI observability
+   across chat turn refactor", which touches `ai/`. Re-run `cd ai && bun run test:ci` after the merge.
+2. **Resume the loop** with `/terreno:pick-roast-loop` (or `terreno-pick-roast-loop`) at
+   **Task 2.0**, then 2.1, then 2.2. Each task is Pick, then an independent Roast, then one commit.
+3. **Brew when Phase 2 is done.** Run the full independent branch review that was deferred at
+   pause, take the PR out of draft, then Taste.
+
+Known environment notes from Phase 1:
+
+- `/tmp` in the cloud VM is a 1 GB tmpfs, and in-memory replica sets use about 200 MB each. Run Mongo
+  test suites sequentially and delete leftover `/tmp/mongo-mem-*` directories after killed runs.
+- `example-backend` seed-heavy tests can hit Bun's 5 s default timeout under load. This also
+  happens on the base branch.
+- In the root `bun run lint`, `check:explicit-any:lint` requires a `// noExplicitAny: <reason>` line
+  above every `biome-ignore … noExplicitAny`.
+
+Every task: bun tests in `ai/` against the in-memory replica set from `@terreno/test`; real
+models are replaced by the mock model pattern in `.claude/rules/ai/00-ai.md`. Docs pages are
+created in 1.1 and extended by every later task — never deferred.
+
+## Phase 1 — Durable clinical tracer
+
+- [x] **Task 1.1**: Harness subpath + `defineTask` + transactional checkpoint/span (tracer)
+  - Delivers: `@terreno/ai/harness` export; `HarnessTask` model; `defineTask` with phases; `InProcessRunner` (no lease yet) runs a two-phase task to `completed`; `rt.commit` writes task + `ObsSpan` + root `ObsTrace` in one transaction; `Harness.open` throws without local observability plugin or without a replica set
+  - Files: `ai/package.json`, `ai/src/harness/harness.ts`, `defineTask.ts`, `runtime.ts`, `commit.ts`, `runners/inProcessRunner.ts`, `models/harnessTask.ts`, `ai/src/types/harness.ts`, tests
+  - Blocked by: none
+  - Skills: `terreno-backend-api`, `mongoose-schema-safety`, `backend-test-env`
+  - Docs: create `docs/explanation/durable-agent-harness.md` (concepts table), `docs/reference/ai-harness.md` (`defineTask`, `Harness.open`, `rt.commit`), link from `docs/reference/ai.md`
+  - Acceptance: AC1 — bun test asserts one CHAIN span per phase; forced throw inside the transaction leaves neither task update nor span
+
+- [x] **Task 1.2**: Owner lease, crash resume, replay semantics
+  - Delivers: `HarnessOwner` lease + heartbeat; standby second process; task leases with fencing token; resume on `start()`; per-phase `replay`; `interrupted` status; `harness.resolveInterrupted(taskId, action)`
+  - Files: `runners/inProcessRunner.ts`, `models/harnessOwner.ts`, `runtime.ts`, `commit.ts`, tests
+  - Blocked by: 1.1
+  - Skills: `terreno-backend-api`, `backend-test-env`
+  - Docs: reference "Replay and interruption" table; explanation section on leases
+  - Acceptance: AC2 + AC3 (phase rows) — simulate death by abandoning a harness mid-phase with an expired lease; new harness resumes safe phase and parks `never` phase as `interrupted`; stale-token commit rejected
+
+- [x] **Task 1.3**: Retries, failure, abort + ownership tree
+  - Delivers: task `retry` policy with backoff+jitter (Luxon), `failed` after `maxAttempts`; `rt.createTask` child ownership; `harness.abort` bottom-up with `abort` handlers; `rt.waitForTasks` (`all` / `failFast`); `background` flag
+  - Files: `runtime.ts`, `retryBackoff.ts`, `ownership.ts`, tests
+  - Blocked by: 1.2
+  - Skills: `terreno-backend-api`
+  - Docs: reference retry/abort/ownership; explanation ownership tree
+  - Acceptance: AC4 (phase retries) + AC5 — ordering assertion on abort handlers; `failFast` aborts siblings
+
+- [x] **Task 1.4**: Agents, conversations, tools, model-call resilience
+  - Delivers: `defineAgent`, `defineTool` (zod params, `replay`), `HarnessConversation` + `HarnessMessage`; built-in `terreno.agent.turn` task (`request` ⇄ `tools` phases, `maxSteps`); tools run as child tasks; interrupted `never` tools reported to the model; model calls retry 429/5xx then `fallbackModels`; interrupted model requests re-requested; LLM/TOOL spans with usage/cost; `ExecutionEnv` interface (types only) on `rt.env`
+  - Files: `defineAgent.ts`, `defineTool.ts`, `agentLoop.ts`, `modelCall.ts`, `executionEnv.ts`, `models/harnessConversation.ts`, `models/harnessMessage.ts`, tests
+  - Blocked by: 1.3
+  - Skills: `terreno-backend-api`, `claude-api` (model ids), `ai-prompt-governance`
+  - Docs: reference agents/tools/model resilience; explanation agent loop
+  - Acceptance: AC3 (tool rows) + AC4 (model rows) — mock model returns 503 twice then succeeds; fallback used after exhaustion; span tree CHAIN→LLM→TOOL
+
+- [x] **Task 1.5**: Subagents
+  - Delivers: `rt.runAgent(agent, {input, output})` creates a task-owned child conversation, idempotent on resume (finds existing by `ownerTaskId`), returns parsed structured output; AGENT span nests the child's spans
+  - Files: `runtime.ts`, `agentLoop.ts`, tests
+  - Blocked by: 1.4
+  - Skills: `terreno-backend-api`
+  - Docs: reference `runAgent`; explanation subagents
+  - Acceptance: bun test — crash between child creation and completion resumes the same child conversation (no duplicate); abort of parent aborts child
+
+- [x] **Task 1.6**: Extensions, hooks, wraps, memos
+  - Delivers: `defineExtension({name, sections, tools, hooks, wraps})`; `section` rebuilt per request and recorded in transcript; hooks `beforeModelRequest` / `beforeTool` (rewrite or `{block}`) / `afterTool`; `wrapTool`; `rt.memo(key, value?)` via `HarnessMemo`
+  - Files: `extensions.ts`, `models/harnessMemo.ts`, `agentLoop.ts`, tests
+  - Blocked by: 1.4
+  - Skills: `terreno-backend-api`
+  - Docs: reference extensions/hooks/memos
+  - Acceptance: AC6 — blocked tool result reaches the model; memo value survives a new harness instance
+
+- [x] **Task 1.7**: Events, waits, sleep
+  - Delivers: `rt.waitFor(event, {timeout})`, `rt.sleep(duration)`, `harness.sendEvent(taskId, event, payload)`; `waiting` status; runner wakes on event or `timeoutAt`; timeout resolves `undefined`
+  - Files: `runtime.ts`, `runners/inProcessRunner.ts`, tests
+  - Blocked by: 1.3
+  - Skills: `terreno-backend-api`
+  - Docs: reference waits/events
+  - Acceptance: bun test with frozen Luxon clock — event before timeout resumes with payload; timeout path; event sent while owner is down is delivered after restart
+
+- [x] **Task 1.8**: Approvals backend
+  - Delivers: `HarnessApproval` model; `rt.approval(key, {title, summary, payload, timeout, notify?})` with approvers declared per key on `defineTask({approvals})`; `approvalGate({tools, approvers})` extension (memo-backed); `HarnessApp` routes `GET /harness/approvals` (filtered by `approvers`), `approve` / `reject` instanceActions with reason; decision span with `decidedBy`
+  - Files: `approvals.ts`, `models/harnessApproval.ts`, `harnessApp.ts`, `routes/approvals.ts`, tests
+  - Blocked by: 1.6, 1.7
+  - Skills: `terreno-backend-api`, `model-router-actions`
+  - Docs: reference approvals + routes; how-to section "Require human approval"
+  - Acceptance: AC7 — supertest: unpermitted user gets 403 and an empty list; permitted user approves; span `output.decidedBy` equals user id; reject without reason → 400
+
+- [x] **Task 1.9**: SSE event stream + submit
+  - Delivers: `HarnessEvent` log (coalesced deltas with TTL, committed events permanent); `rt.output`; `GET /harness/conversations/:id/events` + `/harness/tasks/:id/events` SSE with `Last-Event-ID` replay then change-stream tail; `submit` instanceAction (`requestId` idempotent, `whenBusy: queue | steer`)
+  - Files: `events.ts`, `models/harnessEvent.ts`, `routes/events.ts`, `routes/conversations.ts`, tests
+  - Blocked by: 1.4
+  - Skills: `terreno-backend-api`, `model-router-actions`
+  - Docs: reference SSE contract (event types, ids, reconnect)
+  - Acceptance: AC9 — two `HarnessApp` instances on one replica set; client disconnects mid-stream, reconnects to the other with `Last-Event-ID`, receives each missed event exactly once
+
+- [x] **Task 1.10**: Version pinning
+  - Delivers: runs resume only on exact `name@version`; `Harness.start()` throws listing unregistered in-flight versions
+  - Files: `harness.ts`, `registry.ts`, tests
+  - Blocked by: 1.2
+  - Skills: `terreno-backend-api`
+  - Docs: reference "Versioning"; how-to "Ship a new task version"
+  - Acceptance: AC10
+
+- [x] **Task 1.11**: Approvals inbox (admin-frontend)
+  - Delivers: `admin-frontend/src/widgets/harness/HarnessApprovalInbox.tsx` (list, detail, approve/reject with reason, loading/empty/error states) via `adminRequest`; admin custom screen `harness-approvals` (group "AI Harness"); standalone export; example-frontend admin route; `bun run sdk`
+  - Files: `admin-frontend/src/widgets/harness/*`, `ai/src/harness/adminScreens.ts`, `example-frontend/app/admin/*`, tests
+  - Blocked by: 1.8
+  - Skills: `building-admin-interfaces`, `verify-ui-changes`
+  - Docs: `docs/reference/admin-frontend.md` inbox section
+  - Acceptance: AC8 — component tests + Playwright run on web: approve one item, reject one; screenshots in `/opt/cursor/artifacts/`
+
+- [x] **Task 1.12**: Clinical tracer in example-backend + how-to
+  - Delivers: `clinic.intakeSummary` (fetch → `runAgent` summary → approval → EHR write with `replay: "never"`) against a fake EHR model; `HarnessApp` registered in `server.ts`; admin action to start a run; e2e script kills the backend during `summarize`, restarts, approves in the inbox, asserts `filed` and the span tree in Traces
+  - Files: `example-backend/src/harness/*`, `example-backend/src/server.ts`, `example-frontend/e2e/harness-intake.spec.ts`
+  - Blocked by: 1.5, 1.9, 1.10, 1.11
+  - Skills: `verify-ui-changes`, `backend-test-env`
+  - Docs: create `docs/how-to/build-a-durable-workflow.md` using this workflow as the one example
+  - Acceptance: AC2 (e2e) — Playwright spec passes; screenshots of the inbox and the trace waterfall attached
+  - Shipped as two proofs: the kill/restart runs in `example-backend/src/harness/clinicalIntake.crash.test.ts` (spawns the real API process, SIGKILLs it mid-`summarize`, restarts); `harness-intake.spec.ts` approves in the inbox and asserts `filed` and the span tree
+
+## Phase 2 — Multi-instance
+
+- [ ] **Task 2.0**: `InProcessRunner` concurrency
+  - Delivers: `new InProcessRunner({concurrency})` (default 8) runs up to N claimed tasks at once, each with its own lease and heartbeat; a slow phase no longer delays other tasks' wakes; `stop()` waits for all in-flight phases
+  - Files: `ai/src/harness/runners/inProcessRunner.ts`, tests
+  - Blocked by: 1.12
+  - Skills: `terreno-backend-api`
+  - Docs: reference "Runners" (concurrency option, ordering guarantees)
+  - Acceptance: bun test — a phase blocked on a gate does not prevent a second task from completing; never more than N concurrent; no task runs twice concurrently
+
+- [ ] **Task 2.1**: `JobsRunner`
+  - Delivers: `@terreno/ai/harness/jobsRunner` with optional peer `@terreno/jobs`; job `terreno.harness.phase` keyed `taskId:phase:attempt`; harness task lease + heartbeat is the authority; importing the root `@terreno/ai` never loads jobs
+  - Files: `ai/src/harness/runners/jobsRunner.ts`, `ai/package.json`, tests
+  - Blocked by: 2.0
+  - Skills: `terreno-backend-api`
+  - Docs: reference "Runners"; how-to "Run on multiple instances"
+  - Acceptance: AC11 — two workers race one phase; exactly one commit; stale owner rejected; knip clean without jobs installed
+
+- [ ] **Task 2.2**: Runs admin screen
+  - Delivers: task list/tree with status filters, abort, resolve-interrupted (retry/abort/complete + reason), link to trace
+  - Files: `admin-frontend/src/widgets/harness/HarnessRuns*.tsx`, `ai/src/harness/routes/tasks.ts`
+  - Blocked by: 2.1
+  - Skills: `building-admin-interfaces`, `verify-ui-changes`
+  - Docs: admin-frontend reference
+  - Acceptance: Playwright resolves an interrupted EHR write; screenshots
+
+## Phase 3 — Long context
+
+- [ ] **Task 3.1**: Background compaction + manual `compact(instructions)`; originals stay searchable
+- [ ] **Task 3.2**: `reset` / handoff tool control
+- [ ] **Task 3.3**: Conversation forks at any message (parent history by reference)
+- [ ] **Task 3.4**: `defineDoc` typed documents (`scope`, `history: rewindable`, `fork: asOf`), committed atomically with transcripts
+
+Each 3.x task: Blocked by 1.12; Docs: reference + explanation sections; Acceptance: bun tests (10k-message conversation fits budget; fork isolation; doc rewind).
+
+## Phase 4 — Durable chat + multiplayer
+
+- [x] **Task 4.0**: Human gate — `GptHistory` migration strategy → **one-shot migration** script at deploy (decided 2026-10-02)
+- [ ] **Task 4.1**: `/gpt/prompt` on the harness; history mapping
+- [ ] **Task 4.2**: Steering, foreground/background abort, `viewState` over SSE
+- [ ] **Task 4.3**: rtk `useHarnessConversation` SSE hook + `GPTChat` adapter in example-frontend
+
+Acceptance: kill backend mid-chat, reconnect resumes; two browsers see the same view (Playwright, screenshots).
+
+## Phase 5 — Coding agents + malleability
+
+- [x] **Task 5.0**: Human gate — sandbox provider → **Cloud Run Jobs container** (decided 2026-10-02)
+- [ ] **Task 5.1**: `NodeExecutionEnv` + `CodingTools` (read/write/edit/bash with replay classes)
+- [ ] **Task 5.2**: Remote sandbox adapter
+- [ ] **Task 5.3**: Registry hot-replace (running calls finish on old code)
+
+Acceptance: a coding agent fixes a failing test in a fixture repo after a forced crash.