@terreno/mcp 57.10.1 → 57.12.0

package/dist/docs/versioned/next/reference/ai-harness.md

@@ -141,8 +141,8 @@ The `task` view passed to `run`: `{id, name, version, input, state, phase, attem
 | --- | --- |
 | `Harness.open({registry, runner?, models?, env?, priceMap?, streaming?, testHooks?})` | Checks requirements, rejects a duplicate `name@version` or agent name, ensures collections and indexes exist. `runner` defaults to `new InProcessRunner()`. See [open options](#open-options). |
 | `start()` | Throws, claiming nothing, when any non-terminal task uses a `name@version` missing from the registry (see [Versioning](#versioning)). Then starts the runner. Throws when already started. The runner recovers expired tasks once it owns execution. |
-| `stop()` | Stops claiming work, waits (without a time limit) for the phase in flight while still renewing the owner lease, then releases it. No-op when not started. |
-| `createTask(definition, input, {requestId?, userId?})` | Inserts a `pending` task, its `ObsTrace`, and its root span in one transaction. Wakes the runner. The definition must be in the registry. |
+| `stop()` | Stops claiming work, waits (without a time limit) for every phase in flight while still renewing the owner lease, then releases it. No-op when not started. |
+| `createTask(definition, input, {requestId?, userId?, trace?, prompts?})` | Inserts a `pending` task, its `ObsTrace`, and its root span in one transaction. Wakes the runner. The definition must be in the registry. See [Trace scope and prompt versions](#trace-scope-and-prompt-versions). |
 | `resolveInterrupted(id, {action, reason, result?, userId?})` | Resolve an `interrupted` task. See [resolveInterrupted](#resolveinterrupted). |
 | `abort(id, {reason, userId?})` | Abort a task and every non-terminal task it owns, bottom-up. See [abort](#abort). |
 | `decideApproval(approvalId, {approved, reason?, userId?})` | Approve or reject a pending approval without checking approvers. See [Approvals](#approvals). |
@@ -158,6 +158,30 @@ create with the same id returns the first task and leaves no extra trace. Reusin
 `requestId` for a different task name, or with a different `userId` (including none versus
 some), throws. A soft-deleted task still owns its `requestId`.
 
+### Trace scope and prompt versions
+
+A task's `ObsTrace` is its run record. `createTask` can label it:
+
+```typescript
+const prompt = await promptRegistry.get({label: "production", name: "intake-summary"});
+await harness.createTask(intakeSummary, {patientId}, {
+  prompts: prompt ? [prompt] : [],
+  trace: {scope: String(workspaceId), tags: ["nightly"]},
+  userId,
+});
+```
+
+| Option | Type | Description |
+| --- | --- | --- |
+| `trace.scope` | `string` | App-defined scope, such as a tenant or workspace id. Stored as `ObsTrace.scope`, indexed with `created` for per-scope metering (sum `LLM` span `usage.costUsd` by scope and month). Blank throws `harness-invalid-request`. |
+| `trace.tags` | `string[]` | App-defined labels, stored once each as `ObsTrace.tags`. A blank tag throws `harness-invalid-request`. |
+| `prompts` | `{name, version, label?}[]` | Prompt versions the task uses, stored once each on `ObsTrace.prompts`. A `PromptVersionRef` from `promptRegistry.get` fits as-is; its `body` is not stored. A ref without a name, with a non-string `label`, or without a positive integer `version` throws `harness-invalid-request`. |
+
+Child tasks and subagents write into the root task's trace, so they share its scope and
+tags. `rt.runAgent` adds its own prompt versions to the same trace (see
+[Subagents](#subagents-rtrunagent)). The admin trace list filters by `scope` and by
+prompt.
+
 ### open options
 
 | Option | Type | Description |
@@ -166,7 +190,7 @@ some), throws. A soft-deleted task still owns its `requestId`.
 | `models` | `({provider, modelId}) => LanguageModel` | Resolves an agent's model refs to Vercel AI SDK models. Required when `registry` lists an agent. |
 | `env` | `ExecutionEnv` | Handed to phases as `rt.env` and to tools as `api.env`. Optional. |
 | `priceMap` | `Record<modelId, {inputPerMTok, outputPerMTok}>` | Prices LLM spans (`usage.costUsd`). Defaults to the registered `ObservabilityApp`'s `priceMap`, read at call time. |
-| `runner` | `HarnessRunner` | Default `new InProcessRunner()`. |
+| `runner` | `HarnessRunner` | Default `new InProcessRunner()`. For many instances, `JobsRunner` (see [JobsRunner](#jobsrunner)). |
 | `streaming` | `{deltaFlushChars?, deltaFlushInterval?, deltaTtl?}` | How streamed model text becomes `delta` events. Defaults: 200 characters, `{milliseconds: 250}`, `{hours: 1}`. Non-positive values throw. See [Event stream (SSE)](#event-stream-sse). |
 | `testHooks` | `HarnessTestHooks` | Test-only. See [Testing](#testing). |
 
@@ -183,7 +207,8 @@ const harness = await Harness.open({
 
 | Option | Default | Description |
 | --- | --- | --- |
-| `pollInterval` | `{milliseconds: 250}` | Idle sleep between claim attempts. `createTask` wakes the runner early. |
+| `concurrency` | `8` | Most claimed tasks run at once. Must be a positive integer, or the constructor throws. `1` runs one task at a time. |
+| `pollInterval` | `{milliseconds: 250}` | Idle sleep between claim attempts. `createTask` and a task that frees a slot wake the runner early. |
 | `leaseDuration` | `{seconds: 30}` | Lifetime of the owner lease and of each task lease without a renewal. Luxon `DurationLike`. |
 | `heartbeatInterval` | `{seconds: 10}` | How often both leases are renewed. Must be positive and shorter than `leaseDuration`, or the constructor throws. |
 | `ownerId` | `hostname:pid:uuid` | Id written into every lease this runner holds. |
@@ -197,13 +222,101 @@ Only the owner claims work. It claims the oldest `pending` task whose `name@vers
 registered, whose `runAt` is empty or past (by Luxon's `DateTime.now()`), and that has no
 `abortRequested`. It sets the task `running` with a fresh task lease and runs phases one
 after another until the task stops: terminal, `pending` for a retry, `waiting`, or
-aborted. One task runs at a time. An idle owner re-polls every `pollInterval`, so a retry
-starts within one poll of its `runAt`.
+aborted. An idle owner re-polls every `pollInterval`, so a retry starts within one poll of
+its `runAt`.
+
+Up to `concurrency` tasks run at once, each under its own task lease and heartbeat, so a
+slow phase (a long model call, a phase blocked on I/O) does not delay other tasks. Ordering
+guarantees:
+
+- Tasks are **claimed** oldest first (`created`), one claim at a time, while a slot is free.
+- With `concurrency` above 1, claimed tasks run in parallel, so a later task can finish
+  first. Use `concurrency: 1` for strict one-at-a-time order.
+- Phases of one task always run in order, and a phase never commits twice: the claim is an
+  atomic update, and every commit is fenced on the task lease. (A runner that froze past
+  its lease may still be executing phase code while recovery replays it; only its commit
+  is rejected.)
+- Turns of one conversation stay serial regardless of `concurrency`; turns of different
+  conversations, and sibling child tasks, run in parallel.
+- `stop()` stops claiming and waits for every task in flight.
+
+Before Phase 2 the runner ran one task at a time. Upgrading to the default of 8 means
+parallel model calls (watch provider rate limits and cost) and more stream-counter retries
+(see [Implementation notes](#implementation-notes)). Pass `concurrency: 1` to keep the old
+behavior.
+
+### JobsRunner
+
+Runs phases as `@terreno/jobs` jobs so any number of instances share the work. Import it
+from its own entry point; `@terreno/jobs` is an optional peer dependency, and importing
+`@terreno/ai` or `@terreno/ai/harness` never loads it. How-to:
+[Run the harness on multiple instances](../how-to/run-harness-on-multiple-instances.md).
+
+```typescript
+import {JobsApp} from "@terreno/jobs";
+import {Harness} from "@terreno/ai/harness";
+import {JobsRunner} from "@terreno/ai/harness/jobsRunner";
+
+const jobs = new JobsApp();
+const harness = await Harness.open({registry, runner: new JobsRunner({jobs})});
+app.register(jobs).build(); // the runner enqueues through the registered jobs service
+await harness.start();
+await jobs.startWorker();
+```
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `jobs` | required | The `JobsApp`. The constructor defines `terreno.harness.phase` on it (`HARNESS_PHASE_JOB_NAME`). Register it and build the app before `harness.start()`; otherwise `start()` throws `harness-config-invalid`. |
+| `pollInterval` | `{milliseconds: 500}` | Sleep between dispatch scans. `createTask`, `sendEvent`, and each finished phase wake the dispatcher early. |
+| `leaseDuration` | `{seconds: 30}` | Lifetime of each task lease without a renewal. |
+| `heartbeatInterval` | `{seconds: 10}` | How often a running phase renews its task lease, and how often this instance sweeps expired leases. Must be positive and shorter than `leaseDuration`, or the constructor throws. |
+| `ownerId` | `hostname:pid:uuid` | Id written into every task lease this runner holds. |
+| `dispatchBatchSize` | `100` | Most runnable tasks one scan enqueues. Must be a positive integer, or the constructor throws. |
+
+Every instance runs a dispatcher and a jobs worker. There is no owner lease.
+
+1. **Dispatch.** Each scan lists runnable tasks (the same rule `InProcessRunner` claims by)
+   and enqueues one `terreno.harness.phase` job per task with payload `{taskId}` and
+   idempotency key `taskId:phase:attempt:claims`. `claims` counts how often the task was
+   claimed, so each runnable visit of a phase (first run, retry, event wake, replay) gets
+   its own job, and repeated scans or instances enqueue it once. When the job with that key
+   already ended without claiming the task (for example it died after its retries, or ran
+   on an instance mid-deploy that did not register the task), the dispatcher follows or
+   extends the chain `<key>:<endedJobId>` until a job is in flight, so the task is not
+   stranded.
+2. **Run one phase.** The job handler claims that task under a fresh fenced task lease,
+   runs **one** phase with heartbeats, and hands a task that is still `running` back as
+   `pending` (lease cleared) for the next job. It then wakes the dispatcher. A handler
+   whose task is no longer runnable (another job claimed it, or it is aborting) does
+   nothing and succeeds.
+3. **Recover.** Each instance sweeps expired task leases once per `heartbeatInterval`, with
+   the same replay and interruption rules as `InProcessRunner`. A replayed task becomes
+   runnable with a new `claims` value, so it is dispatched again.
+
+Load: with N instances, the recovery sweep (expired leases, settled child waits, stranded
+queued conversations) runs N times per `heartbeatInterval`, where `InProcessRunner` runs it
+once, and each scan does one job lookup per runnable task (up to `dispatchBatchSize`). The
+sweeps are fenced, so this costs queries, not correctness. Raise `heartbeatInterval` and
+`pollInterval` for large fleets.
+
+The task lease, not the job lock, is the authority. Two jobs for one phase race on the
+atomic claim, and only one runs it. A worker that froze past its lease cannot commit: its
+`rt.commit` throws `HarnessCommitConflictError`. A phase that runs longer than the jobs
+lock TTL is safe for the same reason. A job that runs on an instance whose harness is not
+started (or is stopping) throws `harness-runner-stopped`, so the jobs worker retries it.
+
+`stop()` stops dispatching and waits for every phase this instance is running. Stop the
+jobs worker (`jobs.stopWorker()`) after `harness.stop()`.
+
+### Custom runners
 
 A custom runner implements `HarnessRunner` (`start(context)`, `stop()`, `wake()`). The
 `context` provides `acquireOwnerLease(lease)`, `releaseOwnerLease(lease)`,
-`recoverExpired()`, `claimNext(lease)`, and `runTask(task, lease)`, where `lease` is
-`{owner, duration, heartbeat}` (Luxon `Duration`s).
+`recoverExpired()`, `claimNext(lease)`, `claimTask(taskId, lease)` (claim one task by id
+when it is runnable now, else `null`), `listRunnable(limit)` (oldest runnable tasks as
+`{taskId, phase, attempt, claims}`, unclaimed), and `runTask(task, lease, {maxPhases?})`,
+where `lease` is `{owner, duration, heartbeat}` (Luxon `Duration`s). With `maxPhases`,
+`runTask` hands a task that is still `running` after that many phases back as `pending`.
 
 ## Runtime (rt)
 
@@ -217,7 +330,7 @@ A custom runner implements `HarnessRunner` (`start(context)`, `stop()`, `wake()`
 | `rt.signal` | `AbortSignal`. Aborts when the task is aborted (at once in this process, within one heartbeat elsewhere), or when this run loses its lease. Pass it to cancellable calls. |
 | `rt.createTask(definition, input, {background?, key?})` | Create a child task. Returns its id. See [Child tasks and waitForTasks](#child-tasks-and-waitfortasks). |
 | `rt.waitForTasks(ids, {policy?})` | Return child outcomes once they settle; until then the task waits. See [Child tasks and waitForTasks](#child-tasks-and-waitfortasks). |
-| `rt.runAgent(agent, {input, output?, instructions?})` | Run a registered agent as a subagent and return its answer; until it finishes the task waits. See [Subagents (rt.runAgent)](#subagents-rtrunagent). |
+| `rt.runAgent(agent, {input, output?, instructions?, prompts?})` | Run a registered agent as a subagent and return its answer; until it finishes the task waits. See [Subagents (rt.runAgent)](#subagents-rtrunagent). |
 | `rt.memo(key)` / `rt.memo(key, value)` | Read, or first-write, a durable value scoped to this task. See [Memos](#memos). |
 | `rt.waitFor(event, {timeout?})` | Return the payload of the next `event` sent to this task, or `undefined` once `timeout` passes; until then the task waits. See [Events, waits, and sleep](#events-waits-and-sleep). |
 | `rt.sleep(duration)` | Return once `duration` has passed; until then the task waits. See [Events, waits, and sleep](#events-waits-and-sleep). |
@@ -261,7 +374,9 @@ Assumptions:
   recovery treats them as expired.
 
 Recovery runs when a runner becomes owner (on `start()` or on takeover) and on every owner
-heartbeat. It scans up to 100 `running` tasks whose lease expired (or that have no lease),
+heartbeat. `JobsRunner` has no owner lease: every instance sweeps once per
+`heartbeatInterval`, and concurrent sweeps are fenced, so each expired task is recovered
+once. It scans up to 100 `running` tasks whose lease expired (or that have no lease),
 oldest first, for registered `name@version`s only. A stopping owner skips the sweep. A task
 whose interruption commit keeps failing for a non-conflict reason is logged and retried on
 the next sweep.
@@ -726,7 +841,9 @@ is not offered to sections, `beforeModelRequest`, or `afterTool` (the tool alrea
 `basePath` defaults to `/harness`; it must start with `/` and not end with `/`.
 `heartbeatInterval` (default `{seconds: 15}`, must be positive) paces SSE heartbeats.
 Conversation, task, and approval routes are `modelRouter`s with instance actions, so they
-appear in `/openapi.json` (tag `harness`). The two SSE routes are plain Express routes
+appear in `/openapi.json`. Each route and its mutations use that collection's tag
+(`harnessconversations`, `harnesstasks`, `harnessapprovals`), so a generated client
+invalidates the list it just changed. The two SSE routes are plain Express routes
 (the listed exception to modelRouter actions) and are not in the OpenAPI spec.
 
 ### Conversation routes
@@ -749,7 +866,7 @@ appear in `/openapi.json` (tag `harness`). The two SSE routes are plain Express
 
 | Method | Path | Permissions | Behavior |
 | --- | --- | --- | --- |
-| GET | `/harness/tasks/:id` | Owner (`userId`) or admin | One task. Others: 403. |
+| GET | `/harness/tasks/:id` | Owner (`userId`) or admin | One task, without `lease` (the fencing token stays on the server). Others: 403. |
 | POST | `/harness/tasks/:id/abort` | Owner or admin | Body `{reason}` (required, non-blank). Runs `harness.abort` with the caller as `userId`; returns the aborted task. |
 | POST | `/harness/tasks/:id/resolveInterrupted` | Admin | Body `{action: "retry" \| "abort" \| "complete", reason, result?}`. Runs `harness.resolveInterrupted` with the caller as `userId`; returns the task. |
 | GET | `/harness/tasks/:id/events` | Owner or admin | SSE for the task and its descendants. See [Event stream (SSE)](#event-stream-sse). |
@@ -1229,7 +1346,8 @@ summarize: {
 | --- | --- | --- | --- |
 | `input` | `unknown` | Yes | The subagent's user message. A string is sent as-is (blank is rejected); anything else as `JSON.stringify(input)`. |
 | `output` | zod schema | No | Structured output. Defaults to the agent's own `output`. |
-| `instructions` | `string` | No | Replaces the agent's instructions for this subagent conversation only. |
+| `instructions` | `string` or `PromptVersionRef` | No | Replaces the agent's instructions for this subagent conversation only. A registry prompt (`promptRegistry.get`) runs with its `body`, and its `{name, version, label}` is added to the trace's `prompts`. |
+| `prompts` | `{name, version, label?}[]` | No | More prompt versions this subagent used, added to the trace's `prompts`. Each version is recorded once per trace. An invalid ref, or an `instructions` prompt without a string `body`, fails the phase with a definition error. |
 
 Returns the final assistant text, or, with a schema, the object the schema parsed. The
 generic `rt.runAgent<T>(...)` types the result when the schema comes from the agent.
@@ -1240,6 +1358,7 @@ What one call does:
    (`ownership: {kind: "task", id: <caller>}`, `status: "busy"`, agent snapshot with
    `instructions` and `outputSchema`), its user message (`seq` 1), and the turn task
    (`ownership: {kind: "task", id: <caller>}`) with an `AGENT` span named after the agent.
+   Prompt versions from `instructions` and `prompts` are added to the trace's `prompts`.
 2. Waits with `rt.waitForTasks([turn])`: the caller commits `waiting` and the phase stops.
 3. When the turn settles the caller wakes and the phase re-runs from its checkpoint. The
    same call finds the same turn and conversation (step 1 writes nothing) and returns.
@@ -1503,6 +1622,7 @@ empty objects are kept (`minimize: false`), so an initial state `{}` is stored a
 | `outcome` | `{status, result, error}` | Set on terminal commit. |
 | `attempt` | Number | Failed attempts of the current phase. Default 0. |
 | `step` | Number | Phase commits so far; names the current phase visit for idempotent child creation. Default 0. |
+| `claims` | Number | Times a runner claimed the task. Numbers each runnable visit; `JobsRunner` puts it in the job idempotency key. Default 0; a missing value reads as 0. |
 | `retry` | `{maxAttempts, backoffMs, maxBackoffMs}` | Copied from the definition. |
 | `ownership` | `{kind: root \| task \| conversation, id}` | Default `root`. `rt.createTask` children are `{kind: "task", id: <parent>}`; agent turns are `{kind: "conversation", id}`, except subagent turns, which are `{kind: "task", id: <caller>}`. |
 | `rootTaskId` | ObjectId | Top of the ownership tree; equals `_id` for root tasks. |
@@ -1686,12 +1806,13 @@ lists every kind:
 | `code` | `status` | `title` | Kinds of failure |
 | --- | --- | --- | --- |
 | `harness-definition-invalid` | 500 | Invalid harness definition | `defineTask` / `defineTool` / `defineAgent` / `defineExtension` / `approvalGate` validation, and `rt.*` misuse inside a phase. Thrown as `HarnessDefinitionError`, which fails the task at once (no retry). |
-| `harness-config-invalid` | 500 | Invalid harness configuration | `Harness.open`, `HarnessApp`, `InProcessRunner`, `streaming`, and registry problems. |
+| `harness-config-invalid` | 500 | Invalid harness configuration | `Harness.open`, `HarnessApp`, `InProcessRunner` (lease timing, `concurrency`), `JobsRunner`, `streaming`, and registry problems. |
 | `harness-replica-set-required` | 500 | MongoDB replica set required | `Harness.open` on a deployment without transactions. |
-| `harness-invalid-request` | 400 | Invalid harness request | A caller passed a bad argument (`abort`, `sendEvent`, `decideApproval`, `resolveInterrupted`, `submit` / `send`, `createConversation`, memos). |
+| `harness-invalid-request` | 400 | Invalid harness request | A caller passed a bad argument (`abort`, `sendEvent`, `decideApproval`, `resolveInterrupted`, `submit` / `send`, `createConversation`, memos, a `terreno.harness.phase` payload without `taskId`). |
 | `harness-not-registered` | 404 | Not registered in this harness | A task definition, agent, or extension the registry lacks. |
 | `harness-not-found` | 404 | Harness record not found | A conversation or child task disappeared. |
-| `harness-already-started` | 409 | Already started | `Harness.start` / `InProcessRunner.start` called twice. |
+| `harness-already-started` | 409 | Already started | `Harness.start` / `InProcessRunner.start` / `JobsRunner.start` called twice. |
+| `harness-runner-stopped` | 503 | Harness runner is not running | A `terreno.harness.phase` job ran on an instance whose harness is not started; the jobs worker retries it. |
 | `harness-commit-conflict` | 409 | Harness commit lost its checkpoint fence | `HarnessCommitConflictError`. |
 | `harness-approval-not-pending` | 409 | Approval can no longer be decided | `HarnessApprovalConflictError`. |
 | `harness-conversation-busy` | 409 | Conversation is busy | `HarnessConversationBusyError`. |
@@ -1809,7 +1930,7 @@ Low-risk choices made in the first slice:
 | The SSE tail starts at a cluster time read before any subscriber's replay query | No gap between replay and tail; overlap is removed by `seq`. |
 | One shared change stream per `HarnessApp`, not one per connection | A change stream holds a pooled connection while it waits; per-connection streams let a few dozen viewers starve commits and lease renewals. |
 | Queue recovery is a sweep by the execution owner, not part of the turn's terminal commit | Starting a turn is its own transaction; the sweep (fenced, idempotent) covers the crash window between the two. |
-| Writers to one stream conflict on its counter document | The price of commit-ordered `seq`s. `withTransaction` retries the loser. `InProcessRunner` runs one task at a time, so only HTTP submits, approvals, and deltas contend; a parallel runner fanning out many children of one tree will see retries. |
+| Writers to one stream conflict on its counter document | The price of commit-ordered `seq`s. `withTransaction` retries the loser. `InProcessRunner` (default `concurrency: 8`) and `JobsRunner` both run sibling tasks of one tree in parallel, so their writes to the shared root stream retry alongside HTTP submits, approvals, and deltas. `InProcessRunner({concurrency: 1})` avoids task-on-task contention. |
 | Interruption spans start at `lease.acquiredAt` | Records when the cut-off phase began. |
 | `resolveInterrupted({action: "abort"})` closes the trace as `error` | `ObsTrace.status` is `ok` or `error`; an abort is not a success. |
 | `requestId` replays are scoped to the same `userId` | Stops one caller from reading another user's task through a shared key. |

package/dist/docs/versioned/next/reference/ai.md

@@ -91,7 +91,7 @@ src/
 - **Plugins:** `AiApp`, `LangfuseApp`
 - **Service:** `AIService`, `TemperaturePresets`, `FileStorageService`, `MCPService`,
   `getMCPTools`
-- **Models:** `AIRequest`, `GptHistory`, `FileAttachment`, `Project`
+- **Models:** `AIRequest`, `GptHistory`, `FileAttachment`, `Project` (lazy; prefer `getProjectModel()`)
 - **Routes:** `addGptRoutes`, `addGptHistoryRoutes`, `addAiRequestsExplorerRoutes`, `addFileRoutes`, `addProjectRoutes`, `addMcpRoutes`
 - **Structured output:** `parseAiJson`, `normalizeLlmJsonTextForStructuredOutput`, re-exported `Output`, `jsonSchema`, `JSONValue`, `FlexibleSchema` from `ai`
 - **Langfuse:** `initLangfuseClient`, `getLangfuseClient`, `shutdownLangfuseClient`, `compilePrompt`, `createPrompt`, `getPrompt`, `createTelemetryConfig`, `preparePromptForAI`, `initTracing`, `shutdownTracing`, `LangfuseCache`, cache helpers
@@ -249,7 +249,8 @@ Metadata for files stored in GCS.
 
 ### Project
 
-GPT project with persistent context and memories.
+GPT project with persistent context and memories. Registered lazily: `getProjectModel()`
+(or any use of the deprecated `Project` export) registers it.
 
 | Field | Type | Description |
 |-------|------|-------------|
@@ -440,6 +441,7 @@ new AiApp({
 | `maxSteps` | Max tool-calling steps (default 5) |
 | `titleModelId` | Cheaper model for conversation title generation |
 | `openApiOptions` | Passed to route OpenAPI builders |
+| `projects` | Default `true`. `false` skips the `/gpt/projects` routes, so `AiApp` never registers the `Project` model |
 
 ## LangfuseApp plugin
 
@@ -508,7 +510,7 @@ server `AIService` when configured, otherwise a request-scoped service built fro
 | `ObsPrompt` | Named prompt (`name` unique) with `folder` and `tags[]` |
 | `ObsPromptVersion` | Immutable `vN` body, `variables[]`, schemas, `sensitive` (default false), `config` |
 | `ObsPromptLabel` | Movable labels; unique `(promptId, label)` |
-| `ObsTrace` | Root trace: user, session, status, `errorSummary`, `sensitive`, `prompts[]`, usage |
+| `ObsTrace` | Root trace: user, session, status, `errorSummary`, `sensitive`, `prompts[]`, `scope`, `tags[]`, usage |
 | `ObsSpan` | Nested span with `kind`, `status`, optional `error`, offsets, usage |
 | `ObsScore` | Scores on a trace/span; many per trace, **no unique index** |
 | `ObsEvaluator` | Evaluator: `type` (`human` \| `llm-judge` \| `json-assert`), `target`, `dimensions[]`, `runModes`, `instructions`, `judgePromptName` (judge), `assertion` (json-assert), `confidenceAlertBelow` (default 0.7) |
@@ -560,7 +562,7 @@ When `prompts.primary` is `local`, `ObservabilityApp.register` mounts admin-only
 
 | Method | Path | Behavior |
 | --- | --- | --- |
-| GET | `/ai/observability/traces` | Admin list. Query `from`, `to`, `prompt`, `status`, `userId`, `sessionId`, `hasScore`, `sensitive`, `flaggedForDataset`, `page`, `limit`. Body is `{data, page, limit, more, total}` so pagination survives RTK `{data}` unwrap. Each row includes `spanCount` and `scoreCount`. `prompts.length` is the `N prompts` count |
+| GET | `/ai/observability/traces` | Admin list. Query `from`, `to`, `prompt`, `status`, `userId`, `sessionId`, `scope`, `hasScore`, `sensitive`, `flaggedForDataset`, `page`, `limit`. Body is `{data, page, limit, more, total}` so pagination survives RTK `{data}` unwrap. Each row includes `spanCount` and `scoreCount`. `prompts.length` is the `N prompts` count |
 | GET | `/ai/observability/traces/:id` | Span tree (kind, offsets, durations, I/O, cost) plus scores. `errorSummary` is the first span with `status: "error"` |
 | POST | `/ai/observability/traces/:id/scores` | Persist a score and fan out to every `ScoreSink` |
 | POST | `/ai/observability/traces/test-multi-stage` | Admin-only smoke workflow, registered only with the local trace sink. Uses `ObservabilityApp.aiService`, or `requestAiServiceFactory({apiKey})` when the server service is absent (`apiKey` comes from `x-ai-api-key`); answers **503** with the same missing-key title as playground when neither exists. Body `{input?: string}` (defaults to a built-in sample). Runs two `AIService.generateJsonObject` calls with `skipTrace: true` and named JSON output schemas (`obs-test-multi-stage-call-1` / `call-2`), a deterministic local `text-metrics` `TOOL` stage, then a final `generateJsonObject` synthesis against `obs-test-multi-stage-final`; exports exactly one parent trace with ordered child spans `LLM`, `LLM`, `TOOL`, `LLM` under a `CHAIN` root. LLM span input includes `outputSchema`. Returns `{traceId, output, stages[]}` where `output` is the final schema object (`sentence`, `phrase`, `keywords`, `metrics`). Child LLM failures export an error trace then rethrow |
@@ -638,6 +640,33 @@ Low-level exports (also used by `addGptRoutes` when `langfuseSystemPromptName` i
 
 Subpath imports for tree-shaking: `@terreno/ai/langfuseClient`, `@terreno/ai/langfuseApp`.
 
+## Subpath exports
+
+Import these instead of `dist/` paths. Deep `dist/` imports break when a bundler (such as
+Vercel's) inlines them and their bare `@terreno/api` imports no longer resolve. Each
+subpath shares module instances with `@terreno/ai/harness`.
+
+| Subpath | Exports |
+| --- | --- |
+| `@terreno/ai/harness` | Harness public API; see [AI harness reference](ai-harness.md) |
+| `@terreno/ai/harness/jobsRunner` | `JobsRunner` |
+| `@terreno/ai/harness/agentLoop` | `createAgentTasks` and the agent turn internals |
+| `@terreno/ai/harness/commit` | `createTaskRecords`, `HarnessModels`, and other commit helpers, for custom commits that write domain records inside a task's fenced commit |
+| `@terreno/ai/harness/events` | `insertMessages`, `resolveStreamingOptions` |
+| `@terreno/ai/harness/internalRuntime` | `internalRuntime` |
+| `@terreno/ai/admin` | `AIAdminApp` |
+| `@terreno/ai/observability/observabilityApp` | `ObservabilityApp` |
+| `@terreno/ai/observability/localPlugin` | `createLocalObservabilityPlugin`, `createLocalObservabilityBundle` |
+| `@terreno/ai/observability/promptStore` | `LocalPromptStore` |
+
+The internals subpaths follow the harness's own changes; they carry no compatibility
+promise beyond the release they ship in.
+
+The root `@terreno/ai` entry registers the `Project` model on first use
+(`getProjectModel()` or the `Project` export), not on import. `AiApp` uses it for
+`/gpt/projects` by default; an app that owns a model named `Project` passes
+`new AiApp({projects: false})` and does not call `addProjectRoutes`.
+
 ## FileStorageService
 
 Google Cloud Storage helper for uploads referenced by `addFileRoutes`.

package/dist/docs/versioned/next/reference/api.md

@@ -41,7 +41,7 @@ REST API framework built on Express and Mongoose. Provides modelRouter (CRUD end
 - Validation: `configureOpenApiValidator`, `validateRequestBody`, `validateQueryParams`, `createValidator`
 - Middleware: `openApiEtagMiddleware`, `sentryAppVersionMiddleware`
 - Extensibility: `TerrenoPlugin` interface
-- Notifiers: `sendToSlack`, `formatSlackUserMention`, `lookupSlackUserIdByEmail`, `sendToGoogleChat`, `sendToZoom`
+- Notifiers: `sendToSlack`, `formatSlackUserMention`, `lookupSlackUserIdByEmail`, `createSlackPrivateChannel`, `inviteSlackUsersToChannel`, `postSlackMessage`, `findSlackChannelByName`, `sendToGoogleChat`, `sendToZoom`
 - HTTP client: `createAuthenticatedClient`, `withApiErrorHandling`, `normalizeApiError`, `markRetryUnsafe`
 
 ## Server Setup
@@ -224,6 +224,30 @@ export const todoRouter = modelRouter("/todos", Todo, {
 Do not add `endpoints: (router) => { router.get(...) }` when an action fits.
 Action config supports `allowAnonymous: true` as a per-action override; it does not change authentication for CRUD or sibling actions.
 
+## modelRouter list queries
+
+`GET /{path}` accepts filters on fields listed in `queryFields` (qs syntax, so operators work:
+`?status[$in][0]=open&created[$gte]=2026-01-01`), plus the reserved `limit`, `page`, and `sort`.
+`limit` is clamped to `maxLimit` (default 500); the response echoes the applied `limit` with
+`{data, limit, more, page, total}`.
+
+A filter on a field missing from `queryFields` is rejected before query validation runs, so
+it can never be stripped and silently ignored:
+
+```json
+{
+  "status": 400,
+  "code": "query-param-not-allowed",
+  "title": "Query parameter not allowed",
+  "detail": "title is not allowed as a query param. Add \"title\" to queryFields on the Todo modelRouter to filter on it. Allowed: completed, created, ownerId.",
+  "source": {"parameter": "title"},
+  "meta": {"queryParam": "title", "model": "Todo", "allowedQueryFields": ["completed", "created", "ownerId"]}
+}
+```
+
+`@terreno/syncdb` raises this as `QueryFieldNotAllowedError` (see
+[Query windows](syncdb.md#query-windows)).
+
 ## modelRouter array operations
 
 When a model has at least one array field, `modelRouter` provides subroutes for adding,
@@ -1581,6 +1605,44 @@ await sendToSlack(`${formatSlackUserMention("U012ABCDEF")} Deployment complete`,
 });
 ``````
 
+Incoming webhooks are fixed to the channel chosen when the webhook was created.
+Private channels, invites, and messages to a channel id use the bot token for
+one Slack workspace (`SLACK_BOT_TOKEN`):
+
+| Function | Slack method | Bot scopes |
+| --- | --- | --- |
+| `createSlackPrivateChannel` | `conversations.create`, `conversations.invite`, `chat.postMessage` | `groups:write`, `groups:write.invites`, `chat:write` |
+| `inviteSlackUsersToChannel` | `conversations.invite` | `groups:write.invites` for private channels, `channels:write.invites` for public |
+| `postSlackMessage` | `chat.postMessage` | `chat:write` |
+| `findSlackChannelByName` | `conversations.list` | `channels:read`, `groups:read` |
+| `lookupSlackUserIdByEmail` | `users.lookupByEmail` | `users:read.email` |
+
+The bot can post and invite only in channels it belongs to. Creating a private
+channel makes the bot a member. An existing private channel must already
+include the app; a bot cannot join a private channel by itself. Store the
+returned channel id. Names are unique, mutable, and limited to lowercase
+letters, numbers, hyphens, and underscores.
+
+``````typescript
+import {createSlackPrivateChannel, inviteSlackUsersToChannel} from "@terreno/api";
+
+const channel = await createSlackPrivateChannel({
+  name: "case-12345",
+  userIds: ["U012ABCDEF"],
+  initialMessage: "Case opened.",
+  mentionUserIds: ["U012ABCDEF"],
+});
+
+await inviteSlackUsersToChannel({
+  channelId: channel.channelId,
+  userIds: ["U098765432"],
+});
+``````
+
+A taken name throws status 409. `findSlackChannelByName({name})` returns a
+public channel, or a private channel the bot is already in. It returns
+`undefined` when that channel is not visible to the bot.
+
 ### Google Chat Notifications
 
 ``````typescript
@@ -1913,7 +1975,7 @@ Complete reference of environment variables used by @terreno/api:
 | Variable | Required | Default | Description |
 |----------|----------|---------|-------------|
 | `SLACK_WEBHOOKS` | No | — | JSON object mapping names to Slack webhook URLs: `{"default":"https://..."}` |
-| `SLACK_BOT_TOKEN` | No | — | Bot token (`xoxb-…`) with `users:read.email` for `lookupSlackUserIdByEmail` |
+| `SLACK_BOT_TOKEN` | No | — | Bot token (`xoxb-…`) for one workspace. Scopes: `users:read.email`, `groups:write`, `groups:write.invites`, `channels:write.invites`, `chat:write`, `channels:read`, `groups:read` |
 | `GOOGLE_CHAT_WEBHOOKS` | No | — | JSON object mapping names to Google Chat webhook URLs |
 | `ZOOM_CHAT_WEBHOOKS` | No | — | JSON object mapping names to Zoom webhook URLs |
 | `WEBHOOK_SECRET` | No | — | Secret for validating incoming webhook signatures |

package/dist/docs/versioned/next/reference/environment-variables.md

@@ -223,7 +223,7 @@ There is **no** `RATE_LIMIT_ENABLED` (or similar) read by `@terreno/api`. Apps t
 |----------|---------|----------|---------|--------|-------|
 | `SLACK_WEBHOOKS` | `@terreno/api` | ❌ | — | Yes | server |
 | `SLACK_WEBHOOK_URL` | scripts | ❌ | — | Yes | server |
-| `SLACK_BOT_TOKEN` | `@terreno/api` `lookupSlackUserIdByEmail` | ❌ | — | Yes | server |
+| `SLACK_BOT_TOKEN` | `@terreno/api` Slack Web API (`lookupSlackUserIdByEmail`, private channels, invites, `chat.postMessage`) | ❌ | — | Yes | server |
 | `GOOGLE_CHAT_WEBHOOKS` | `@terreno/api` | ❌ | — | Yes | server |
 | `GOOGLE_CHAT_WEBHOOK_URL` | scripts | ❌ | — | Yes | server |
 | `ZOOM_CHAT_WEBHOOKS` | `@terreno/api` | ❌ | — | Yes | server |

package/dist/docs/versioned/next/reference/lifecycle-plugin.md

@@ -16,6 +16,17 @@ loops that invoke those stages; they are not stages and must not appear as `stag
 | `terreno-4-brew` | Roast PASS for every in-scope task + branch/evidence | pushed head + PR + product-CI trigger check + review-bot wait + attached evidence | Taste |
 | `terreno-5-taste` | PR + current state | one current-head reaction; before push: fetch latest `master` (merge only when needed), re-verify last-run failed tests locally, run root `prepush` when present (otherwise affected-package checks) in a no-context subagent, then watch CI; invoked directly, repeats bounded reactions until terminal | null or fresh Taste |
 
+Pre-Grow skill (not a stage):
+
+| Skill | Input | Primary output | Next |
+| --- | --- | --- | --- |
+| `terreno-prd` | app owner + conversation, prototype, repository | interview, then a published full product PRD (problem, why now, solution, user stories, success metrics and guardrails, constraints, rollout, out of scope); every metric measurable or `TBD` with its decider | Grow, with the PRD as its spec; metrics become acceptance criteria, `TBD`s become grilling questions, settled product decisions are not re-asked |
+
+It is explicit-invoke only (`/terreno-prd`, `$terreno-prd`, `/terreno:prd`). It
+interviews with Grow's grilling procedure over a product tree, is sized for one app owner
+and one or two developers, and carries no implementation or testing decisions; those stay
+in Grow.
+
 Outer loops (not stages):
 
 | Skill | What it walks | Default |