@terreno/mcp 57.9.0 → 57.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/docs/upgrades/57.10.1.md +54 -0
- package/dist/docs/versioned/next/README.md +1 -0
- package/dist/docs/versioned/next/explanation/README.md +4 -0
- package/dist/docs/versioned/next/explanation/admin-plugin-frontend.md +2 -2
- package/dist/docs/versioned/next/explanation/agent-ui-asks.md +299 -0
- package/dist/docs/versioned/next/explanation/agent-ui-blocks.md +67 -0
- package/dist/docs/versioned/next/explanation/ai-observability.md +101 -0
- package/dist/docs/versioned/next/explanation/durable-agent-harness.md +531 -0
- package/dist/docs/versioned/next/explanation/example-coverage.md +3 -1
- package/dist/docs/versioned/next/explanation/loop-engineering.md +1 -1
- package/dist/docs/versioned/next/explanation/roadmap-seed-issues.md +29 -1
- package/dist/docs/versioned/next/explanation/typescript-configuration.md +1 -1
- package/dist/docs/versioned/next/explanation/versioning-policy.md +1 -1
- package/dist/docs/versioned/next/how-to/README.md +6 -0
- package/dist/docs/versioned/next/how-to/agent-ui-asks.md +730 -0
- package/dist/docs/versioned/next/how-to/agent-ui-blocks.md +104 -0
- package/dist/docs/versioned/next/how-to/ai-feature-development.md +156 -0
- package/dist/docs/versioned/next/how-to/build-a-durable-workflow.md +232 -0
- package/dist/docs/versioned/next/how-to/charts-and-dashboards.md +2 -0
- package/dist/docs/versioned/next/how-to/circleci.md +5 -3
- package/dist/docs/versioned/next/how-to/observe-llm-calls.md +197 -0
- package/dist/docs/versioned/next/how-to/run-tests-locally.md +2 -1
- package/dist/docs/versioned/next/how-to/ship-a-new-task-version.md +76 -0
- package/dist/docs/versioned/next/how-to/upgrade-terreno.md +1 -1
- package/dist/docs/versioned/next/implementationPlans/agent-ui-asks.md +688 -0
- package/dist/docs/versioned/next/implementationPlans/agent-ui-blocks.md +10 -6
- package/dist/docs/versioned/next/implementationPlans/ai-observability.md +475 -157
- package/dist/docs/versioned/next/implementationPlans/durable-agent-harness.md +340 -0
- package/dist/docs/versioned/next/implementationPlans/terreno-langfuse-integration.md +1 -1
- package/dist/docs/versioned/next/reference/README.md +3 -0
- package/dist/docs/versioned/next/reference/admin-config.md +2 -0
- package/dist/docs/versioned/next/reference/admin-frontend.md +216 -0
- package/dist/docs/versioned/next/reference/agent-ui-asks.md +1059 -0
- package/dist/docs/versioned/next/reference/ai-harness.md +1881 -0
- package/dist/docs/versioned/next/reference/ai.md +251 -25
- package/dist/docs/versioned/next/reference/api.md +46 -0
- package/dist/docs/versioned/next/reference/blocks.md +178 -0
- package/dist/docs/versioned/next/reference/environment-variables.md +1 -0
- package/dist/docs/versioned/next/reference/mcp-server.md +17 -0
- package/dist/docs/versioned/next/reference/test.md +11 -2
- package/dist/docs/versioned/next/reference/ui.md +325 -1
- package/dist/docs/versioned/next/tasks/agent-ui-asks.md +213 -0
- package/dist/docs/versioned/next/tasks/agent-ui-blocks.md +2 -1
- package/dist/docs/versioned/next/tasks/ai-observability.md +233 -104
- package/dist/docs/versioned/next/tasks/durable-agent-harness.md +183 -0
- package/dist/tools.d.ts +4 -0
- package/dist/tools.d.ts.map +1 -1
- package/dist/tools.js +37 -0
- package/dist/tools.js.map +1 -1
- package/package.json +3 -2
- package/src/docs/upgrades/57.10.1.md +54 -0
- package/src/docs/versioned/next/README.md +1 -0
- package/src/docs/versioned/next/explanation/README.md +4 -0
- package/src/docs/versioned/next/explanation/admin-plugin-frontend.md +2 -2
- package/src/docs/versioned/next/explanation/agent-ui-asks.md +299 -0
- package/src/docs/versioned/next/explanation/agent-ui-blocks.md +67 -0
- package/src/docs/versioned/next/explanation/ai-observability.md +101 -0
- package/src/docs/versioned/next/explanation/durable-agent-harness.md +531 -0
- package/src/docs/versioned/next/explanation/example-coverage.md +3 -1
- package/src/docs/versioned/next/explanation/loop-engineering.md +1 -1
- package/src/docs/versioned/next/explanation/roadmap-seed-issues.md +29 -1
- package/src/docs/versioned/next/explanation/typescript-configuration.md +1 -1
- package/src/docs/versioned/next/explanation/versioning-policy.md +1 -1
- package/src/docs/versioned/next/how-to/README.md +6 -0
- package/src/docs/versioned/next/how-to/agent-ui-asks.md +730 -0
- package/src/docs/versioned/next/how-to/agent-ui-blocks.md +104 -0
- package/src/docs/versioned/next/how-to/ai-feature-development.md +156 -0
- package/src/docs/versioned/next/how-to/build-a-durable-workflow.md +232 -0
- package/src/docs/versioned/next/how-to/charts-and-dashboards.md +2 -0
- package/src/docs/versioned/next/how-to/circleci.md +5 -3
- package/src/docs/versioned/next/how-to/observe-llm-calls.md +197 -0
- package/src/docs/versioned/next/how-to/run-tests-locally.md +2 -1
- package/src/docs/versioned/next/how-to/ship-a-new-task-version.md +76 -0
- package/src/docs/versioned/next/how-to/upgrade-terreno.md +1 -1
- package/src/docs/versioned/next/implementationPlans/agent-ui-asks.md +688 -0
- package/src/docs/versioned/next/implementationPlans/agent-ui-blocks.md +10 -6
- package/src/docs/versioned/next/implementationPlans/ai-observability.md +475 -157
- package/src/docs/versioned/next/implementationPlans/durable-agent-harness.md +340 -0
- package/src/docs/versioned/next/implementationPlans/terreno-langfuse-integration.md +1 -1
- package/src/docs/versioned/next/reference/README.md +3 -0
- package/src/docs/versioned/next/reference/admin-config.md +2 -0
- package/src/docs/versioned/next/reference/admin-frontend.md +216 -0
- package/src/docs/versioned/next/reference/agent-ui-asks.md +1059 -0
- package/src/docs/versioned/next/reference/ai-harness.md +1881 -0
- package/src/docs/versioned/next/reference/ai.md +251 -25
- package/src/docs/versioned/next/reference/api.md +46 -0
- package/src/docs/versioned/next/reference/blocks.md +178 -0
- package/src/docs/versioned/next/reference/environment-variables.md +1 -0
- package/src/docs/versioned/next/reference/mcp-server.md +17 -0
- package/src/docs/versioned/next/reference/test.md +11 -2
- package/src/docs/versioned/next/reference/ui.md +325 -1
- package/src/docs/versioned/next/tasks/agent-ui-asks.md +213 -0
- package/src/docs/versioned/next/tasks/agent-ui-blocks.md +2 -1
- package/src/docs/versioned/next/tasks/ai-observability.md +233 -104
- package/src/docs/versioned/next/tasks/durable-agent-harness.md +183 -0
|
@@ -0,0 +1,688 @@
|
|
|
1
|
+
# Agent UI Asks — agents ask the user for a typed answer inside the chat
|
|
2
|
+
|
|
3
|
+
**Status:** In progress — Phases 1–2 shipped for review 2026-09-28. On 2026-10-01 this plan also owns the body-component contract (the former Agent UI Blocks IP). Pick executes Phase 5 tasks B1.1–B4.2 from [`docs/tasks/agent-ui-asks.md`](../tasks/agent-ui-asks.md). Asks Tasks 3.1 and 3.2 start after B1.1 and B2.1. The design record stays in [`agent-ui-blocks.md`](./agent-ui-blocks.md). Approved 2026-09-27 (Q1–Q12 confirmed; `assumed` rows are engineering defaults Pick may revise with a note)
|
|
4
|
+
**Roadmap:** Area=`ai`, Target=`Next`, Impact=`Feature`
|
|
5
|
+
**Branch:** `cursor/agent-ui-asks-grow-a1e2`
|
|
6
|
+
**Owner:** unassigned
|
|
7
|
+
**Created:** 2026-09-26
|
|
8
|
+
**Task list:** [`docs/tasks/agent-ui-asks.md`](../tasks/agent-ui-asks.md)
|
|
9
|
+
**Linear:** none
|
|
10
|
+
**Roadmap issue:** none yet (the roadmap-reconcile workflow scaffolds the entry from the `Roadmap:` header and opens the tracking issue when this plan merges)
|
|
11
|
+
**Related:** [`agent-ui-blocks.md`](./agent-ui-blocks.md) (display catalog, `@terreno/blocks`, [#1389](https://github.com/TerrenoLabs/terreno/issues/1389)), [`charts-and-dashboards.md`](./charts-and-dashboards.md) (chart components, [PR #1302](https://github.com/TerrenoLabs/terreno/pull/1302)), [`ai-agents-and-failover.md`](./ai-agents-and-failover.md) (defers human-in-the-loop approval), [`app-mcp-server.md`](./app-mcp-server.md) (defers MCP Apps HTML), [`infra-mcp.md`](./infra-mcp.md) (MCP elicitation precedent)
|
|
12
|
+
**Primary packages:** `@terreno/blocks` (ask schemas, validators, simple cards), `@terreno/ai` (ask tools, pause/resume, headless turn endpoints, file handling, HTML sanitizer), `@terreno/ui` (ask renderers in `GPTChat`, `SimpleAskCard`, `HtmlFrame`), `demo`, `example-frontend`, `example-backend`
|
|
13
|
+
|
|
14
|
+
## Goal
|
|
15
|
+
|
|
16
|
+
Let an agent **ask the user something and get a typed answer back in the same turn**. The
|
|
17
|
+
agent calls an ask tool: pick one, pick many, upload images or files, edit a markdown draft,
|
|
18
|
+
confirm an action, or fill a short form. `GPTChat` renders the matching `@terreno/ui`
|
|
19
|
+
control inline. The user answers. The answer returns to the agent as a validated tool
|
|
20
|
+
result and the turn continues from where it paused.
|
|
21
|
+
|
|
22
|
+
Asks are strict. Every ask kind is a Zod `.strict()` schema with hard limits, so the model
|
|
23
|
+
can only request controls the chat can render, and the user can only return values that
|
|
24
|
+
ask allows. The server validates the model's ask when it is made and the user's answer
|
|
25
|
+
when it arrives, with the same pure functions the client uses to enable its Submit button.
|
|
26
|
+
|
|
27
|
+
Every ask also comes as a **simple card**: a short text and at most three buttons (Approve /
|
|
28
|
+
Cancel, or up to three short options), each button carrying the exact answer it sends.
|
|
29
|
+
The server computes it, so a client that knows nothing about ask kinds — a SwiftUI Apple
|
|
30
|
+
Watch app, a notification, Wear OS, a Slack bot — can render the card and answer with one
|
|
31
|
+
tap. Asks that cannot shrink (file uploads, markdown edits, long forms) say so and point
|
|
32
|
+
the user to the full app. A small client can also tell the server it is small, and the
|
|
33
|
+
agent then only asks button-sized questions.
|
|
34
|
+
|
|
35
|
+
The plan also adds the one display element Agent UI Blocks ruled out — **sandboxed,
|
|
36
|
+
display-only HTML** — and maps which other `@terreno/ui` components are worth rendering in
|
|
37
|
+
a chat. Charts stay where they are already designed: Agent UI Blocks `chart` blocks on top
|
|
38
|
+
of the `LineChart` / `BarChart` / `AreaChart` / `DonutChart` components that shipped in
|
|
39
|
+
PR #1302.
|
|
40
|
+
|
|
41
|
+
## Overlap with prior plans
|
|
42
|
+
|
|
43
|
+
| Plan | Status | Already covers | This plan adds |
|
|
44
|
+
| --- | --- | --- | --- |
|
|
45
|
+
| [Agent UI Blocks](./agent-ui-blocks.md) | Folded into this plan 2026-10-01 (design record; execution is Phase 5 here) | Display catalog, whole-reply YAML, `BlocksView`, `GPTChat.uiBlocks`, button actions, `POST /gpt/actions` | Owned here as [body components](#body-components-agent-ui-blocks). Asks still add the answer path plus `html`, `callout`, `image`, and `details`. |
|
|
46
|
+
| [Charts and dashboards](./charts-and-dashboards.md) | Chart components shipped | `LineChart`, `BarChart`, `AreaChart`, `DonutChart`, `DashboardGrid` | Nothing. Charts reach the chat through Agent UI Blocks. |
|
|
47
|
+
| [AI agents and failover](./ai-agents-and-failover.md) | Draft | Server-executed tools in `Agent` | It lists human-in-the-loop approval/resume as a non-goal. Asks are that capability for chat. |
|
|
48
|
+
| [App MCP server](./app-mcp-server.md) | Draft | Runtime MCP tools, prompts, resources | It defers MCP Apps (`ui://` HTML apps). This plan's HTML is display-only and does not implement MCP Apps. |
|
|
49
|
+
| [Infra MCP](./infra-mcp.md) | Planned | MCP elicitation for write confirmation | Same answer envelope (`accept` / `decline` / `cancel`), different host (IDE MCP client vs `GPTChat`). |
|
|
50
|
+
| [GPT chat mascot](./gpt-chat-mascot.md), [consent forms](./consent-forms.md) | Complete | Empty-chat slot; signature and markdown form UX | Nothing new; renderers reuse the same `@terreno/ui` fields. |
|
|
51
|
+
| [Comms: Expo push](./comms-adapter-expo-push.md), [notification center](./notification-center.md) | Complete | Push with title, body, data, badge, sound; `notify()` fans out title and body only. Notification action buttons are an explicit non-goal. | Nothing in v1 (Q7 = A). Actionable ask notifications, which iOS forwards to Apple Watch, stay in Future Work and would extend both. |
|
|
52
|
+
|
|
53
|
+
How asks and blocks divide the work:
|
|
54
|
+
|
|
55
|
+
| Need | Mechanism | Owner |
|
|
56
|
+
| --- | --- | --- |
|
|
57
|
+
| Show data (chart, table, metric, callout, image, HTML preview) | Block in the reply document | This plan, Phase 5 (design record: Agent UI Blocks) plus `html`, `callout`, `image`, `details` (Tasks 3.1, 3.2) |
|
|
58
|
+
| Suggest a follow-up message | `actions` button with `reply` | This plan, Phase 5 |
|
|
59
|
+
| Run host code (export, open a record) | `actions` button with `callback` → `POST /gpt/actions` | This plan, Phase 5 |
|
|
60
|
+
| Get an answer the agent needs to continue | Ask tool call → pause → answer → resume | This plan |
|
|
61
|
+
| Answer from a watch, notification, or other small client | Simple card + headless turn endpoint | This plan |
|
|
62
|
+
|
|
63
|
+
## Non-Goals
|
|
64
|
+
|
|
65
|
+
- Interactive HTML (scripts, forms, a bridge that returns a result). MCP Apps parity is future work.
|
|
66
|
+
- App-defined ask kinds with custom renderers. v1 is a closed catalog; apps choose which kinds to enable.
|
|
67
|
+
- Asks outside `GPTChat` and `/gpt/prompt`: MCP elicitation for IDE clients, programmatic `AIService` asks.
|
|
68
|
+
- Camera capture, audio recording, or a desktop drag-and-drop zone beyond `FilePickerButton`.
|
|
69
|
+
- A co-editing canvas. The `markdown` ask is one round trip, not a persistent document.
|
|
70
|
+
- Changing chat attachments (`attachments` on `/gpt/prompt`) or migrating the chat to the AI SDK `useChat` UI message stream.
|
|
71
|
+
- More than one pending ask per conversation.
|
|
72
|
+
- A native Apple Watch app or actionable push notifications (Q7). This plan ships the contract those clients consume; building them stays in Future Work.
|
|
73
|
+
- Finishing long agent turns in a background job after a headless answer. The headless endpoint waits for the turn (see D24).
|
|
74
|
+
- Collecting secrets. There is no password field, and the prompt forbids asking for passwords, card numbers, or API keys.
|
|
75
|
+
|
|
76
|
+
## Approaches considered
|
|
77
|
+
|
|
78
|
+
| # | Approach | How the answer returns | Strengths | Costs | Verdict |
|
|
79
|
+
| --- | --- | --- | --- | --- | --- |
|
|
80
|
+
| A | **Client-side ask tools on the existing `/gpt/prompt` SSE.** Each kind is an AI SDK `tool()` with Zod `inputSchema` + `outputSchema` and no `execute`. | The turn pauses at the tool call; the client re-posts the answer; the server resumes the turn with the answer as the tool result. | `toolCallId` binds question to answer. Provider function calling constrains the ask at generation time. Works with plain markdown replies and with `uiBlocks`. Same pattern as AI SDK client tools, MCP elicitation, AG-UI interrupts, and CopilotKit `useHumanInTheLoop`. | New resume path on `/gpt/prompt`; history must replay the paused turn exactly. | **Recommended (Q2)** |
|
|
81
|
+
| B | Same tools, but migrate chat to AI SDK `useChat` + `addToolOutput` (UI message stream). | Same | SDK-managed client tool state; `useTerrenoChat` stub already exists in `@terreno/rtk`. | Rewrites the SSE contract every host parses, `GptHistory` storage, and the Agent UI Blocks SSE events. | Rejected for v1 |
|
|
82
|
+
| C | Form blocks inside the Agent UI Blocks YAML reply (Slack Block Kit `input` + `view_submission`). | Submit posts the values as the next user message. | One wire format. | No binding to a specific question; values arrive as user text or an ad-hoc message; requires `uiBlocks`; free-text YAML is more error-prone than function calling. | Rejected |
|
|
83
|
+
| D | Interactive HTML mini-apps (MCP Apps / OpenAI Apps SDK). | `postMessage` bridge | Unlimited UI | No React Native host exists (MCP Apps specifies a web double iframe); largest phishing and exfiltration surface; the opposite of a strict catalog. | Rejected for v1 (Q3) |
|
|
84
|
+
|
|
85
|
+
## Decisions
|
|
86
|
+
|
|
87
|
+
| ID | Question | Decision | Status |
|
|
88
|
+
| --- | --- | --- | --- |
|
|
89
|
+
| D1 | How does this plan relate to Agent UI Blocks? (Q1) | One plan as of 2026-10-01. Ask schemas stay in `@terreno/blocks` under `src/asks/`. The body-component contract (whole-reply YAML, catalog, `BlocksView`, datasets, actions) is Phase 5 of this plan. `html`, `callout`, `image`, and `details` stay Tasks 3.1 and 3.2 (D3, D7). The original design record remains [`agent-ui-blocks.md`](./agent-ui-blocks.md). | **confirmed** (Q1 = A, revised 2026-10-01) |
|
|
90
|
+
| D2 | How does the answer get back to the agent? (Q2) | Approach A: client-side ask tools, pause at the tool call, resume via `POST /gpt/prompt` with `askResponse` (chat) or the `turn` action (small clients, D24). | **confirmed** (Q2 = A) |
|
|
91
|
+
| D3 | Full HTML? (Q3) | Display-only `html` block, opt-in per app (`uiBlocks.html: true` on the server, `allowHtml` on the client). Server-sanitized, rendered in a sandboxed iframe (web) or a JavaScript-disabled WebView (native), no scripts, no network, no links, at most 100 KB. | **confirmed** (Q3 = A) |
|
|
92
|
+
| D4 | Which ask kinds ship in v1? (Q4) | `choice` (one or many, optional "Other"), `files` (images and documents), `markdown` (edit a draft), `confirm` (approve/deny, destructive style), `form` (1–8 flat fields). Rating scale and signature are deferred. | **confirmed**: `choice`, `files`, `markdown` (original request), `confirm` (follow-up: "simple approve/cancel buttons"), `form` (Q4 = form). Rating scale and signature **deferred** (Q4). |
|
|
93
|
+
| D5 | Where do uploaded files live? (Q5) | Adaptive. When `FileStorageService` is configured, the client uploads through `POST /files/upload` and the answer carries `fileId`s (owner-checked `FileAttachment`). Otherwise the answer carries data URLs, like chat attachments today. | **confirmed** (Q5 = A) |
|
|
94
|
+
| D6 | Can the user type a message while an ask is pending? (Q8) | Yes. Sending a message stores a `cancel` answer for the pending ask, then appends the message, so the model sees both. | **confirmed** (Q8 = A) |
|
|
95
|
+
| D7 | Which extra display blocks join the catalog with `html`? (Q9) | `callout` (`Banner`, not dismissible), `image` (`Image`, `alt` required; allowed sources per D27), `details` (`Accordion`). | **confirmed** (Q9 = A) |
|
|
96
|
+
| D8 | Can an ask show blocks (chart, table, HTML) above its control? (Q11) | Not in v1. The ask `prompt` is plain text. Block "context" on asks is future work. | **confirmed** (Q11 = A, with the shared understanding) |
|
|
97
|
+
| D27 | Which image URLs may the `image` block load? (Q12) | File refs and `data:` URLs always; `https` only from hosts the app lists in `uiBlocks.imageHosts` (default empty), otherwise `IMAGE_HOST_NOT_ALLOWED`. An agent that was fed hostile content cannot leak conversation data through image URLs it controls. Markdown images in `text` blocks keep today's `MarkdownView` behavior. | **confirmed** (Q12 = A, with the shared understanding) |
|
|
98
|
+
| D9 | Can a host tool require an approval the server enforces? (Q10) | Yes, through AI SDK tool approval. A host tool sets `needsApproval` (boolean or a function of its input). The server turns each approval request into a server-made `confirm` ask with Approve / Deny, so the model cannot skip it and a watch can answer it. Details in [Server-enforced approval](#server-enforced-approval-q10). | **confirmed** (Q10 = A) |
|
|
99
|
+
| D21 | How do small screens (Apple Watch, notifications, voice, Slack) render asks? (Q6) | The server derives a **simple card** for every ask with a pure function: text ≤ 140 chars and up to 3 buttons, each carrying its exact answer. Asks that cannot shrink set `handoff: true` ("Continue on your phone"). Small clients also send `surface: "compact"` so the agent only asks button-sized questions (D25). | **confirmed** (Q6 = A) |
|
|
100
|
+
| D22 | How much Apple Watch support ships in this plan? (Q7) | Protocol-ready only: simple cards, a pending-asks list, and a non-streaming turn endpoint any native client can call, plus a watch-sized preview in the demo and a SwiftUI client sketch in the how-to. Actionable push notifications and a sample watchOS app stay in Future Work; no follow-up plan is committed. | **confirmed** (Q7 = "just A") |
|
|
101
|
+
| D28 | Phase 3 needs the block schema and renderer. Ship Phases 1–2 now, build those prerequisites here, or leave them on a separate blocks branch? (Q13) | Revised 2026-10-01: this plan implements the full body-component IP. Tasks B1.1–B4.2 are the former Agent UI Blocks tasks. Tasks 3.1 and 3.2 stay here and start after B1.1 and B2.1. Phases 1–2 already shipped. | **confirmed** (user directive 2026-10-01; supersedes Q13 = A) |
|
|
102
|
+
| D10 | Tool shape | One tool per kind: `ask_choice`, `ask_confirm`, `ask_markdown`, `ask_form`, `ask_files`. Each input root is an object (OpenAI strict mode and Gemini function declarations require it). Every schema is `.strict()`. Host tools whose names start with `ask_` fail at startup. | assumed |
|
|
103
|
+
| D11 | Pause semantics | Ask tools have no `execute`, so the AI SDK step loop ends with finishReason `tool-calls`. One pending ask per history, held in `GptHistory.pendingAsk`. If one step emits two ask calls, the first becomes pending and the second gets a stored `cancel` answer with `reason: "one_ask_at_a_time"`, which the model sees on resume. Revised in Pick: two turns on one history that pause at the same time follow the same rule; `pendingAsk` is set with a conditional update only when empty, and the losing turn's ask is stored as that `cancel`. Approvals (D9) share the one slot: in a step with several approval requests, or an ask plus an approval request, the first becomes the pending ask and every extra approval request is answered `approved: false` with reason `one_ask_at_a_time`, so the tool never runs unapproved and the model can request it again later. | assumed |
|
|
104
|
+
| D12 | Answer envelope | `{action: "accept", content}` \| `{action: "decline"}` \| `{action: "cancel", reason?}` — the MCP elicitation shape. `decline` means the user pressed Skip. `cancel` means the ask was superseded (D6) or dropped (D11). | assumed |
|
|
105
|
+
| D13 | Validation | Twice, same functions. Model side: the AI SDK checks the tool input against `inputSchema`; a `superRefine` adds semantic rules (unique ids, defaults among options, min ≤ max). Invalid input returns to the model as a tool error within `maxSteps` and is never sent to the client. User side: `validateAskResponse(input, response)` runs in `GPTChat` before Submit and on the server before resume (400 with `fields` on failure). | assumed |
|
|
106
|
+
| D14 | Resume fidelity | On pause, store the turn's AI SDK `response.messages` on `pendingAsk`. They include provider metadata such as Gemini 3 thought signatures, which the Google provider otherwise replaces with a skip-validator sentinel. Resume replays them verbatim, then appends the tool result. For completed turns, `buildMessages` includes ask call/result pairs. Non-ask tool rows stay skipped, so existing apps see no change. | assumed |
|
|
107
|
+
| D15 | SSE events | Adds `{ask: {toolCallId, kind, input, simple}}`, `{askResolved: {toolCallId, action}}`, and `pendingAsk: {toolCallId}` on `{done}`. Ask tools never emit the generic `{toolCall}` / `{toolResult}` events. | assumed |
|
|
108
|
+
| D16 | Files reach the model | The tool's `toModelOutput` returns an AI SDK `content` output. Images become `image-data` parts and PDFs `file-data` parts; `@ai-sdk/google` 3.0.122 forwards both as multimodal function-response parts. Text, CSV, and JSON files become text parts, truncated at 100 KB with a note. MIME type is sniffed from the bytes and must match the declared type and the ask's `accept` list. | assumed |
|
|
109
|
+
| D17 | HTML render safety | Details in [HTML block](#html-block-display-only). In short: server sanitizer allowlist; CSP `default-src 'none'`; iframe `sandbox=""`; WebView with JavaScript and navigation off; fixed height; a visible "Agent-generated preview" label; rendered only from the completed, sanitized document. | assumed |
|
|
110
|
+
| D18 | Quick replies | A `choice` with `select: one` whose options fit a simple card (at most 3, labels ≤ 20 chars) renders in `GPTChat` as that card's buttons, submitting on tap, so phone and watch show the same buttons. | assumed |
|
|
111
|
+
| D19 | Logging | `AIRequest.metadata.ask = {kind, toolCallId, phase: "asked" \| "answered", action?}`. `requestType` stays `general` because both halves are chat generations. | assumed |
|
|
112
|
+
| D20 | Limits | One source of truth, `ASK_LIMITS`, shared by schema, prompt, docs, and tests. Values in [Limits](#limits). | assumed |
|
|
113
|
+
| D23 | Simple card contract | `toSimpleCard(toolCallId, input)` in `@terreno/blocks`, rules in [Simple cards](#simple-cards-for-small-screens). At most 3 buttons: Android shows 3 notification actions, iOS banners show 2 of up to 4. Labels ≤ 20 chars. Every button's `response` must pass `validateAskResponse` (property test over all fixtures). Computed once when the ask is made, stored on `pendingAsk.simple`, sent in the `{ask}` event. Revised in Pick: label-collision fallback (when the buttons cannot tell options apart, the card shows only Skip and sets `handoff`). | assumed |
|
|
114
|
+
| D24 | Headless endpoints | `modelRouter` actions on `/gpt/histories`: collection action `GET pendingAsks` (the caller's pending asks with simple cards) and instance action `POST :id/turn` (non-streaming; body is one of `{prompt}`, `{askResponse}`, `{toolCallId, buttonId}`, plus optional `surface`). Returns `{text, pendingAsk?}` when the turn finishes. `/gpt/prompt` (SSE) and `turn` share one turn runner. A turn finishes and is saved even if the client disconnects. Revised in Pick: both actions exist only with asks on, so a host that never turned asks on gets no new endpoints after upgrading. `AiApp` adds them when its `asks` option is set; `addGptHistoryRoutes` adds them when its `chat` option, the options given to `addGptRoutes`, turns `asks` on. `turn` returns `{historyId, text, title?, pendingAsk?, error?}`; a model error after the turn starts returns 200 with `error` and the text so far. | assumed |
|
|
115
|
+
| D25 | Compact surface mode | `surface: "compact"` on `/gpt/prompt` or `turn` offers only `ask_confirm` and a narrowed `ask_choice` (`select: one`, 2–3 options, labels ≤ 20), and appends one prompt line: keep replies to two short sentences and ask only yes/no or up to three short options. | assumed |
|
|
116
|
+
| D26 | Button order and safety | Primary answer first, Cancel last. A `destructive` confirm marks its approve button `destructive`, so Apple Watch Double Tap, which triggers the first non-destructive action, can never approve it. | assumed |
|
|
117
|
+
|
|
118
|
+
## Architecture
|
|
119
|
+
|
|
120
|
+
```
|
|
121
|
+
MODEL
|
|
122
|
+
system prompt ◄── TERRENO_ASKS_SYSTEM_PROMPT(enabled kinds, limits) @terreno/ai
|
|
123
|
+
tools ◄── createAskTools(kinds) = {ask_choice, ask_confirm, …} (Zod from @terreno/blocks)
|
|
124
|
+
│
|
|
125
|
+
▼ tool call ask_choice({prompt, options, select})
|
|
126
|
+
AI SDK validates input (inputSchema + superRefine) ── invalid ──► tool error back to model (retry)
|
|
127
|
+
│ valid, no execute → step loop ends (finishReason: tool-calls)
|
|
128
|
+
▼
|
|
129
|
+
simple = toSimpleCard(toolCallId, input) text ≤ 140 + ≤ 3 buttons, each with its exact answer
|
|
130
|
+
│
|
|
131
|
+
POST /gpt/prompt ── SSE {ask: {toolCallId, kind, input, simple}} ── SSE {done, historyId, pendingAsk}
|
|
132
|
+
│ GptHistory.pendingAsk = {toolCallId, kind, input, simple, responseMessages}
|
|
133
|
+
│ GptHistory.prompts += {type: "tool-call", toolName: "ask_choice", ask: {status: "pending"}}
|
|
134
|
+
▼
|
|
135
|
+
┌─ Full client ─────────────────────────────┐ ┌─ Small client (watch app, notification, Slack) ─┐
|
|
136
|
+
│ GPTChat ── AskCard(kind) │ │ GET /gpt/histories/pendingAsks → simple cards │
|
|
137
|
+
│ validateAskResponse on every change │ │ renders text + buttons, knows no ask kinds │
|
|
138
|
+
│ onAskSubmit({toolCallId, response}) │ │ tap → POST /gpt/histories/:id/turn │
|
|
139
|
+
│ POST /gpt/prompt {historyId, askResponse} │ │ {toolCallId, buttonId, surface: compact} │
|
|
140
|
+
└────────────────────┬───────────────────────┘ └────────────────────┬─────────────────────────────┘
|
|
141
|
+
▼ ▼
|
|
142
|
+
shared turn runner: pendingAsk matches toolCallId? (409) · owner? (403)
|
|
143
|
+
│ buttonId → the stored button's response · validateAskResponse + file checks (400 {fields})
|
|
144
|
+
│ prompts += tool-result; ask.status = answered; pendingAsk cleared (atomic)
|
|
145
|
+
▼
|
|
146
|
+
streamText(messages = history before turn + user message + responseMessages + tool result)
|
|
147
|
+
▼
|
|
148
|
+
SSE {askResolved} → {text} / {toolCall} / {blocks} / {ask} … → {done} (full client)
|
|
149
|
+
JSON {text, pendingAsk?: {toolCallId, simple}} (small client)
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
Layers and ownership:
|
|
153
|
+
|
|
154
|
+
| Layer | Package | Owns |
|
|
155
|
+
| --- | --- | --- |
|
|
156
|
+
| Contract | `@terreno/blocks` (`src/asks/`) | Zod input and answer schemas per kind, `askResponseSchema`, `validateAskInput` (semantic rules), `validateAskResponse`, `toSimpleCard`, compact-surface schema variants, `ASK_LIMITS`, `ASK_ERROR_CODES`, `askPromptSection(kinds, surface)`, fixtures, exported JSON Schema for native clients; `html` block schema |
|
|
157
|
+
| Producer | `@terreno/ai` | `asks` route option, `createAskTools`, `TERRENO_ASKS_SYSTEM_PROMPT`, the shared turn runner behind `/gpt/prompt` (SSE) and `turn` (JSON), `pendingAsks` and `turn` actions on `/gpt/histories`, `GptHistory.pendingAsk`, `buildMessages` ask pairs, file resolution + MIME sniffing + `toModelOutput`, HTML sanitizer, logging |
|
|
158
|
+
| Renderer | `@terreno/ui` | `AskCard` and one renderer per kind, `SimpleAskCard` (renders any simple card), `GPTChat` ask props, answered summaries, `HtmlFrame`, `html` block renderer in `BlocksView` |
|
|
159
|
+
| Proof | `demo`, `example-*` | `AskCard`, `SimpleAskCard` (watch-sized frame), and `HtmlFrame` stories, example AI tab wiring, e2e with mocked SSE, supertest for the headless endpoints |
|
|
160
|
+
|
|
161
|
+
## The ask catalog (v1, recommended)
|
|
162
|
+
|
|
163
|
+
Every ask input shares these fields:
|
|
164
|
+
|
|
165
|
+
| Field | Type | Rule |
|
|
166
|
+
| --- | --- | --- |
|
|
167
|
+
| `prompt` | string | Required, 1–500 chars, plain text (no markdown, no clickable links) |
|
|
168
|
+
| `title` | string | Optional, at most 80 chars |
|
|
169
|
+
| `submitLabel` | string | Optional, at most 24 chars, default "Submit" |
|
|
170
|
+
| `allowDecline` | boolean | Default `true` (shows Skip); `confirm` defaults to `false` because Deny is its negative answer |
|
|
171
|
+
|
|
172
|
+
### `choice` — pick one or many
|
|
173
|
+
|
|
174
|
+
```yaml
|
|
175
|
+
tool: ask_choice
|
|
176
|
+
input:
|
|
177
|
+
prompt: Which plan should I set up?
|
|
178
|
+
select: one # one | many
|
|
179
|
+
options:
|
|
180
|
+
- {id: starter, label: Starter, description: "$0, one seat"}
|
|
181
|
+
- {id: team, label: Team, description: "$20 per seat"}
|
|
182
|
+
- {id: enterprise, label: Enterprise}
|
|
183
|
+
default: [team]
|
|
184
|
+
answer: {action: accept, content: {selected: [team]}}
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
Optional: `minSelected` (default 1), `maxSelected` (default 1 for `one`, option count for
|
|
188
|
+
`many`), `allowOther` with `otherLabel` (adds a free-text entry; the answer gains `other`).
|
|
189
|
+
Rules: 2–50 options; ids match `^[a-z0-9][a-z0-9_-]{0,63}$` and are unique; `default` ⊆ ids;
|
|
190
|
+
`minSelected` ≤ `maxSelected` ≤ option count. Server rejects ids that were not offered.
|
|
191
|
+
Revised in Pick (Task 2.1): `allowOther` is allowed only with `select: "many"` ("one option or
|
|
192
|
+
your own" is `many` with `maxSelected: 1`); Other counts as one choice, so with Other on the
|
|
193
|
+
default `maxSelected` is the option count + 1 and the cap is option count + 1; `minSelected`
|
|
194
|
+
may be 0 (a `default` below it is valid input but gets no Use suggested button).
|
|
195
|
+
|
|
196
|
+
### `confirm` — approve or deny
|
|
197
|
+
|
|
198
|
+
```yaml
|
|
199
|
+
tool: ask_confirm
|
|
200
|
+
input:
|
|
201
|
+
prompt: Delete 14 completed todos? This cannot be undone.
|
|
202
|
+
confirmLabel: Delete 14 todos
|
|
203
|
+
denyLabel: Keep them
|
|
204
|
+
destructive: true
|
|
205
|
+
answer: {action: accept, content: {confirmed: true}}
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
Revised in Pick (Task 2.2): `confirm` does not take the shared `submitLabel` (its two buttons
|
|
209
|
+
are the submit). `confirmLabel` / `denyLabel` default to "Confirm" / "Cancel" and must differ
|
|
210
|
+
after trimming, defaults included (`DUPLICATE_LABEL`). With `allowDecline: true` the chat shows
|
|
211
|
+
Skip, but the simple card never does (its two buttons already cover both answers). The
|
|
212
|
+
"confirm before a tool that deletes, sends, spends, or can't be undone" guidance ships in the
|
|
213
|
+
confirm prompt rules and the `ask_confirm` tool description, only when `confirm` is enabled.
|
|
214
|
+
|
|
215
|
+
### `markdown` — edit a draft
|
|
216
|
+
|
|
217
|
+
```yaml
|
|
218
|
+
tool: ask_markdown
|
|
219
|
+
input:
|
|
220
|
+
prompt: Here is a draft announcement. Edit anything, then send it back.
|
|
221
|
+
initial: |
|
|
222
|
+
# We're live
|
|
223
|
+
Today we launched …
|
|
224
|
+
maxLength: 20000
|
|
225
|
+
answer: {action: accept, content: {markdown: "# We're live\n…", changed: true}}
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
Revised in Pick (Task 2.3): `minLength` defaults to 0 and counts the text without spaces at
|
|
229
|
+
either end; `maxLength` defaults to 20,000 and counts UTF-16 code units, like `String.length`.
|
|
230
|
+
`minLength` above `maxLength` (or above the 20,000 cap) fails with `RANGE_INVALID`. An
|
|
231
|
+
`initial` that breaks `minLength` or `maxLength` is still a valid ask, but its simple card has
|
|
232
|
+
no Approve draft button. The server checks `changed` against the text: `changed` must be true
|
|
233
|
+
exactly when `markdown` differs from `initial` (empty when unset), or the answer fails with a
|
|
234
|
+
new code, `CHANGED_MISMATCH`. Cancel shows on the simple card only when `allowDecline` is not
|
|
235
|
+
false; the chat shows Skip for the same answer. `markdown` is never offered on the compact
|
|
236
|
+
surface, and `SimpleAskCard` shows "Edit on your phone" for its handoff line.
|
|
237
|
+
|
|
238
|
+
### `form` — a few fields, one submit
|
|
239
|
+
|
|
240
|
+
```yaml
|
|
241
|
+
tool: ask_form
|
|
242
|
+
input:
|
|
243
|
+
prompt: A few details for the invoice.
|
|
244
|
+
fields:
|
|
245
|
+
- {id: company, type: text, label: Company name, required: true, maxLength: 120}
|
|
246
|
+
- {id: seats, type: number, label: Seats, min: 1, max: 500, integer: true}
|
|
247
|
+
- {id: start, type: date, label: Start date}
|
|
248
|
+
- {id: region, type: select, label: Region, options: [{id: us, label: US}, {id: eu, label: EU}]}
|
|
249
|
+
- {id: notify, type: boolean, label: Email me the invoice, default: true}
|
|
250
|
+
answer: {action: accept, content: {values: {company: Acme, seats: 12, start: "2026-10-01", region: us, notify: true}}}
|
|
251
|
+
```
|
|
252
|
+
|
|
253
|
+
Field types: `text`, `textarea`, `email`, `url`, `phone`, `number`, `date`, `time`,
|
|
254
|
+
`datetime`, `boolean`, `select`, `multiselect`. Every field has `id`, `label`, optional
|
|
255
|
+
`helperText`, `required`, and `default`, plus type rules (`minLength` / `maxLength`,
|
|
256
|
+
`min` / `max` / `integer`, `options`). Dates are ISO strings validated with Luxon. Fields are
|
|
257
|
+
flat: no nesting, no conditional fields, no password or secret type.
|
|
258
|
+
|
|
259
|
+
Revised in Pick (Task 2.4): dates are validated with Zod's ISO validators, not Luxon, because
|
|
260
|
+
`@terreno/blocks` depends only on Zod. `date` is `YYYY-MM-DD` and must be a real calendar date;
|
|
261
|
+
`time` is 24-hour `HH:mm`; `datetime` needs `Z` or an offset, so the agent never guesses a time
|
|
262
|
+
zone, and seconds are optional (`2026-10-01T09:30Z` is accepted). Every `default` must be a value its field accepts, checked with the answer rules. A blank
|
|
263
|
+
string or an empty list counts as unanswered. `phone` accepts 7–15 digits with optional `+`,
|
|
264
|
+
spaces, dots, dashes, and parentheses; the chat sends E.164 when the number parses. Limits:
|
|
265
|
+
labels 120, helper text 280, `text` 2,000, `textarea` 10,000, `select` / `multiselect` options
|
|
266
|
+
2–50. New codes `REQUIRED_FIELD`, `FIELD_TYPE_MISMATCH`, `OUT_OF_RANGE`, and `INVALID_DATE`;
|
|
267
|
+
form values also reuse `TOO_LONG`, `TOO_SHORT`, `OPTION_NOT_OFFERED`, `DUPLICATE_ID`,
|
|
268
|
+
`DEFAULT_NOT_IN_OPTIONS`, `UNKNOWN_KEY`, and `RANGE_INVALID`. The simple card offers Submit
|
|
269
|
+
defaults only when at least one field has a default and every required field has one. `form` is
|
|
270
|
+
appended after `markdown` in `ASK_KINDS`, never offered on the compact surface, and
|
|
271
|
+
`SimpleAskCard` shows "Fill it in on your phone" for its handoff line. The chat renders `number`
|
|
272
|
+
with `TextField`, not `NumberField`, because `NumberField` rejects keystrokes below `min` (so
|
|
273
|
+
"50" cannot be typed when `min` is 10) and negative numbers; the shared rules report range and
|
|
274
|
+
whole-number errors instead.
|
|
275
|
+
|
|
276
|
+
### `files` — upload images or documents
|
|
277
|
+
|
|
278
|
+
```yaml
|
|
279
|
+
tool: ask_files
|
|
280
|
+
input:
|
|
281
|
+
prompt: Upload a photo of the receipt.
|
|
282
|
+
accept: [image, pdf] # image | pdf | text | csv | json
|
|
283
|
+
minFiles: 1
|
|
284
|
+
maxFiles: 3
|
|
285
|
+
answer:
|
|
286
|
+
action: accept
|
|
287
|
+
content:
|
|
288
|
+
files:
|
|
289
|
+
- {fileId: 6710c2…, filename: receipt.jpg, mimeType: image/jpeg, size: 482113}
|
|
290
|
+
```
|
|
291
|
+
|
|
292
|
+
`accept` maps to the `/files/upload` allowlist: `image` → JPEG, PNG, GIF, WebP; `pdf`;
|
|
293
|
+
`text` → `text/plain`; `csv`; `json`. A file ref is `{fileId}` (uploaded, D5) or `{url}` (a
|
|
294
|
+
`data:` URL). The per-file cap is the host's upload cap (10 MB default); at most 10 files.
|
|
295
|
+
|
|
296
|
+
Revised in Pick (Task 2.5): `maxFiles` defaults to 10, and each `accept` value may appear once
|
|
297
|
+
(`DUPLICATE_ID`). A data URL's media type must equal the declared `mimeType` (`MIME_MISMATCH`),
|
|
298
|
+
and the server never fetches a remote URL. The byte sniffer (`sniffFileBytes`,
|
|
299
|
+
`checkAskFileBytes`) lives in `@terreno/blocks` with no new dependency: images and PDFs need their
|
|
300
|
+
signature, and text, CSV, and JSON must be UTF-8 without NUL (JSON must parse). The history and the
|
|
301
|
+
`AIRequest` log store only `{fileId?, filename, mimeType, size}` with the real byte count, never
|
|
302
|
+
`url`; the bytes reach the model only on the answering turn, and later turns replay the metadata.
|
|
303
|
+
The model output is `content`: the stored answer as JSON text, then per file a
|
|
304
|
+
`File i of n: name (mime, N bytes)` line and its part, with
|
|
305
|
+
`[The file is cut to its first N of M bytes.]` after a cut text file. To support `fileId`,
|
|
306
|
+
`POST /files/upload` now returns the attachment `id`, `FileStorageService` gains
|
|
307
|
+
`download(gcsKey)`, `GptRouteOptions` gains `fileStorageService` (`AiApp` passes it only when
|
|
308
|
+
`gcsBucket` is set; without it a `fileId` fails with `FILE_NOT_OWNED`), and
|
|
309
|
+
`asks.maxFileSizeBytes` sets both the answer cap and the `/files/upload` cap. How picked files
|
|
310
|
+
become refs is the host's choice: `GPTChat`'s `resolveAskFiles` defaults to data URLs, and the
|
|
311
|
+
example app uploads through `/files/upload` and falls back to data URLs on 404. `SelectedFile`
|
|
312
|
+
gains `size`, and `FilePickerButton` gains `documentTypes`, `includeImages`, and `text`. The
|
|
313
|
+
Submit label defaults to "Submit". The simple card has only Skip (when `allowDecline` is not
|
|
314
|
+
false) with `handoff: true`, `files` is never offered on the compact surface, and `SimpleAskCard`
|
|
315
|
+
shows "Upload on your phone". Known gap: an upload over the cap fails in multer's error handler,
|
|
316
|
+
not with a clean 413.
|
|
317
|
+
|
|
318
|
+
## Server-enforced approval (Q10)
|
|
319
|
+
|
|
320
|
+
Confirmed (Q10 = A). A prompt-driven `confirm` depends on the model remembering to ask.
|
|
321
|
+
Server-enforced approval does not: the host marks a tool, and the tool cannot run until the
|
|
322
|
+
user approves. AI SDK 6 already supports it (`needsApproval` on `tool()`, a
|
|
323
|
+
`tool-approval-request` part on the assistant message, a `tool-approval-response` part on
|
|
324
|
+
the tool message), so this reuses the ask pause, the Approve / Deny simple card, and the
|
|
325
|
+
headless `turn` endpoint.
|
|
326
|
+
|
|
327
|
+
```ts
|
|
328
|
+
const deleteCompletedTodos = tool({
|
|
329
|
+
description: "Delete the user's completed todos",
|
|
330
|
+
inputSchema: z.object({}).strict(),
|
|
331
|
+
needsApproval: true,
|
|
332
|
+
execute: async () => ({deleted: 14}),
|
|
333
|
+
});
|
|
334
|
+
|
|
335
|
+
addGptRoutes(router, {
|
|
336
|
+
aiService,
|
|
337
|
+
tools: {deleteCompletedTodos},
|
|
338
|
+
asks: {
|
|
339
|
+
approvals: {
|
|
340
|
+
deleteCompletedTodos: () => ({
|
|
341
|
+
prompt: "Delete all completed todos?",
|
|
342
|
+
confirmLabel: "Delete",
|
|
343
|
+
denyLabel: "Keep them",
|
|
344
|
+
destructive: true,
|
|
345
|
+
}),
|
|
346
|
+
},
|
|
347
|
+
},
|
|
348
|
+
});
|
|
349
|
+
```
|
|
350
|
+
|
|
351
|
+
| Step | Behavior |
|
|
352
|
+
| --- | --- |
|
|
353
|
+
| Request | The model calls the tool; the SDK emits `tool-approval-request {approvalId, toolCallId}` and does not execute. |
|
|
354
|
+
| Pause | The server stores a pending `confirm` ask with `origin: "approval"`, `toolName`, and `approvalId`. Its input comes from `asks.approvals[toolName](input)`, or defaults to "Allow <toolName>?" with the tool description and Allow / Deny labels. The `{ask}` event and simple card carry it like any other ask. |
|
|
355
|
+
| Answer | `{confirmed: true}` appends `tool-approval-response {approvalId, approved: true}` and resumes; the SDK runs the tool once. `{confirmed: false}`, `decline`, or `cancel` appends `approved: false`; the model is told the user denied it. |
|
|
356
|
+
| Integrity | The approval request and response live only in server-stored messages, so a client cannot forge an approval; the client only sends the answer envelope for the pending ask. |
|
|
357
|
+
| Later turns | The approval row is display-only, like other server tool rows today (D14). |
|
|
358
|
+
|
|
359
|
+
Revised in Pick (Task 2.6):
|
|
360
|
+
|
|
361
|
+
- The approval ask's `toolCallId` is the SDK `approvalId`. `pendingAsk` stores it again as
|
|
362
|
+
`approvalId`, and the resume answers only that approval, never an id from the client.
|
|
363
|
+
- Because approval rows and host tool rows are both skipped on replay, a `prompt` that cancels a
|
|
364
|
+
pending approval leaves no trace of the call for the model: it sees only the user's messages.
|
|
365
|
+
The paused step's messages are dropped with the pending ask, as for a model ask.
|
|
366
|
+
- Extra approvals in a step get no rows at pause time. On resume they are denied with
|
|
367
|
+
`one_ask_at_a_time`, which stores and streams `{approved: false, reason}` tool results.
|
|
368
|
+
- Approvals work whenever asks are on, even when the surface's offered kinds leave out
|
|
369
|
+
`confirm`. With asks off, a `needsApproval` tool never runs: the SDK stops the step and the
|
|
370
|
+
server logs a warning.
|
|
371
|
+
- `decline` is accepted only when the approval input sets `allowDecline: true`, because a
|
|
372
|
+
`confirm` refuses Skip by default. Denial reasons are `user_denied`, `user_declined`, and
|
|
373
|
+
`user_cancelled` (or the cancel's own reason).
|
|
374
|
+
- The `{ask}` event and `AIRequest.metadata.ask` carry `origin` and `toolName`, but the headless
|
|
375
|
+
summaries (`turn` and `pendingAsks`) do not, so the published JSON Schemas are unchanged.
|
|
376
|
+
- The approval route tests live in a new `ai/src/routes/gptApprovals.test.ts` instead of
|
|
377
|
+
`gpt.test.ts` and `gptHistories.test.ts`; the example tool lives in
|
|
378
|
+
`example-backend/src/api/todoTools.ts`, and the keyless demo agent calls it for "delete my
|
|
379
|
+
completed todos".
|
|
380
|
+
|
|
381
|
+
## Simple cards for small screens
|
|
382
|
+
|
|
383
|
+
Every ask carries a simple card, the lowest common denominator a client can render
|
|
384
|
+
without knowing ask kinds. It plays the role of Slack's `text` fallback, Adaptive Cards'
|
|
385
|
+
`fallbackText`, and Assistant suggestion chips, with one addition: each button already
|
|
386
|
+
holds the exact answer it sends.
|
|
387
|
+
|
|
388
|
+
```yaml
|
|
389
|
+
simple:
|
|
390
|
+
toolCallId: call_8f2c1
|
|
391
|
+
kind: confirm
|
|
392
|
+
title: Clean up todos # ≤ 40 chars
|
|
393
|
+
text: Delete 14 completed todos? This cannot be undone. # ≤ 140 chars, plain text
|
|
394
|
+
handoff: false # true → "Continue on your phone"
|
|
395
|
+
buttons: # 0–3
|
|
396
|
+
- id: approve
|
|
397
|
+
label: Delete 14 todos # ≤ 20 chars
|
|
398
|
+
style: destructive # default | primary | destructive | cancel
|
|
399
|
+
response: {action: accept, content: {confirmed: true}}
|
|
400
|
+
- id: deny
|
|
401
|
+
label: Keep them
|
|
402
|
+
style: cancel
|
|
403
|
+
response: {action: accept, content: {confirmed: false}}
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
Derivation rules (`toSimpleCard`, pure, deterministic; labels over 20 chars are cut at a
|
|
407
|
+
word boundary with "…"; `text` is `prompt` cut to 140 chars):
|
|
408
|
+
|
|
409
|
+
| Ask | Buttons | `handoff` |
|
|
410
|
+
| --- | --- | --- |
|
|
411
|
+
| `confirm` | `confirmLabel` (style `destructive` when the ask is, else `primary`) → `{confirmed: true}`; `denyLabel` (`cancel`) → `{confirmed: false}` | never |
|
|
412
|
+
| `choice` one, at most 3 options, cut labels all different | One button per option → `{selected: [id]}`: the `default` first (style `primary`), then the rest in option order (style `default`); without a `default`, option order, all `default`. Skip (`cancel`) → `decline` last, when `allowDecline` and room | never |
|
|
413
|
+
| `choice` one, more options | With a `default` (an id array of length 1): Use "<label>" → `{selected: default}`, then Skip | yes |
|
|
414
|
+
| `choice` one, the buttons cannot tell options apart: two cut labels are the same (at most 3 options), or the cut label in Use "<label>" also fits another option (more options) | Skip → `decline` when `allowDecline`; no option or Use "<label>" button | yes |
|
|
415
|
+
| `choice` many | With a valid `default`: Use suggested → `{selected: default}`, then Skip | yes |
|
|
416
|
+
| `markdown` | Approve draft → `{markdown: initial, changed: false}` when `initial` meets the length rules; Cancel → `decline` | yes ("Edit on your phone") |
|
|
417
|
+
| `form` | Submit defaults → `{values: defaults}` when every required field has a valid default; Cancel → `decline` | yes |
|
|
418
|
+
| `files` | Skip → `decline` when `allowDecline` | yes |
|
|
419
|
+
|
|
420
|
+
Compact surface mode (D25): a small client sends `surface: "compact"` on `/gpt/prompt` or
|
|
421
|
+
`turn`. The agent then gets only `ask_confirm` and a narrowed `ask_choice` (one pick, 2–3
|
|
422
|
+
options, labels ≤ 20), plus a prompt line asking for two-sentence replies. Every card made
|
|
423
|
+
in that mode has `handoff: false`.
|
|
424
|
+
|
|
425
|
+
Headless endpoints (D24), for clients that cannot or should not parse SSE:
|
|
426
|
+
|
|
427
|
+
| Endpoint | Kind | Returns |
|
|
428
|
+
| --- | --- | --- |
|
|
429
|
+
| `GET /gpt/histories/pendingAsks` | `collectionActions.pendingAsks` (`IsAuthenticated`, caller's histories) | `[{historyId, title, toolCallId, kind, simple, created}]`, newest first |
|
|
430
|
+
| `POST /gpt/histories/:id/turn` | `instanceActions.turn` (`IsOwner`) | Body is one of `{prompt}`, `{askResponse}`, `{toolCallId, buttonId}`, plus optional `surface`. Runs the turn to completion and returns `{text, pendingAsk?: {toolCallId, kind, simple}}`. Unknown `buttonId` → 400; stale `toolCallId` → 409. |
|
|
431
|
+
|
|
432
|
+
How an Apple Watch plugs in (documented in the how-to; not built here, Q7 = A):
|
|
433
|
+
|
|
434
|
+
| Path | What it needs | Status |
|
|
435
|
+
| --- | --- | --- |
|
|
436
|
+
| SwiftUI watch app | Native watchOS target (React Native does not run on watchOS; `@bacons/apple-targets` adds one to an Expo app). The session token reaches the watch over WatchConnectivity; the app calls the two headless endpoints with `URLSession`, always with `surface: "compact"`. | Future Work |
|
|
437
|
+
| Actionable push, no watch app | iOS forwards iPhone notifications with up to 4 action buttons to the watch. Needs `categoryId` on comms push, `data` on `notify()`, templated categories (labels are fixed when the app registers a category), actions with `opensAppToForeground: false`, a small native iOS handler (expo-notifications completes before a JS `fetch` finishes), and a one-time action token per ask. | Future Work |
|
|
438
|
+
|
|
439
|
+
## Component map — what the chat can render
|
|
440
|
+
|
|
441
|
+
Asks:
|
|
442
|
+
|
|
443
|
+
| Ask | `@terreno/ui` component | Rule |
|
|
444
|
+
| --- | --- | --- |
|
|
445
|
+
| `choice` one, ≤ 3 options with labels ≤ 20 | `Button` row (the simple card's buttons) | Tap submits (D18) |
|
|
446
|
+
| `choice` one, ≤ 8 options | `RadioField` | |
|
|
447
|
+
| `choice` one, > 8 options | `SelectField` (searchable) | |
|
|
448
|
+
| `choice` many | `MultiselectField` + `TextField` for "Other" | |
|
|
449
|
+
| `confirm` | Two `Button`s | `destructive` → `variant="destructive"` |
|
|
450
|
+
| `markdown` | `MarkdownEditorField` | Edit and preview; stacks on narrow screens |
|
|
451
|
+
| `form` | `Field` dispatched by type | `TextField`, `TextArea`, `EmailField`, `PhoneNumberField`, `NumberField`, `DateTimeField`, `BooleanField`, `SelectField`, `MultiselectField` |
|
|
452
|
+
| `files` | `FilePickerButton` + `AttachmentPreview` | Image library when `accept` has `image`; document picker otherwise |
|
|
453
|
+
| Any ask, simple card | `SimpleAskCard`: `Heading`, `Text`, up to 3 `Button`s | For narrow layouts and the watch-sized demo preview; native watch apps render the same JSON themselves |
|
|
454
|
+
| Deferred | `Slider`, `ThumbsUpDownFeedback` (rating); `SignatureCaptureField` (signature); `AddressField` (needs a Maps key); `EmojiSelector` | |
|
|
455
|
+
|
|
456
|
+
Display (the reply document):
|
|
457
|
+
|
|
458
|
+
| Block | Component | Source |
|
|
459
|
+
| --- | --- | --- |
|
|
460
|
+
| `heading`, `text`, `metric`, `badge`, `divider`, `context`, `table`, `actions`, `columns`, `card` | `Heading`, `MarkdownView`, `Card`, `Badge`, `SectionDivider`, `Text`, `DataTable`, `Button` / `SegmentedControl`, `Box` | This plan, Phase 5 (BD5) |
|
|
461
|
+
| `chart` (`line`, `bar`, `area`, `donut`; inline or `ref` datasets) | `LineChart`, `BarChart`, `AreaChart`, `DonutChart` | This plan, Phase 5 (BD4, BD15). Chart components shipped in PR #1302. |
|
|
462
|
+
| `html` | `HtmlFrame` (new) | This plan (D3) |
|
|
463
|
+
| `callout`, `image`, `details` | `Banner`, `Image`, `Accordion` | This plan (D7, D27) |
|
|
464
|
+
| Not recommended | `Avatar`, `Tooltip`, `Popover`, `SelectBadge`, `TapToEdit`, `DraggableList` | Poor fit for a transcript |
|
|
465
|
+
| Missing | Progress bar (props type only), star rating, sparkline | Future components |
|
|
466
|
+
|
|
467
|
+
## HTML block (display-only)
|
|
468
|
+
|
|
469
|
+
```yaml
|
|
470
|
+
- type: html
|
|
471
|
+
title: Invoice preview
|
|
472
|
+
height: md # sm | md | lg → 240 / 400 / 640 px, scrolls inside
|
|
473
|
+
html: |
|
|
474
|
+
<h1>Invoice #1042</h1>
|
|
475
|
+
<table><tr><td>Seats</td><td>12</td></tr></table>
|
|
476
|
+
```
|
|
477
|
+
|
|
478
|
+
| Layer | Rule |
|
|
479
|
+
| --- | --- |
|
|
480
|
+
| Opt-in | Server `uiBlocks.html: true` adds `html` to the prompt section and validator; otherwise `HTML_DISABLED`. Client `allowHtml` on `GPTChat` / `BlocksView`; when false the block renders a placeholder. |
|
|
481
|
+
| Size | At most 100,000 bytes (`HTML_TOO_LARGE`). |
|
|
482
|
+
| Sanitizer (server, `sanitize-html`) | Removes `script`, `iframe`, `frame`, `object`, `embed`, `form`, `input`, `button`, `select`, `textarea`, `meta`, `base`, `link`; all `on*` attributes; every URL except `data:image/*`; `href` on anchors (text is kept). Runs on the final document; a changed document is stored and re-sent with `{replace: text}`. |
|
|
483
|
+
| Web | `<iframe sandbox="" srcdoc=… referrerpolicy="no-referrer">` (no scripts, same-origin, forms, popups, or top navigation). A CSP meta tag is injected first: `default-src 'none'; img-src data:; style-src 'unsafe-inline'; font-src data:; form-action 'none'; base-uri 'none'`. |
|
|
484
|
+
| Native | `react-native-webview` with `javaScriptEnabled={false}`, `onShouldStartLoadWithRequest` rejecting every load after the first, `incognito`, `dataDetectorTypes="none"`, no `onMessage`, no injected JavaScript. `originWhitelist` is not used as a sandbox: failing URLs would open in the system browser. |
|
|
485
|
+
| Framing | Inside a `Card` captioned "Agent-generated preview", fixed height, no fullscreen, so it cannot pass for host UI. |
|
|
486
|
+
| Streaming | Renders a placeholder until the stream completes, then renders the sanitized document. Raw model HTML never reaches a frame. |
|
|
487
|
+
|
|
488
|
+
## Models
|
|
489
|
+
|
|
490
|
+
| Model | Change | Notes |
|
|
491
|
+
| --- | --- | --- |
|
|
492
|
+
| `GptHistory` | Add `pendingAsk?: {toolCallId, kind, input (Mixed), simple (Mixed), responseMessages (Mixed), created}` | One slot enforces one pending ask. `simple` is the card snapshot taken when the ask is made, so later derivation changes never alter a card already shown on a watch. Set atomically only when empty (D11); cleared atomically on answer or cancel (`findOneAndUpdate` on `pendingAsk.toolCallId`). Additive and optional; every field has a `description` (mongoose-schema-safety). |
|
|
493
|
+
| `GptHistory.prompts[]` | Add `ask?: {kind, status: "pending" \| "answered" \| "cancelled"}` on `tool-call` rows | Drives rendering: pending → interactive card, otherwise a read-only summary. `tool-result` rows store the answer envelope in `result`. |
|
|
494
|
+
| `FileAttachment` | None | Reused for `fileId` refs (owner check). |
|
|
495
|
+
|
|
496
|
+
## APIs
|
|
497
|
+
|
|
498
|
+
| Surface | Change |
|
|
499
|
+
| --- | --- |
|
|
500
|
+
| `addGptRoutes(router, {asks?: boolean \| AsksOptions})` | `AsksOptions = {kinds?: AskKind[], maxFileSizeBytes?: number, approvals?: Record<toolName, (input) => ConfirmAskInput>}` (D9). The Agent UI Blocks `uiBlocks` option gains `html?: boolean` (D3) and `imageHosts?: string[]` (D27). When enabled, merges `createAskTools(kinds)` into the tool set and appends `TERRENO_ASKS_SYSTEM_PROMPT`. Off by default; with it off, tools, prompt, and SSE are unchanged. |
|
|
501
|
+
| `POST /gpt/prompt` body | Adds `askResponse?: {toolCallId, action, content?}` and `surface?: "full" \| "compact"` (D25). `prompt` becomes optional when `askResponse` is present (400 when both are missing). A `prompt` sent while an ask is pending records `cancel` first (D6). |
|
|
502
|
+
| `POST /gpt/prompt` responses | 400 `{fields}` for an invalid answer (no model call); 403 for another user's history; 409 when `toolCallId` is not the pending ask. |
|
|
503
|
+
| SSE | `{ask: {toolCallId, kind, input, simple}}`, `{askResolved}`, `{done: …, pendingAsk?}` (D15). `docs/reference/ai.md` gets the first complete SSE event table. |
|
|
504
|
+
| `GET /gpt/histories/pendingAsks` | New collection action (D24): the caller's pending asks with their simple cards. |
|
|
505
|
+
| `POST /gpt/histories/:id/turn` | New instance action (D24): non-streaming turn for small clients; accepts a prompt, a full answer, or a simple-card `buttonId`. |
|
|
506
|
+
| `@terreno/ai` exports | `createAskTools`, `AsksOptions`, `AskKind`, answer and simple-card types. |
|
|
507
|
+
| `@terreno/blocks` exports | Ask input and answer schemas, `askResponseSchema`, `validateAskInput`, `validateAskResponse`, `toSimpleCard`, `simpleCardSchema`, `askPromptSection`, `ASK_LIMITS`, `ASK_ERROR_CODES`, types, a JSON Schema file for native clients (Swift `Codable` types can be generated from it); `html` block schema. |
|
|
508
|
+
|
|
509
|
+
Error shape matches Agent UI Blocks: `{path, code, message, fix}`. Codes include
|
|
510
|
+
`UNKNOWN_KEY`, `DUPLICATE_ID`, `DEFAULT_NOT_IN_OPTIONS`, `RANGE_INVALID`, `OPTION_NOT_OFFERED`,
|
|
511
|
+
`SELECTION_COUNT`, `OTHER_NOT_ALLOWED`, `REQUIRED_FIELD`, `FIELD_TYPE_MISMATCH`,
|
|
512
|
+
`OUT_OF_RANGE`, `INVALID_DATE`, `TOO_LONG`, `FILE_TYPE_NOT_ACCEPTED`, `FILE_TOO_LARGE`,
|
|
513
|
+
`FILE_COUNT`, `FILE_NOT_OWNED`, `MIME_MISMATCH`, `UNKNOWN_BUTTON`, `HTML_DISABLED`,
|
|
514
|
+
`HTML_TOO_LARGE`, `IMAGE_HOST_NOT_ALLOWED`.
|
|
515
|
+
|
|
516
|
+
## Limits
|
|
517
|
+
|
|
518
|
+
| Limit | Value |
|
|
519
|
+
| --- | --- |
|
|
520
|
+
| `prompt` / `title` / labels / descriptions | 500 / 80 / 120 / 280 chars |
|
|
521
|
+
| `confirm` `confirmLabel` / `denyLabel` | 20 chars each, so they fit a simple card unchanged |
|
|
522
|
+
| `choice` options | 2–50; compact surface 2–3 with labels ≤ 20 |
|
|
523
|
+
| Simple card | `title` ≤ 40, `text` ≤ 140, 0–3 buttons, labels ≤ 20 chars |
|
|
524
|
+
| `form` fields | 1–8; `text` ≤ 2,000 chars; `textarea` ≤ 10,000; `select` options 2–50 |
|
|
525
|
+
| `markdown` `initial` and answer | ≤ 20,000 chars |
|
|
526
|
+
| `files` | ≤ 10 files; per file ≤ host cap (10 MB default); text-like files reach the model truncated at 100 KB |
|
|
527
|
+
| `html` | ≤ 100,000 bytes |
|
|
528
|
+
| Pending asks per history | 1 |
|
|
529
|
+
|
|
530
|
+
## UI
|
|
531
|
+
|
|
532
|
+
| Component | Package | Notes |
|
|
533
|
+
| --- | --- | --- |
|
|
534
|
+
| `GPTChat` | `@terreno/ui` | `GPTChatMessage` gains `ask?: {toolCallId, kind, input, status, response?}`. New props `onAskSubmit?: ({toolCallId, response}) => void \| Promise<void>`, `askErrors?: Record<toolCallId, AskFieldErrors>`, `allowHtml?: boolean`. A pending ask renders `AskCard` in the transcript and moves focus to it; answered asks render a one-line summary ("You chose: Team"); long markdown answers collapse. File answers pass `SelectedFile[]`; the host uploads or encodes them (D5). |
|
|
535
|
+
| `AskCard` + `AskChoice`, `AskConfirm`, `AskMarkdown`, `AskForm`, `AskFiles` | `@terreno/ui` (`src/asks/`) | Submit enabled only when `validateAskResponse` passes; inline field errors from the client check or server `fields`; Skip when `allowDecline`; loading state while `onAskSubmit` is pending. Lazy via `heavyOptionalExports` (pulls `MarkdownEditorField`, file picker). |
|
|
536
|
+
| `SimpleAskCard` | `@terreno/ui` (`src/asks/`) | Props `{card: SimpleCard, onPress: (button) => void, pendingButtonId?}`. Renders `title`, `text`, up to 3 buttons (`destructive` / `cancel` styles), and a "Continue on your phone" line when `handoff`. Knows nothing about ask kinds. |
|
|
537
|
+
| `HtmlFrame` | `@terreno/ui` | Web iframe / native WebView per [HTML block](#html-block-display-only); follows the `MarkdownEmbed` platform split. |
|
|
538
|
+
| Stories | `demo` | `AskCard` (every kind, pending / error / answered), `SimpleAskCard` inside a 198×242 pt frame (Apple Watch 45 mm) for every fixture, and `HtmlFrame` (sanitized sample, disabled placeholder). |
|
|
539
|
+
|
|
540
|
+
## Phases
|
|
541
|
+
|
|
542
|
+
| Phase | Slice | Proves |
|
|
543
|
+
| --- | --- | --- |
|
|
544
|
+
| 1 | Tracer: `choice` (select one) end to end, then the small-screen path | Mock-model supertest: pause → SSE `ask` → answer → resumed model call sees the tool result; `GPTChat` renders and submits; example app e2e with mocked SSE; a simple-card `buttonId` sent to `turn` resumes the same way |
|
|
545
|
+
| 2 | Remaining kinds: `choice` many + Other, `confirm`, `markdown`, `form`, `files` | One vertical slice per kind: schema, answer validation, simple-card rule, renderer, docs, tests |
|
|
546
|
+
| 3 | `html`, `callout`, `image`, and `details` blocks (after B1.1 and B2.1) | Sanitizer and frame security tests; image host rule; `BlocksView` renders; demo screenshots |
|
|
547
|
+
| 4 | Asks wrap-up for Phases 1–2 | Changelog, rules, docs indexes (shipped 2026-09-27; Tasks 3.1 and 3.2 add their own lines) |
|
|
548
|
+
| 5 | Body components: contract, renderer, producer, examples | Whole-reply YAML, `BlocksView`, `uiBlocks`, datasets, actions. See [Body components](#body-components-agent-ui-blocks). Blocks AC1–AC20 |
|
|
549
|
+
|
|
550
|
+
## Feature Flags & Migrations
|
|
551
|
+
|
|
552
|
+
None. `asks` and `uiBlocks.html` are off unless the host passes them. Schema changes are
|
|
553
|
+
additive optional fields; no backfill.
|
|
554
|
+
|
|
555
|
+
## Activity Log & User Updates
|
|
556
|
+
|
|
557
|
+
None beyond `AIRequest` metadata (D19).
|
|
558
|
+
|
|
559
|
+
## Not Included / Future Work
|
|
560
|
+
|
|
561
|
+
- Block context above an ask's control (D8): chart, table, or HTML previews such as "Send this email?".
|
|
562
|
+
- Rating scale (`Slider` / `ThumbsUpDownFeedback`), signature (`SignatureCaptureField`), address (`AddressField`).
|
|
563
|
+
- Camera capture and audio recording.
|
|
564
|
+
- App-defined ask kinds with custom renderers.
|
|
565
|
+
- MCP elicitation mapping so IDE agents on the app `/mcp` endpoint can use the same asks.
|
|
566
|
+
- Interactive HTML with a result bridge (MCP Apps).
|
|
567
|
+
- Pending-ask expiry.
|
|
568
|
+
- Actionable ask notifications (Q7 B): comms push `categoryId` / `collapseId`, `data` on `notify()`, templated iOS categories, a native iOS action handler, one-time action tokens scoped to ask + user + buttons with expiry, `isAuthenticationRequired` on destructive buttons, and a device spike on a real iPhone + Watch (locked phone, force-quit app).
|
|
569
|
+
- A sample SwiftUI watchOS app in `example-frontend` via `@bacons/apple-targets` (Q7 C), and Wear OS.
|
|
570
|
+
- Finishing long turns in a `@terreno/jobs` background job so a headless answer can return immediately.
|
|
571
|
+
|
|
572
|
+
## Files to Create / Modify
|
|
573
|
+
|
|
574
|
+
| Package | Files |
|
|
575
|
+
| --- | --- |
|
|
576
|
+
| `blocks/` | `src/asks/schema.ts`, `src/asks/limits.ts`, `src/asks/errors.ts`, `src/asks/validateInput.ts`, `src/asks/validateResponse.ts`, `src/asks/simpleCard.ts`, `src/asks/prompt.ts`, `src/asks/jsonSchema.ts`, `src/asks/fixtures/**`, `src/asks/*.test.ts`, `src/index.ts`; `src/schema.ts` (`html`, `callout`, `image`, `details` blocks); package scaffold (`package.json`, `tsconfig.json`, `biome.jsonc`, CI job) only if Agent UI Blocks Task 1.1 has not landed |
|
|
577
|
+
| `ai/` | `src/service/asks.ts` (`createAskTools`, `toModelOutput`, resume helpers), `src/service/chatTurn.ts` (turn runner shared by SSE and JSON), `src/service/askFiles.ts` (ref resolution, MIME sniffing), `src/service/sanitizeHtml.ts`, `src/service/prompts.ts` (`TERRENO_ASKS_SYSTEM_PROMPT`), `src/routes/gpt.ts`, `src/routes/gptHistories.ts` (`pendingAsks`, `turn` actions), `src/models/gptHistory.ts`, `src/service/aiService.ts` (`buildMessages`), `src/types/index.ts`, `src/index.ts`, `package.json` (`@terreno/blocks`, `sanitize-html`), tests |
|
|
578
|
+
| `ui/` | `src/asks/AskCard.tsx`, `src/asks/SimpleAskCard.tsx`, `src/asks/AskChoice.tsx`, `src/asks/AskConfirm.tsx`, `src/asks/AskMarkdown.tsx`, `src/asks/AskForm.tsx`, `src/asks/AskFiles.tsx`, `src/asks/askSummary.ts`, `src/HtmlFrame.tsx`, `src/GPTChat.tsx`, `src/lazyBoundaries/heavyOptionalExports.tsx`, `src/index.tsx`, `package.json` (`@terreno/blocks`), tests; `src/blocks/blockRenderers.tsx` (`html`, `callout`, `image`, `details`) after Agent UI Blocks Task 2.1 |
|
|
579
|
+
| `demo/` | `stories/AskCard.stories.tsx`, `story-config/AskCard.config.tsx`, `stories/SimpleAskCard.stories.tsx`, `story-config/SimpleAskCard.config.tsx`, `stories/HtmlFrame.stories.tsx`, `story-config/HtmlFrame.config.tsx`, `demoConfig.tsx` |
|
|
580
|
+
| `example-backend/` | `src/api/ai.ts` (`asks: true`, later `uiBlocks.html`) |
|
|
581
|
+
| `example-frontend/` | `app/(tabs)/ai.tsx`, `e2e/helpers/mockGpt.ts`, `e2e/ai-chat.spec.ts` |
|
|
582
|
+
| root | `package.json` (catalog `sanitize-html`), `knip.jsonc`, `changelog/unreleased/agent-ui-asks.md` |
|
|
583
|
+
| docs | `docs/explanation/agent-ui-asks.md` (new), `docs/reference/agent-ui-asks.md` (new), `docs/how-to/agent-ui-asks.md` (new), `docs/reference/ai.md`, `docs/reference/ui.md`, `docs/explanation/example-coverage.md`, `docs/explanation/README.md`, `docs/reference/README.md`, `docs/how-to/README.md`, `.rulesync/rules/ai/00-ai.md`, `.rulesync/rules/ui/00-ui.md` |
|
|
584
|
+
|
|
585
|
+
## Body components (Agent UI Blocks)
|
|
586
|
+
|
|
587
|
+
On 2026-10-01 the user directed Pick to roll the body-component IP into this plan.
|
|
588
|
+
[`agent-ui-blocks.md`](./agent-ui-blocks.md) stays the design record (goal, approaches,
|
|
589
|
+
grammar examples, architecture diagram, models, and the original AC wording). This
|
|
590
|
+
section is the execution source: decisions BD1–BD20 are that plan's D1–D20, and tasks
|
|
591
|
+
B1.1–B4.2 in [`docs/tasks/agent-ui-asks.md`](../tasks/agent-ui-asks.md) are that plan's
|
|
592
|
+
tasks. `@terreno/blocks` already exists for asks; Phase 5 extends it. It does not
|
|
593
|
+
create a second package.
|
|
594
|
+
|
|
595
|
+
When `uiBlocks` is on, every assistant reply is one YAML document: `v`, then optional
|
|
596
|
+
`datasets`, then `blocks`. Prose is a `text` block. The agent can only emit the closed
|
|
597
|
+
catalog. Charts bind to named datasets (`inline` ≤ 500 rows, or `ref` to an `AIDataset`
|
|
598
|
+
handle a tool returned). Buttons use `reply`, `open`, `select`, or `callback`.
|
|
599
|
+
|
|
600
|
+
| ID | Decision | Status |
|
|
601
|
+
| --- | --- | --- |
|
|
602
|
+
| BD1 | Own the grammar (Approach A). Contract lives in `@terreno/blocks`. | confirmed |
|
|
603
|
+
| BD2 | The whole assistant reply is one YAML document (`v` → `datasets` → `blocks`). A leading or trailing fence is stripped. JSON is accepted. A non-document is wrapped as one `text` block and reported as `NOT_A_DOCUMENT`. | confirmed |
|
|
604
|
+
| BD3 | Nested, depth ≤ 2 (`blocks` → `columns`/`card` → leaf). Ids optional except on interactive blocks. | assumed |
|
|
605
|
+
| BD4 | Datasets are `inline` (typed columns + rows, ≤ 500) or `ref` (`{source: ref, id, grain?, limit?}`). Charts and tables use `data` plus column names. One-off charts may use `points`. | confirmed |
|
|
606
|
+
| BD5 | v1 catalog: `heading`, `text`, `metric`, `badge`, `divider`, `context`, `chart`, `table`, `actions`, `columns`, `card`. Asks Tasks 3.1 and 3.2 add `html`, `callout`, `image`, `details`. | assumed |
|
|
607
|
+
| BD6 | Actions: `reply`, `open`, `select`, `callback`. `callback` posts `POST /gpt/actions`; hosts register `hostActions` with a Zod payload. Unknown names fail `UNKNOWN_HOST_ACTION`. | confirmed |
|
|
608
|
+
| BD7 | Unknown keys fail. ≤ 50 blocks, depth ≤ 2, ≤ 25 action elements, ≤ 4 columns, ≤ 8 datasets, inline ≤ 500×12, `ref` limit ≤ 1,000 rendered rows (≤ 60 bar/donut categories), ≤ 8 donut slices, text ≤ 4,000 per block, ≤ 20,000 per document. | assumed |
|
|
609
|
+
| BD8 | Contract package is the existing `blocks/` (`zod` catalog, `yaml` catalog). No React, no Express. | confirmed |
|
|
610
|
+
| BD9 | Styling is semantic enums already on `@terreno/ui` (`size`, `status`, `variant`, `trend`). No hex, no spacing numbers. | assumed |
|
|
611
|
+
| BD10 | `parseBlocksPartial` returns completed top-level blocks while streaming; `GPTChat` shows a `Spinner` for the tail. Full validation replaces that render on `{done}`. | assumed |
|
|
612
|
+
| BD11 | One `validateBlocks`: post-stream SSE `{blocks}`, optional one repair pass, MCP `terreno_validate_ui_blocks`, `AIService.generateBlocks`, CLI `terreno-blocks validate`. | assumed |
|
|
613
|
+
| BD12 | Errors are `{path, code, message, fix}` sorted by path. Warnings share the shape and do not block rendering. | assumed |
|
|
614
|
+
| BD13 | Required `v: 1`. Unknown major → `UNSUPPORTED_VERSION`. | assumed |
|
|
615
|
+
| BD14 | Demo `BlocksView` story and Blocks Playground; example AI tab turns `uiBlocks` on; example-backend registers one host action and one dataset tool. | assumed |
|
|
616
|
+
| BD15 | `chart` targets the shipped single-series chart components (PR #1302). Multi-series stays rejected. | confirmed |
|
|
617
|
+
| BD16 | One document per assistant message, stored as the raw string. | assumed |
|
|
618
|
+
| BD17 | Only server tools call `registerAiDataset`. The agent cannot mint handles. `GET /gpt/datasets/:id` is `IsOwner`. | confirmed |
|
|
619
|
+
| BD18 | `AIDataset` retention defaults to unlimited (`datasetTtlDays: 0`). Cap `datasetMaxRows` default 50,000. `grain` buckets dates; line/area over `limit` uses LTTB. | confirmed |
|
|
620
|
+
| BD19 | `POST /gpt/actions` body `{historyId, messageId, blockId, elementId, name, payload}`; owner check; 10 s handler timeout; `requestType: "ui_action"`. | assumed |
|
|
621
|
+
| BD20 | No prose before tool calls. Only the final step's text is the document. | assumed |
|
|
622
|
+
|
|
623
|
+
Block reference (field tables land in `docs/reference/blocks.md` with B1.1 and later tasks):
|
|
624
|
+
|
|
625
|
+
| Block | Required | Renders | Task |
|
|
626
|
+
| --- | --- | --- | --- |
|
|
627
|
+
| `heading`, `text`, `metric`, `badge`, `divider`, `context`, `columns`, `card` | See design record | `@terreno/ui` equivalents | B1.1 schema, B2.1 renderer |
|
|
628
|
+
| `chart`, `table`, datasets | `kind` + `data` or `points`; `data` for tables | Chart components, `DataTable` | B1.2, B2.2 |
|
|
629
|
+
| `actions` | `id`, `elements` | `Button`, `SegmentedControl` | B1.3, B2.3 |
|
|
630
|
+
| `html`, `callout`, `image`, `details` | Asks D3, D7, D27 | `HtmlFrame`, `Banner`, `Image`, `Accordion` | Tasks 3.1, 3.2 after B1.1 and B2.1 |
|
|
631
|
+
|
|
632
|
+
`parseBlocks` strips an optional fence, parses YAML or JSON (`schema: "core"`, unique keys, no anchors or custom tags), and returns `NOT_A_DOCUMENT` when the value is not a mapping with `v`. `wrapAsTextDocument` builds the display fallback. `validateBlocks` is pure and sync: Zod `strict()` structure, then semantic lint (lint starts in B1.2).
|
|
633
|
+
|
|
634
|
+
### Body component acceptance
|
|
635
|
+
|
|
636
|
+
These ids are **Blocks AC**. They do not renumber the asks AC table below.
|
|
637
|
+
|
|
638
|
+
| # | Criterion | Verification |
|
|
639
|
+
| --- | --- | --- |
|
|
640
|
+
| Blocks AC1 | Every fixture under `blocks/src/fixtures/valid/` validates with zero errors; every fixture under `invalid/` fails with exactly the expected `{path, code}` pairs | `bun test blocks/` golden test |
|
|
641
|
+
| Blocks AC2 | Unknown keys, wrong enums, wrong key order, missing dataset/column refs, non-document input, and each BD7 limit produce a distinct documented error code | `blocks/src/validate.test.ts`; `docs/reference/blocks.md` lists the same codes |
|
|
642
|
+
| Blocks AC3 | `validateBlocks` on a 50-block, 8-dataset document completes in < 5 ms median | `blocks/src/validate.perf.test.ts` |
|
|
643
|
+
| Blocks AC4 | `blocksJsonSchema` accepts and rejects the same fixtures as the Zod schema | ajv round-trip |
|
|
644
|
+
| Blocks AC5 | `BlocksView` renders every v1 block type with `@terreno/ui` components only | `BlocksView.test.tsx` + `rg` guard |
|
|
645
|
+
| Blocks AC6 | Invalid document → error `Banner` and collapsed raw YAML; non-document text → one `text` block | `BlocksView.test.tsx` |
|
|
646
|
+
| Blocks AC7 | `reply` calls `onSubmit`; `open` reaches `onBlockAction`; `callback` reaches `onBlockCallback` | `GPTChat.test.tsx` |
|
|
647
|
+
| Blocks AC8 | `segmented` switches the target chart's dataset and reports `select` | `BlocksView.test.tsx` |
|
|
648
|
+
| Blocks AC9 | `/gpt/prompt` with `uiBlocks` appends the prompt, treats the final step as the document, and emits SSE `{blocks}` before `{done}`; `repair: true` runs one repair call | `ai/src/routes/gpt.test.ts` |
|
|
649
|
+
| Blocks AC10 | `generateBlocks` returns a validated document; one repair call includes the error codes | `aiService.test.ts` |
|
|
650
|
+
| Blocks AC11 | MCP `terreno_validate_ui_blocks` matches the CLI report | mcp-server test + CLI test |
|
|
651
|
+
| Blocks AC12 | Demo `BlocksView` story and Blocks Playground; demo coverage passes | CI + screenshots |
|
|
652
|
+
| Blocks AC13 | example-frontend renders a bar chart from a mocked document and a follow-up reply | e2e + recording |
|
|
653
|
+
| Blocks AC14 | Explanation, reference, and how-to pages exist and are linked; `ai.md`, `ui.md`, `mcp-server.md` updated | `bun run website:build` |
|
|
654
|
+
| Blocks AC15 | `bun run prepush` green | CI |
|
|
655
|
+
| Blocks AC16 | `GET /gpt/datasets/:id` buckets, downsamples, and paginates; TTL 0 vs 7 | dataset tests |
|
|
656
|
+
| Blocks AC17 | `POST /gpt/actions` rejects unknown name, bad payload, and non-owner; a valid call returns `{blocks, replace}` | `gptActions.test.ts` |
|
|
657
|
+
| Blocks AC18 | Over-cap `ref` limit → `TOO_MANY_POINTS`; unregistered callback → `UNKNOWN_HOST_ACTION` | lint + gpt tests |
|
|
658
|
+
| Blocks AC19 | Callback button shows `loading`; `replace: "block"` replaces that block; `{text}` appends a message | `GPTChat.test.tsx` |
|
|
659
|
+
| Blocks AC20 | Streaming renders completed top-level blocks plus a `Spinner` tail | `parsePartial.test.ts`, `GPTChat.test.tsx` |
|
|
660
|
+
|
|
661
|
+
## Task List
|
|
662
|
+
|
|
663
|
+
[`docs/tasks/agent-ui-asks.md`](../tasks/agent-ui-asks.md)
|
|
664
|
+
|
|
665
|
+
## Acceptance Criteria
|
|
666
|
+
|
|
667
|
+
| # | Criterion | Verification |
|
|
668
|
+
| --- | --- | --- |
|
|
669
|
+
| AC1 | Every fixture under `blocks/src/asks/fixtures/valid/` passes; every fixture under `invalid/` fails with exactly the expected `{path, code}` pairs (unknown keys, over-limit strings and arrays, duplicate ids, defaults not among options, `minSelected` > `maxSelected`) | `bun test blocks/src/asks` golden test |
|
|
670
|
+
| AC2 | `validateAskResponse` returns a distinct documented code for each answer error: ids not offered, count out of bounds, Other when not allowed, required field missing, wrong field type, out of range, invalid ISO date, text too long, file type not accepted, too large, too many, not owned, MIME mismatch | One test per code; doc-code parity test against `docs/reference/agent-ui-asks.md` |
|
|
671
|
+
| AC3 | With `asks: true`, a mock model that calls `ask_choice` produces SSE `{ask: {toolCallId, kind: "choice", input, simple}}` then `{done: true, historyId, pendingAsk: {toolCallId}}`; `GptHistory.pendingAsk` holds the input, the simple card, and `responseMessages`; the display row has `ask.status: "pending"`; no `{toolCall}` event is sent for it | `ai/src/routes/gpt.test.ts` (supertest) |
|
|
672
|
+
| AC4 | `POST /gpt/prompt {historyId, askResponse: {toolCallId, action: "accept", content: {selected: ["team"]}}}` makes exactly one model call whose messages end with the stored `responseMessages` plus a tool result carrying the answer; SSE starts with `{askResolved}` and ends with `{done}`; `pendingAsk` is cleared and the row shows `answered` | Supertest asserting the mock `doStream` call arguments |
|
|
673
|
+
| AC5 | An invalid answer returns 400 with `fields` and makes no model call; a stale or unknown `toolCallId` returns 409; another user's history returns 403; a `prompt` sent while an ask is pending stores a `cancel` result before the new user message, and the next model call sees both | Supertest |
|
|
674
|
+
| AC6 | A model ask that fails `inputSchema` or semantic rules never reaches the client: the model receives a tool error, and a corrected ask in the next step is the only `{ask}` event | Supertest with a mock model emitting invalid then valid calls |
|
|
675
|
+
| AC7 | `buildMessages` includes ask call/result pairs from completed turns and still skips non-ask tool rows | `ai/src/service/aiService.test.ts` |
|
|
676
|
+
| AC8 | With `asks` unset, the tool set, system prompt, and SSE stream are identical to today | Regression supertest |
|
|
677
|
+
| AC9 | `GPTChat` renders each ask kind with `@terreno/ui` components only; Submit stays disabled until the answer validates; `onAskSubmit` receives `{toolCallId, response}`; server `fields` errors render inline; answered asks render a summary; a pending ask restored from history is interactive | `ui/src/asks/*.test.tsx`, `GPTChat.test.tsx`, `rg` guard test (no raw `View`/`Text` in `ui/src/asks/`) |
|
|
678
|
+
| AC10 | An image answer reaches the model as an `image-data` part and a text file as a text part ≤ 100 KB; a `fileId` owned by another user returns 400 `FILE_NOT_OWNED`; bytes that do not match the declared MIME type return 400 `MIME_MISMATCH`; both storage modes (D5) pass | Supertest + `askFiles.test.ts` |
|
|
679
|
+
| AC11 | HTML: the sanitizer strips scripts, event handlers, forms, frames, `meta`/`base`/`link`, anchor `href`s, and non-`data:image` URLs; oversize → `HTML_TOO_LARGE`; disabled → `HTML_DISABLED`; web renders `<iframe sandbox="">` whose `srcdoc` starts with the CSP meta tag; native renders a WebView with JavaScript off and navigation blocked; streaming shows a placeholder | `sanitizeHtml.test.ts`, `HtmlFrame.test.tsx`, `BlocksView.test.tsx`, demo screenshot |
|
|
680
|
+
| AC12 | example-frontend: a mocked SSE `ask` renders a choice card; selecting an option and submitting posts `askResponse`; the mocked resumed stream renders the continuation | `example-frontend/e2e/ai-chat.spec.ts` + recording under `/opt/cursor/artifacts/` |
|
|
681
|
+
| AC13 | Docs: explanation, reference, and how-to pages exist and are linked from their READMEs; `docs/reference/ai.md` lists every SSE event; `docs/reference/ui.md` documents the ask props | `bun run website:build` + doc-code parity test |
|
|
682
|
+
| AC14 | `bun run prepush` passes (lint, knip, no-barrel-imports, source rules, demo coverage) | CI |
|
|
683
|
+
| AC15 | For every valid fixture, `toSimpleCard` returns a card that passes `simpleCardSchema` (≤ 3 buttons, labels ≤ 20, text ≤ 140), follows the per-kind rule table including `handoff`, and every button's `response` passes `validateAskResponse`; a destructive confirm marks its approve button `destructive` | `blocks/src/asks/simpleCard.test.ts` (golden + property test over all fixtures) |
|
|
684
|
+
| AC16 | With `surface: "compact"`, the model is offered only `ask_confirm` and the narrowed `ask_choice`; a 4-option choice fails as a tool error; the system prompt contains the compact line; every card produced has `handoff: false` | Supertest inspecting the mock model's tools and prompt |
|
|
685
|
+
| AC17 | `GET /gpt/histories/pendingAsks` lists only the caller's pending asks with their cards; `POST /gpt/histories/:id/turn {toolCallId, buttonId}` resumes with that button's stored answer and returns `{text, pendingAsk?}`; an unknown `buttonId` returns 400 `UNKNOWN_BUTTON`; another user's history returns 403; a stale `toolCallId` returns 409; closing the request mid-turn still saves the finished turn | Supertest (the last case aborts the request, then reloads the history) |
|
|
686
|
+
| AC18 | `SimpleAskCard` renders every fixture's card inside the watch-sized demo frame with `@terreno/ui` components only; pressing a button calls `onPress` with that button; `handoff` shows the continue-on-phone line | `ui/src/asks/SimpleAskCard.test.tsx` + demo screenshot under `/opt/cursor/artifacts/` |
|
|
687
|
+
| AC19 | A host tool with `needsApproval: true` never runs before approval: the mock model's call produces an `{ask}` with `kind: "confirm"`, `origin: "approval"` and `execute` is not called; approving (via `/gpt/prompt` or `turn` with `buttonId: approve`) runs it exactly once and the model sees its result; denying, declining, or cancelling never runs it and the model sees the denial | Supertest with a spy `execute` |
|
|
688
|
+
| AC20 | `callout`, `image`, and `details` blocks validate and render with `Banner`, `Image`, and `Accordion` only; `image` requires `alt`; an `https` image whose host is not in `uiBlocks.imageHosts` fails with `IMAGE_HOST_NOT_ALLOWED` while file refs and `data:` URLs pass | `blocks` golden fixtures, `BlocksView.test.tsx`, demo screenshot |
|