@terreno/mcp 57.9.0 → 57.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/docs/upgrades/57.10.1.md +54 -0
- package/dist/docs/upgrades/57.11.0.md +80 -0
- package/dist/docs/versioned/next/README.md +1 -0
- package/dist/docs/versioned/next/explanation/README.md +4 -0
- package/dist/docs/versioned/next/explanation/admin-plugin-frontend.md +2 -2
- package/dist/docs/versioned/next/explanation/agent-ui-asks.md +299 -0
- package/dist/docs/versioned/next/explanation/agent-ui-blocks.md +67 -0
- package/dist/docs/versioned/next/explanation/ai-observability.md +101 -0
- package/dist/docs/versioned/next/explanation/durable-agent-harness.md +544 -0
- package/dist/docs/versioned/next/explanation/example-coverage.md +3 -1
- package/dist/docs/versioned/next/explanation/local-first-data.md +1 -0
- package/dist/docs/versioned/next/explanation/loop-engineering.md +1 -1
- package/dist/docs/versioned/next/explanation/roadmap-seed-issues.md +29 -1
- package/dist/docs/versioned/next/explanation/typescript-configuration.md +1 -1
- package/dist/docs/versioned/next/explanation/versioning-policy.md +1 -1
- package/dist/docs/versioned/next/how-to/README.md +7 -0
- package/dist/docs/versioned/next/how-to/agent-ui-asks.md +730 -0
- package/dist/docs/versioned/next/how-to/agent-ui-blocks.md +104 -0
- package/dist/docs/versioned/next/how-to/ai-feature-development.md +156 -0
- package/dist/docs/versioned/next/how-to/build-a-durable-workflow.md +232 -0
- package/dist/docs/versioned/next/how-to/call-external-apis.md +1 -1
- package/dist/docs/versioned/next/how-to/charts-and-dashboards.md +2 -0
- package/dist/docs/versioned/next/how-to/circleci.md +5 -3
- package/dist/docs/versioned/next/how-to/migrate-rtk-to-syncdb.md +31 -1
- package/dist/docs/versioned/next/how-to/observe-llm-calls.md +197 -0
- package/dist/docs/versioned/next/how-to/preview-demo.md +7 -3
- package/dist/docs/versioned/next/how-to/run-harness-on-multiple-instances.md +93 -0
- package/dist/docs/versioned/next/how-to/run-tests-locally.md +2 -1
- package/dist/docs/versioned/next/how-to/ship-a-new-task-version.md +76 -0
- package/dist/docs/versioned/next/how-to/upgrade-terreno.md +1 -1
- package/dist/docs/versioned/next/implementationPlans/agent-ui-asks.md +688 -0
- package/dist/docs/versioned/next/implementationPlans/agent-ui-blocks.md +10 -6
- package/dist/docs/versioned/next/implementationPlans/ai-observability.md +475 -157
- package/dist/docs/versioned/next/implementationPlans/durable-agent-harness.md +340 -0
- package/dist/docs/versioned/next/implementationPlans/terreno-langfuse-integration.md +1 -1
- package/dist/docs/versioned/next/reference/README.md +3 -0
- package/dist/docs/versioned/next/reference/admin-config.md +2 -0
- package/dist/docs/versioned/next/reference/admin-frontend.md +216 -0
- package/dist/docs/versioned/next/reference/agent-ui-asks.md +1059 -0
- package/dist/docs/versioned/next/reference/ai-harness.md +1974 -0
- package/dist/docs/versioned/next/reference/ai.md +251 -25
- package/dist/docs/versioned/next/reference/api.md +110 -2
- package/dist/docs/versioned/next/reference/blocks.md +178 -0
- package/dist/docs/versioned/next/reference/environment-variables.md +2 -1
- package/dist/docs/versioned/next/reference/lifecycle-plugin.md +11 -0
- package/dist/docs/versioned/next/reference/mcp-server.md +17 -0
- package/dist/docs/versioned/next/reference/syncdb.md +156 -6
- package/dist/docs/versioned/next/reference/test.md +11 -2
- package/dist/docs/versioned/next/reference/ui.md +331 -6
- package/dist/docs/versioned/next/tasks/agent-ui-asks.md +213 -0
- package/dist/docs/versioned/next/tasks/agent-ui-blocks.md +2 -1
- package/dist/docs/versioned/next/tasks/ai-observability.md +233 -104
- package/dist/docs/versioned/next/tasks/durable-agent-harness.md +185 -0
- package/dist/tools.d.ts +4 -0
- package/dist/tools.d.ts.map +1 -1
- package/dist/tools.js +37 -0
- package/dist/tools.js.map +1 -1
- package/package.json +3 -2
- package/src/docs/upgrades/57.10.1.md +54 -0
- package/src/docs/upgrades/57.11.0.md +80 -0
- package/src/docs/versioned/next/README.md +1 -0
- package/src/docs/versioned/next/explanation/README.md +4 -0
- package/src/docs/versioned/next/explanation/admin-plugin-frontend.md +2 -2
- package/src/docs/versioned/next/explanation/agent-ui-asks.md +299 -0
- package/src/docs/versioned/next/explanation/agent-ui-blocks.md +67 -0
- package/src/docs/versioned/next/explanation/ai-observability.md +101 -0
- package/src/docs/versioned/next/explanation/durable-agent-harness.md +544 -0
- package/src/docs/versioned/next/explanation/example-coverage.md +3 -1
- package/src/docs/versioned/next/explanation/local-first-data.md +1 -0
- package/src/docs/versioned/next/explanation/loop-engineering.md +1 -1
- package/src/docs/versioned/next/explanation/roadmap-seed-issues.md +29 -1
- package/src/docs/versioned/next/explanation/typescript-configuration.md +1 -1
- package/src/docs/versioned/next/explanation/versioning-policy.md +1 -1
- package/src/docs/versioned/next/how-to/README.md +7 -0
- package/src/docs/versioned/next/how-to/agent-ui-asks.md +730 -0
- package/src/docs/versioned/next/how-to/agent-ui-blocks.md +104 -0
- package/src/docs/versioned/next/how-to/ai-feature-development.md +156 -0
- package/src/docs/versioned/next/how-to/build-a-durable-workflow.md +232 -0
- package/src/docs/versioned/next/how-to/call-external-apis.md +1 -1
- package/src/docs/versioned/next/how-to/charts-and-dashboards.md +2 -0
- package/src/docs/versioned/next/how-to/circleci.md +5 -3
- package/src/docs/versioned/next/how-to/migrate-rtk-to-syncdb.md +31 -1
- package/src/docs/versioned/next/how-to/observe-llm-calls.md +197 -0
- package/src/docs/versioned/next/how-to/preview-demo.md +7 -3
- package/src/docs/versioned/next/how-to/run-harness-on-multiple-instances.md +93 -0
- package/src/docs/versioned/next/how-to/run-tests-locally.md +2 -1
- package/src/docs/versioned/next/how-to/ship-a-new-task-version.md +76 -0
- package/src/docs/versioned/next/how-to/upgrade-terreno.md +1 -1
- package/src/docs/versioned/next/implementationPlans/agent-ui-asks.md +688 -0
- package/src/docs/versioned/next/implementationPlans/agent-ui-blocks.md +10 -6
- package/src/docs/versioned/next/implementationPlans/ai-observability.md +475 -157
- package/src/docs/versioned/next/implementationPlans/durable-agent-harness.md +340 -0
- package/src/docs/versioned/next/implementationPlans/terreno-langfuse-integration.md +1 -1
- package/src/docs/versioned/next/reference/README.md +3 -0
- package/src/docs/versioned/next/reference/admin-config.md +2 -0
- package/src/docs/versioned/next/reference/admin-frontend.md +216 -0
- package/src/docs/versioned/next/reference/agent-ui-asks.md +1059 -0
- package/src/docs/versioned/next/reference/ai-harness.md +1974 -0
- package/src/docs/versioned/next/reference/ai.md +251 -25
- package/src/docs/versioned/next/reference/api.md +110 -2
- package/src/docs/versioned/next/reference/blocks.md +178 -0
- package/src/docs/versioned/next/reference/environment-variables.md +2 -1
- package/src/docs/versioned/next/reference/lifecycle-plugin.md +11 -0
- package/src/docs/versioned/next/reference/mcp-server.md +17 -0
- package/src/docs/versioned/next/reference/syncdb.md +156 -6
- package/src/docs/versioned/next/reference/test.md +11 -2
- package/src/docs/versioned/next/reference/ui.md +331 -6
- package/src/docs/versioned/next/tasks/agent-ui-asks.md +213 -0
- package/src/docs/versioned/next/tasks/agent-ui-blocks.md +2 -1
- package/src/docs/versioned/next/tasks/ai-observability.md +233 -104
- package/src/docs/versioned/next/tasks/durable-agent-harness.md +185 -0
|
@@ -0,0 +1,730 @@
|
|
|
1
|
+
# Add agent asks to a chat
|
|
2
|
+
|
|
3
|
+
Pass `asks: true` to the GPT routes, then show and answer asks with `GPTChat`. The example app
|
|
4
|
+
does both: `example-backend/src/api/ai.ts` and `example-frontend/app/(tabs)/ai.tsx`. Fields, events,
|
|
5
|
+
and errors are in the [reference](../reference/agent-ui-asks.md).
|
|
6
|
+
|
|
7
|
+
To answer asks from a watch or another client that does not read server-sent events, see
|
|
8
|
+
[Answer asks from an Apple Watch or another small client](#answer-asks-from-an-apple-watch-or-another-small-client).
|
|
9
|
+
|
|
10
|
+
## Try it without an API key
|
|
11
|
+
|
|
12
|
+
The example backend uses a scripted demo agent, `terreno-demo-agent`, when no model is configured.
|
|
13
|
+
|
|
14
|
+
1. Start `example-backend` without `GEMINI_API_KEY` or `GOOGLE_VERTEX_PROJECT`. The log says
|
|
15
|
+
chat uses the scripted Terreno demo agent.
|
|
16
|
+
2. Start `example-frontend`, log in, and open the AI tab. Leave the Gemini API key empty: a saved
|
|
17
|
+
key is sent as `x-ai-api-key` and switches chat to that model.
|
|
18
|
+
3. Send "Help me pick a plan" (a suggested prompt).
|
|
19
|
+
4. Tap a plan. The agent replies with your pick, and the card collapses to "You chose: Team".
|
|
20
|
+
5. Reload the page and open the conversation again. The summary is still there.
|
|
21
|
+
6. Send "Pick toppings for my pizza". Check up to three toppings, type your own in "Another
|
|
22
|
+
topping", and press "Add toppings". The agent names every topping you picked, and the card
|
|
23
|
+
collapses to "You chose: Extra cheese, Olives. Another topping: Basil".
|
|
24
|
+
7. Send "Archive old chats" (a suggested prompt). The card shows a red "Archive 12 chats" button
|
|
25
|
+
first and "Keep them" last, with no Skip. Press "Keep them": the agent says it kept your chats,
|
|
26
|
+
and the card collapses to "You declined: Keep them".
|
|
27
|
+
8. Send "Send the weekly report" and press "Send report". The card collapses to "You confirmed:
|
|
28
|
+
Send report". Reload the page: both summaries are still there.
|
|
29
|
+
9. Send "Draft an announcement" (a suggested prompt). The card shows a markdown editor with the agent's draft and a
|
|
30
|
+
length hint. Add a line and press "Post it": the agent quotes your text, and the card
|
|
31
|
+
collapses to "You edited the draft (N characters)" with the text under it. Press "Post it"
|
|
32
|
+
without editing and the summary says "You approved the draft as is".
|
|
33
|
+
10. Send "Fill in the invoice details" (a suggested prompt). The card shows eight fields, most
|
|
34
|
+
filled in from defaults. Pick a start date, type "900" in Seats (it says "Enter a number from
|
|
35
|
+
1 to 500." and Send details stays disabled), change it to "12", and press "Send details". The
|
|
36
|
+
agent lists the details you sent, and the card collapses to "You sent the form (N fields)"
|
|
37
|
+
with each field's label and value under it.
|
|
38
|
+
11. Send "Upload a receipt" (a suggested prompt). Press "Choose files", pick an image and a text
|
|
39
|
+
or CSV file, and press "Send receipt". The agent names each file with its type and size, and
|
|
40
|
+
the first line of each text or CSV file. The card collapses to "You sent 2 files: …". Without
|
|
41
|
+
`GCS_BUCKET`, the files travel as data URLs; see
|
|
42
|
+
[Accept uploads with or without GCS](#accept-uploads-with-or-without-gcs).
|
|
43
|
+
12. On the Todos tab, add two todos and check both. On the AI tab, send "Delete my completed
|
|
44
|
+
todos". The agent calls the real `deleteCompletedTodos` tool, and the server pauses on an
|
|
45
|
+
approval card: a red "Delete" button first and "Keep them" last. Press "Keep them": the agent
|
|
46
|
+
says it kept your todos, and the Todos tab still shows them. Send the message again and press
|
|
47
|
+
"Delete": the agent names the deleted todos, and they are gone from the Todos tab. See
|
|
48
|
+
[Require approval before a tool runs](#require-approval-before-a-tool-runs).
|
|
49
|
+
|
|
50
|
+
| You send | The demo agent |
|
|
51
|
+
| --- | --- |
|
|
52
|
+
| A message with the word topping or toppings, such as "pick toppings" or "pick several toppings" | Asks "Which toppings should I add? Pick up to three." with `select: "many"`, six toppings (Extra cheese and Mushrooms preselected), `maxSelected: 3`, and an Other field titled "Another topping" |
|
|
53
|
+
| A message with "weekly report", such as "send the weekly report" | Asks `ask_confirm` "Send the weekly report to the team now? It goes to 8 people." with "Send report" (primary) and "Not now" |
|
|
54
|
+
| A message with the word archive, such as "archive old chats" | Asks `ask_confirm` "Archive the 12 chats older than 90 days? You can't undo this." with `destructive: true`, "Archive 12 chats" (destructive) and "Keep them" |
|
|
55
|
+
| A message with the word announcement, such as "draft an announcement" | Asks `ask_markdown` with a launch announcement draft as `initial`, `minLength: 40`, `maxLength: 2000`, and "Post it" as `submitLabel` |
|
|
56
|
+
| A message with the word invoice or form, such as "invoice details" or "fill out a form" | Asks `ask_form` "Invoice details" with eight fields: company name and billing email (required, with defaults), callback phone, seats (a whole number from 1 to 500, default 5), start date, region (select, default United States), "Email me the invoice" (default on), and notes |
|
|
57
|
+
| A message with the word receipt or receipts, or "upload" followed by file, document, photo, or image (optionally after a, an, my, some, or the), such as "upload a receipt" or "upload a file". "I uploaded the slides" does not match. | Asks `ask_files` "Upload a receipt" for up to 3 images, PDFs, text, or CSV files, with "Send receipt" as `submitLabel` |
|
|
58
|
+
| "delete", "clear", or "remove", then "completed" later in the same sentence, such as "delete my completed todos" or "clear the completed ones". "I completed the report, please delete the draft" does not match. | Calls the host tool `deleteCompletedTodos` (`needsApproval: true`). The server asks "Delete all of your completed todos? You can't undo this." with "Delete" (destructive) and "Keep them". After Delete, the agent names the deleted todos; after Keep them or a cancel, it says it kept them. Without the tool, or with asks off, it says why it did nothing. |
|
|
59
|
+
| Any other message with a word like pick, choose, or plan, such as "choose between several plans" | Asks "Which plan should I set up for your workspace?" with Starter, Team (the default), and Enterprise |
|
|
60
|
+
| The same kind of message, on routes without `asks` | Says asks are turned off and how to turn them on |
|
|
61
|
+
| An answer, or Skip | Replies with the plan or the toppings you picked, including the topping you typed, says whether it would send the report or archive the chats, quotes an edited draft or says you approved it, lists the invoice details you sent, names the files you sent, or says it skipped the question |
|
|
62
|
+
| Anything else, with or without `asks` | Explains that it follows a script and how to use a real model |
|
|
63
|
+
| Any of these with `surface: "compact"` | Asks the same plan and confirm questions, which already fit a watch, and replies in one or two short sentences without markdown. The compact surface offers only select one for `choice` and no `markdown`, `form`, or `files`, so a toppings, announcement, invoice, or receipt message gets a text reply that says to open the chat on a phone. |
|
|
64
|
+
|
|
65
|
+
To script another exchange, add an entry to `DEMO_SCENARIOS` in
|
|
66
|
+
`example-backend/src/api/demoAgent.ts`: a trigger pattern, one ask input, and a reply for the
|
|
67
|
+
answer. Scenarios match in order, so put one with narrower trigger words first. Set
|
|
68
|
+
`compactFallback` when the input does not fit the compact surface.
|
|
69
|
+
|
|
70
|
+
## Confirm before a destructive tool
|
|
71
|
+
|
|
72
|
+
With asks on, the model can call `ask_confirm` before a tool that deletes data, sends something on
|
|
73
|
+
the user's behalf, spends money, or cannot be undone. The asks system prompt tells it to, and to
|
|
74
|
+
take the action only after `{"confirmed": true}`. `ask_confirm` has no `execute`, so the turn
|
|
75
|
+
pauses until the user presses a button.
|
|
76
|
+
|
|
77
|
+
1. Say in the tool's description what it changes and that it needs a confirm first:
|
|
78
|
+
|
|
79
|
+
```typescript
|
|
80
|
+
const deleteCompletedTodos = tool({
|
|
81
|
+
description:
|
|
82
|
+
"Deletes the user's completed todos. Cannot be undone. Call ask_confirm with " +
|
|
83
|
+
"destructive: true first, and call this only after {confirmed: true}.",
|
|
84
|
+
execute: async () => ({deleted: await removeCompletedTodos(userId)}),
|
|
85
|
+
inputSchema: z.object({}),
|
|
86
|
+
});
|
|
87
|
+
|
|
88
|
+
addGptRoutes(router, {aiService, asks: true, tools: {deleteCompletedTodos}});
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
2. Check the rule in the tool too. The model decides when to ask, so a tool that must never run
|
|
92
|
+
unconfirmed should refuse on its own, or let the server enforce it with `needsApproval`; see
|
|
93
|
+
[Require approval before a tool runs](#require-approval-before-a-tool-runs).
|
|
94
|
+
|
|
95
|
+
The card shows `confirmLabel` first and `denyLabel` last. With `destructive: true` the approve
|
|
96
|
+
button uses the destructive style, and a watch never makes it the Double Tap button. A deny sends
|
|
97
|
+
`{"confirmed": false}`, so the model hears "no" and skips the action. Skip appears only when the
|
|
98
|
+
ask sets `allowDecline: true`. See [confirm](../reference/agent-ui-asks.md#confirm).
|
|
99
|
+
|
|
100
|
+
## Require approval before a tool runs
|
|
101
|
+
|
|
102
|
+
`ask_confirm` relies on the model to ask first. When a tool must never run without the user's
|
|
103
|
+
yes, set the AI SDK's `needsApproval: true` on it. The server then pauses the turn on a `confirm`
|
|
104
|
+
card of its own before the tool runs, whatever the model does. The example backend does this in
|
|
105
|
+
`example-backend/src/api/todoTools.ts`.
|
|
106
|
+
|
|
107
|
+
1. Build the tool per request, so it acts only on the caller's data, and mark it
|
|
108
|
+
`needsApproval`:
|
|
109
|
+
|
|
110
|
+
```typescript
|
|
111
|
+
import {tool, zodSchema} from "ai";
|
|
112
|
+
|
|
113
|
+
export const createTodoTools = ({userId}) => ({
|
|
114
|
+
deleteCompletedTodos: tool({
|
|
115
|
+
description: "Delete all of the signed-in user's completed todos.",
|
|
116
|
+
execute: async () => {
|
|
117
|
+
const todos = await Todo.find({completed: true, ownerId: userId});
|
|
118
|
+
for (const todo of todos) {
|
|
119
|
+
todo.deleted = true;
|
|
120
|
+
await todo.save();
|
|
121
|
+
}
|
|
122
|
+
return {deleted: todos.length, titles: todos.map((todo) => todo.title)};
|
|
123
|
+
},
|
|
124
|
+
inputSchema: zodSchema(z.object({}).strict()),
|
|
125
|
+
needsApproval: true,
|
|
126
|
+
}),
|
|
127
|
+
});
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
`Todo` is synced, and `syncPlugin` refuses bulk writes, so the tool saves each soft delete.
|
|
131
|
+
Each save reaches open Todos screens through sync.
|
|
132
|
+
|
|
133
|
+
2. Word the card with `asks.approvals`, and pass the tool with `createRequestTools`:
|
|
134
|
+
|
|
135
|
+
```typescript
|
|
136
|
+
addGptRoutes(router, {
|
|
137
|
+
aiService,
|
|
138
|
+
asks: {
|
|
139
|
+
approvals: {
|
|
140
|
+
deleteCompletedTodos: () => ({
|
|
141
|
+
confirmLabel: "Delete",
|
|
142
|
+
denyLabel: "Keep them",
|
|
143
|
+
destructive: true,
|
|
144
|
+
prompt: "Delete all of your completed todos? You can't undo this.",
|
|
145
|
+
title: "Delete completed todos",
|
|
146
|
+
}),
|
|
147
|
+
},
|
|
148
|
+
},
|
|
149
|
+
createRequestTools: (req) => createTodoTools({userId: req.user?._id}),
|
|
150
|
+
});
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
The function gets the call's input, so the prompt can name what the call changes. Without an
|
|
154
|
+
entry the card says "Allow deleteCompletedTodos?" with the tool's description and Allow / Deny
|
|
155
|
+
buttons. Set `destructive: true` for a tool that deletes data, so the approve button is red and a
|
|
156
|
+
watch never makes it the Double Tap button.
|
|
157
|
+
|
|
158
|
+
3. Show the card like any ask. `GPTChat` renders it as a `confirm`; the `{ask}` event adds
|
|
159
|
+
`origin: "approval"` and the host `toolName`. A watch answers it with the `approve` or `deny`
|
|
160
|
+
button through `turn`.
|
|
161
|
+
|
|
162
|
+
Approve runs the tool once and the model gets its result. Deny or a cancel never runs it, and the
|
|
163
|
+
model gets a denial with a reason such as `user_denied`. A new message also cancels the approval
|
|
164
|
+
without running the tool; the model then sees only the new message. Asks must be on: with
|
|
165
|
+
them off, the tool never runs. When the model calls two approval tools in one step, the user
|
|
166
|
+
approves the first, and the second is denied with `one_ask_at_a_time`. See
|
|
167
|
+
[Approval asks](../reference/agent-ui-asks.md#approval-asks).
|
|
168
|
+
|
|
169
|
+
## Have the user edit a draft
|
|
170
|
+
|
|
171
|
+
With asks on, the model can call `ask_markdown` with a draft in `initial`, such as an email or
|
|
172
|
+
release notes, before it sends or saves the text. The chat shows a markdown editor on the draft.
|
|
173
|
+
The answer is `{markdown, changed}`: `changed: false` means the user approved the draft as is,
|
|
174
|
+
and the server rejects a `changed` flag that does not match the text (`CHANGED_MISMATCH`). Set
|
|
175
|
+
`minLength` and `maxLength` to bound the answer; an `initial` outside them makes the user edit
|
|
176
|
+
before Submit is enabled. A phone or watch that shows only the simple card gets "Approve draft"
|
|
177
|
+
(when the draft fits the bounds) and Cancel, with "Edit on your phone". See
|
|
178
|
+
[markdown](../reference/agent-ui-asks.md#markdown).
|
|
179
|
+
|
|
180
|
+
## Collect a few details in one form
|
|
181
|
+
|
|
182
|
+
With asks on, the model can call `ask_form` when it needs several values together, such as the
|
|
183
|
+
details for an invoice or a booking. Each of the 1–8 fields has an `id`, a `label`, and a `type`:
|
|
184
|
+
`text`, `textarea`, `email`, `url`, `phone`, `number`, `date`, `time`, `datetime`, `boolean`,
|
|
185
|
+
`select`, or `multiselect`. The chat shows the matching `@terreno/ui` control for each field and
|
|
186
|
+
answers `{values}`, keyed by field id. Dates come back as `YYYY-MM-DD`, times as 24-hour `HH:mm`,
|
|
187
|
+
and datetimes with an offset. The server checks every value against its field, so a 400 names
|
|
188
|
+
each bad field (`REQUIRED_FIELD`, `FIELD_TYPE_MISMATCH`, `OUT_OF_RANGE`, `INVALID_DATE`) at
|
|
189
|
+
`content.values.<id>`, and the card shows each error on its field. Give required fields a
|
|
190
|
+
`default` when you know the likely answer: a phone or watch that shows only the simple card then
|
|
191
|
+
gets "Submit defaults" and Cancel, with "Fill it in on your phone". See
|
|
192
|
+
[form](../reference/agent-ui-asks.md#form).
|
|
193
|
+
|
|
194
|
+
## Accept uploads with or without GCS
|
|
195
|
+
|
|
196
|
+
With asks on, the model can call `ask_files` to have the user send a receipt, a screenshot, or an
|
|
197
|
+
export. Each file in the answer names its bytes one of two ways, and the host picks which:
|
|
198
|
+
|
|
199
|
+
| Host | Ref | Setup |
|
|
200
|
+
| --- | --- | --- |
|
|
201
|
+
| No file storage | `{url}`: a base64 `data:` URL | Nothing. `GPTChat` sends data URLs by default. |
|
|
202
|
+
| A GCS bucket | `{fileId}`: the id `POST /files/upload` returns | Give the server a `FileStorageService`, and pass `GPTChat` a `resolveAskFiles` that uploads |
|
|
203
|
+
|
|
204
|
+
Either way the server checks each file's bytes against its declared type before the model sees it
|
|
205
|
+
(`MIME_MISMATCH`), caps each file at 10 MB (`FILE_TOO_LARGE`), and stores only
|
|
206
|
+
`{fileId?, filename, mimeType, size}` in the history. Data URLs are not saved, so a long
|
|
207
|
+
conversation does not grow by the size of its files. See [files](../reference/agent-ui-asks.md#files).
|
|
208
|
+
|
|
209
|
+
### Without GCS
|
|
210
|
+
|
|
211
|
+
1. Turn asks on: `addGptRoutes(router, {aiService, asks: true})`.
|
|
212
|
+
2. Render `GPTChat` with `onAskSubmit`. Leave `resolveAskFiles` unset.
|
|
213
|
+
|
|
214
|
+
The answer's request body carries each file as a data URL, about a third larger than the file.
|
|
215
|
+
`TerrenoApp` parses JSON bodies up to 50 MB, so one answer fits about three 10 MB files. For more,
|
|
216
|
+
use uploads, or lower the per-file cap with `asks: {maxFileSizeBytes: 5_000_000}`.
|
|
217
|
+
|
|
218
|
+
### With GCS
|
|
219
|
+
|
|
220
|
+
1. Create the storage service and pass it to both the file routes and the chat. With `AiApp`,
|
|
221
|
+
set both options and it does this for you:
|
|
222
|
+
|
|
223
|
+
```typescript
|
|
224
|
+
const fileStorageService = new FileStorageService({bucketName: process.env.GCS_BUCKET});
|
|
225
|
+
|
|
226
|
+
new AiApp({aiService, asks: true, fileStorageService, gcsBucket: process.env.GCS_BUCKET});
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
With the route functions, pass the same service to `addFileRoutes` and, as
|
|
230
|
+
`fileStorageService`, to the chat options you give `addGptRoutes` and `addGptHistoryRoutes`.
|
|
231
|
+
Without it, the chat cannot read uploads, and every `fileId` fails with `FILE_NOT_OWNED`.
|
|
232
|
+
2. Pass `GPTChat` a resolver that uploads each file and returns its id. The example app's
|
|
233
|
+
`createAskFilesResolver` in `example-frontend/lib/gptAsks.ts` does this, and falls back to data
|
|
234
|
+
URLs when `/files/upload` answers 404, so the same app works with and without a bucket:
|
|
235
|
+
|
|
236
|
+
```tsx
|
|
237
|
+
const [postFilesUpload] = usePostFilesUploadMutation();
|
|
238
|
+
const resolveAskFiles = useMemo(
|
|
239
|
+
() =>
|
|
240
|
+
createAskFilesResolver({
|
|
241
|
+
upload: createAskFileUploader({
|
|
242
|
+
send: async (file) => postFilesUpload(await uploadFormData(file)),
|
|
243
|
+
}),
|
|
244
|
+
}),
|
|
245
|
+
[postFilesUpload]
|
|
246
|
+
);
|
|
247
|
+
|
|
248
|
+
<GPTChat resolveAskFiles={resolveAskFiles} {...chatProps} />;
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
`postFilesUpload` is an RTK Query mutation in `example-frontend/store/sdk.ts` whose body is a
|
|
252
|
+
`FormData` with a `file` part; the base query sends it with the session token. The generated
|
|
253
|
+
SDK has no file endpoints, because `/files/*` exists only on a server with a bucket.
|
|
254
|
+
`createAskFileUploader` (in `gptAsks.ts`) returns `undefined` on 404, the upload's `id` and
|
|
255
|
+
`size` on success, and throws with the server's detail otherwise (such as 401 when the session
|
|
256
|
+
has ended), so the ask stays open. See `example-frontend/app/(tabs)/ai.tsx`.
|
|
257
|
+
|
|
258
|
+
A resolver that throws keeps the ask open, and the card says the files could not be sent. A
|
|
259
|
+
`fileId` must name an upload of the user who answers, so one user cannot send another's files.
|
|
260
|
+
|
|
261
|
+
## 1. Enable asks on the backend
|
|
262
|
+
|
|
263
|
+
```typescript
|
|
264
|
+
addGptRoutes(router, {aiService, asks: true});
|
|
265
|
+
```
|
|
266
|
+
|
|
267
|
+
`new AiApp({aiService, asks: true})` takes the same option. A real model decides when to ask; the
|
|
268
|
+
system prompt that asks add tells it how. See [Enable asks](../reference/agent-ui-asks.md#enable-asks).
|
|
269
|
+
|
|
270
|
+
## 2. Show asks from the stream
|
|
271
|
+
|
|
272
|
+
`POST /gpt/prompt` adds three events. Map them onto `GPTChat` messages:
|
|
273
|
+
|
|
274
|
+
| Event | Transcript change |
|
|
275
|
+
| --- | --- |
|
|
276
|
+
| `{ask: {toolCallId, kind, input, simple}, historyId}` | Append a `tool-call` message with `ask: {kind, input, simple, status: "pending", toolCallId}`, and remember `historyId` for the ask so it can be answered before `{done}` |
|
|
277
|
+
| `{askResolved: {toolCallId, action}}` | Mark the ask `answered` (`cancelled` for `cancel`), set its `response`, and insert a `tool-result` message right after it |
|
|
278
|
+
| `{done: true, historyId, pendingAsk?}` | The turn ended, also after `{error}`. `pendingAsk.toolCallId` names the ask the conversation waits on. |
|
|
279
|
+
|
|
280
|
+
```typescript
|
|
281
|
+
if (data.ask) {
|
|
282
|
+
askHistoryIdsRef.current.set(data.ask.toolCallId, data.historyId);
|
|
283
|
+
setCurrentMessages((prev) => [...withoutEmptyAssistant(prev), askMessage(data.ask)]);
|
|
284
|
+
} else if (data.askResolved) {
|
|
285
|
+
const {action, toolCallId} = data.askResolved;
|
|
286
|
+
setCurrentMessages((prev) => withResolvedAsk({action, messages: prev, submitted, toolCallId}));
|
|
287
|
+
}
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
Copy `askMessage`, `withResolvedAsk`, `withoutEmptyAssistant`, and `answerHistoryId` from
|
|
291
|
+
`example-frontend/lib/gptAsks.ts`. `submitted` is the answer this turn sent, so the summary can
|
|
292
|
+
name the chosen option. `withoutEmptyAssistant` drops the empty assistant placeholder but keeps a
|
|
293
|
+
reply that holds only an image or a file. `askHistoryIdsRef` is a `useRef(new Map<string, string>())`.
|
|
294
|
+
|
|
295
|
+
Insert the `tool-result` message even though `GPTChat` hides it. The server stores the ask and its
|
|
296
|
+
answer as two rows, and ratings are sent by message index.
|
|
297
|
+
|
|
298
|
+
A turn can stream only `{done}`, for example when the server cancels a second ask because one is
|
|
299
|
+
already pending. Reload the conversation with `GET /gpt/histories/:id` when a turn streamed nothing
|
|
300
|
+
visible, or when `done.pendingAsk` names an ask the stream did not send. The `@terreno/rtk` base
|
|
301
|
+
query returns the `data` of single-document responses, so an RTK query for that route resolves to
|
|
302
|
+
the history itself, not `{data}` (see [emptyApi](../reference/legacy/rtk.md#emptyapi--emptysplitapi)).
|
|
303
|
+
|
|
304
|
+
Refetch the history list after a turn that streams ask events or reloads the conversation. The
|
|
305
|
+
example app reopens a conversation from the list's cached rows, which still hold the pending ask.
|
|
306
|
+
|
|
307
|
+
## 3. Show asks in saved conversations
|
|
308
|
+
|
|
309
|
+
An ask's `tool-call` row has `ask: {kind, status}` and the ask input in `args`.
|
|
310
|
+
`GptHistory.pendingAsk` holds the ask the conversation waits on, with its simple card.
|
|
311
|
+
|
|
312
|
+
```typescript
|
|
313
|
+
const prompts = history.prompts.map((p): GPTChatMessage => {
|
|
314
|
+
const ask = askFromHistoryPrompt({pendingAsk: history.pendingAsk, prompt: p});
|
|
315
|
+
return {
|
|
316
|
+
...(ask ? {ask} : {}),
|
|
317
|
+
content: p.text,
|
|
318
|
+
role: p.type,
|
|
319
|
+
...(p.toolCallId && p.type === "tool-call"
|
|
320
|
+
? {toolCall: {args: p.args ?? {}, toolCallId: p.toolCallId, toolName: p.toolName ?? ""}}
|
|
321
|
+
: {}),
|
|
322
|
+
...(p.toolCallId && p.type === "tool-result"
|
|
323
|
+
? {toolResult: {result: p.result, toolCallId: p.toolCallId, toolName: p.toolName ?? ""}}
|
|
324
|
+
: {}),
|
|
325
|
+
};
|
|
326
|
+
});
|
|
327
|
+
```
|
|
328
|
+
|
|
329
|
+
`askFromHistoryPrompt` is in `example-frontend/lib/gptAsks.ts`. Keep `tool-result` rows as
|
|
330
|
+
`tool-result` messages: `GPTChat` hides the ones that answer an ask and reads the summary from them.
|
|
331
|
+
|
|
332
|
+
## 4. Send answers
|
|
333
|
+
|
|
334
|
+
```tsx
|
|
335
|
+
const handleAskSubmit = useCallback(
|
|
336
|
+
async ({response, toolCallId}: AskSubmission): Promise<void> => {
|
|
337
|
+
// A new chat's ask can be answered before `{done}` sets currentHistoryId.
|
|
338
|
+
const historyId = answerHistoryId({
|
|
339
|
+
askHistoryIds: askHistoryIdsRef.current,
|
|
340
|
+
currentHistoryId,
|
|
341
|
+
toolCallId,
|
|
342
|
+
});
|
|
343
|
+
if (!historyId) {
|
|
344
|
+
return;
|
|
345
|
+
}
|
|
346
|
+
await runTurn({
|
|
347
|
+
body: {askResponse: {...response, toolCallId}, historyId},
|
|
348
|
+
submitted: {response, toolCallId},
|
|
349
|
+
});
|
|
350
|
+
},
|
|
351
|
+
[currentHistoryId, runTurn]
|
|
352
|
+
);
|
|
353
|
+
|
|
354
|
+
return (
|
|
355
|
+
<GPTChat
|
|
356
|
+
askErrors={askErrors}
|
|
357
|
+
currentMessages={currentMessages}
|
|
358
|
+
histories={histories}
|
|
359
|
+
onAskSubmit={handleAskSubmit}
|
|
360
|
+
onCreateHistory={handleCreateHistory}
|
|
361
|
+
onDeleteHistory={handleDeleteHistory}
|
|
362
|
+
onSelectHistory={handleSelectHistory}
|
|
363
|
+
onSubmit={handleSubmit}
|
|
364
|
+
/>
|
|
365
|
+
);
|
|
366
|
+
```
|
|
367
|
+
|
|
368
|
+
`runTurn` is the function that already posts prompts and reads the SSE stream. An answer's turn
|
|
369
|
+
starts with `{askResolved}`, then streams the continuation. Return the promise: the pressed button
|
|
370
|
+
shows a loading state until it settles.
|
|
371
|
+
|
|
372
|
+
## 5. Handle errors
|
|
373
|
+
|
|
374
|
+
| Response | Handling |
|
|
375
|
+
| --- | --- |
|
|
376
|
+
| 400 `Invalid askResponse` with `fields` | Set `askErrors[toolCallId]` to `fields`. The card shows each `message`. Clear the entry when the user answers again. |
|
|
377
|
+
| 409 `This ask is no longer pending` on an answer | Another tab or device resolved the ask first. Reload the conversation and the history list so the card shows how it ended. |
|
|
378
|
+
| A `prompt` sent while the transcript shows a pending ask, with no `{askResolved}` for it | Another tab or device resolved the ask first, and the message went ahead as a normal prompt. Reload the conversation so the card shows how the ask ended. A `prompt` never gets 409. |
|
|
379
|
+
|
|
380
|
+
## Answer asks from an Apple Watch or another small client
|
|
381
|
+
|
|
382
|
+
A client that does not read server-sent events, such as a watch app, a notification action, or a
|
|
383
|
+
chat bot, uses two JSON endpoints on `/gpt/histories`:
|
|
384
|
+
|
|
385
|
+
| Endpoint | Use |
|
|
386
|
+
| --- | --- |
|
|
387
|
+
| `GET /gpt/histories/pendingAsks` | List the asks waiting on the user, newest first, each with its simple card |
|
|
388
|
+
| `POST /gpt/histories/:id/turn` | Send a message or a pressed button, and get the reply once the turn finishes |
|
|
389
|
+
|
|
390
|
+
The client needs no knowledge of ask kinds. It shows the card's text and buttons and sends back the
|
|
391
|
+
pressed button's `id`. React Native does not run on watchOS, so a watch client is a native SwiftUI
|
|
392
|
+
target. [`@bacons/apple-targets`](https://github.com/evanbacon/expo-apple-targets) adds one to an
|
|
393
|
+
Expo app. Terreno does not ship a watch app; the steps below sketch one. Fields and errors are in
|
|
394
|
+
[Headless endpoints](../reference/agent-ui-asks.md#headless-endpoints).
|
|
395
|
+
|
|
396
|
+
### 1. Register the endpoints
|
|
397
|
+
|
|
398
|
+
Both endpoints exist only with asks on. `new AiApp({aiService, asks: true})` registers both. With
|
|
399
|
+
the route functions, pass `addGptHistoryRoutes` the options you give `addGptRoutes`, as `chat`, so
|
|
400
|
+
`turn` runs the same chat as `/gpt/prompt`. Without `chat`, or with `asks` off in it, it adds
|
|
401
|
+
neither endpoint:
|
|
402
|
+
|
|
403
|
+
```typescript
|
|
404
|
+
const chat = {aiService, asks: true};
|
|
405
|
+
addGptHistoryRoutes(router, {chat});
|
|
406
|
+
addGptRoutes(router, chat);
|
|
407
|
+
```
|
|
408
|
+
|
|
409
|
+
### 2. Try them with curl
|
|
410
|
+
|
|
411
|
+
Run the example backend as in [Try it without an API key](#try-it-without-an-api-key) and seed
|
|
412
|
+
it with `bun run backend:seed`. This script signs in, asks the demo agent to pick a plan from a
|
|
413
|
+
small screen, lists the pending ask's buttons, and presses Starter. It needs `jq`.
|
|
414
|
+
|
|
415
|
+
```bash
|
|
416
|
+
API=http://localhost:4000
|
|
417
|
+
JSON="Content-Type: application/json"
|
|
418
|
+
|
|
419
|
+
TOKEN=$(curl -s -X POST $API/api/auth/sign-in/email -H "$JSON" \
|
|
420
|
+
-d '{"email": "test@example.com", "password": "testpassword123"}' | jq -r .token)
|
|
421
|
+
AUTH="Authorization: Bearer $TOKEN"
|
|
422
|
+
|
|
423
|
+
HISTORY=$(curl -s -X POST $API/gpt/histories -H "$AUTH" -H "$JSON" -d '{}' | jq -r .data._id)
|
|
424
|
+
ASK=$(curl -s -X POST $API/gpt/histories/$HISTORY/turn -H "$AUTH" -H "$JSON" \
|
|
425
|
+
-d '{"prompt": "Help me pick a plan", "surface": "compact"}' | jq -r .data.pendingAsk.toolCallId)
|
|
426
|
+
curl -s $API/gpt/histories/pendingAsks -H "$AUTH" | jq -c '[.data[0].simple.buttons[].id]'
|
|
427
|
+
curl -s -X POST $API/gpt/histories/$HISTORY/turn -H "$AUTH" -H "$JSON" \
|
|
428
|
+
-d "{\"toolCallId\": \"$ASK\", \"buttonId\": \"option:starter\", \"surface\": \"compact\"}" |
|
|
429
|
+
jq -r .data.text
|
|
430
|
+
```
|
|
431
|
+
|
|
432
|
+
It prints the card's button ids, the default plan first, then the demo agent's compact reply:
|
|
433
|
+
|
|
434
|
+
```text
|
|
435
|
+
["option:team","option:starter","option:enterprise"]
|
|
436
|
+
You picked the Starter plan. A real agent would set it up now.
|
|
437
|
+
```
|
|
438
|
+
|
|
439
|
+
### 3. Hand the session token to the watch
|
|
440
|
+
|
|
441
|
+
The watch signs its requests with the phone's Better Auth session token, sent as
|
|
442
|
+
`Authorization: Bearer <token>`, the header the example app sends on every request. On the phone,
|
|
443
|
+
the token is `session.token` from `authClient.getSession()`, which `getSessionToken()` in
|
|
444
|
+
`example-frontend/lib/betterAuth.ts` reads.
|
|
445
|
+
|
|
446
|
+
Send the token to the watch in the WatchConnectivity application context. From JavaScript,
|
|
447
|
+
`react-native-watch-connectivity` wraps `WCSession`. It needs a development build and works only
|
|
448
|
+
on iOS:
|
|
449
|
+
|
|
450
|
+
```typescript
|
|
451
|
+
import {Platform} from "react-native";
|
|
452
|
+
import {updateApplicationContext} from "react-native-watch-connectivity";
|
|
453
|
+
|
|
454
|
+
import {getSessionToken} from "@/lib/betterAuth";
|
|
455
|
+
|
|
456
|
+
/** Call after sign-in and after sign-out: an empty token tells the watch to forget it. */
|
|
457
|
+
export const shareSessionWithWatch = async (): Promise<void> => {
|
|
458
|
+
if (Platform.OS !== "ios") {
|
|
459
|
+
return;
|
|
460
|
+
}
|
|
461
|
+
updateApplicationContext({sessionToken: (await getSessionToken()) ?? ""});
|
|
462
|
+
};
|
|
463
|
+
```
|
|
464
|
+
|
|
465
|
+
The application context holds only the latest value, and the watch receives it the next time the
|
|
466
|
+
watch app runs. On the watch, keep the token in the keychain, never in `UserDefaults`:
|
|
467
|
+
|
|
468
|
+
```swift
|
|
469
|
+
import WatchConnectivity
|
|
470
|
+
|
|
471
|
+
/// Keeps the session token the phone sends. Call `start()` when the watch app launches.
|
|
472
|
+
final class SessionTokenReceiver: NSObject, WCSessionDelegate {
|
|
473
|
+
static let shared = SessionTokenReceiver()
|
|
474
|
+
|
|
475
|
+
func start() {
|
|
476
|
+
WCSession.default.delegate = self
|
|
477
|
+
WCSession.default.activate()
|
|
478
|
+
}
|
|
479
|
+
|
|
480
|
+
func session(
|
|
481
|
+
_ session: WCSession,
|
|
482
|
+
activationDidCompleteWith activationState: WCSessionActivationState,
|
|
483
|
+
error: Error?
|
|
484
|
+
) {
|
|
485
|
+
store(session.receivedApplicationContext)
|
|
486
|
+
}
|
|
487
|
+
|
|
488
|
+
func session(_ session: WCSession, didReceiveApplicationContext context: [String: Any]) {
|
|
489
|
+
store(context)
|
|
490
|
+
}
|
|
491
|
+
|
|
492
|
+
private func store(_ context: [String: Any]) {
|
|
493
|
+
guard let token = context["sessionToken"] as? String else {
|
|
494
|
+
return
|
|
495
|
+
}
|
|
496
|
+
if token.isEmpty {
|
|
497
|
+
TokenKeychain.delete()
|
|
498
|
+
} else {
|
|
499
|
+
TokenKeychain.save(token)
|
|
500
|
+
}
|
|
501
|
+
}
|
|
502
|
+
}
|
|
503
|
+
```
|
|
504
|
+
|
|
505
|
+
`TokenKeychain` stands for your keychain wrapper: one generic password item written with
|
|
506
|
+
`SecItemAdd` and `kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly`, read with
|
|
507
|
+
`SecItemCopyMatching`, and removed with `SecItemDelete`. The token has the same access as the phone
|
|
508
|
+
app. It stops working when the session expires or the user signs out.
|
|
509
|
+
|
|
510
|
+
### 4. Call the endpoints with URLSession
|
|
511
|
+
|
|
512
|
+
```swift
|
|
513
|
+
import Foundation
|
|
514
|
+
|
|
515
|
+
struct SimpleCardButton: Decodable, Identifiable {
|
|
516
|
+
let id: String
|
|
517
|
+
let label: String
|
|
518
|
+
/// "default", "primary", "destructive", or "cancel"
|
|
519
|
+
let style: String
|
|
520
|
+
}
|
|
521
|
+
|
|
522
|
+
struct SimpleCard: Decodable {
|
|
523
|
+
let title: String?
|
|
524
|
+
let text: String
|
|
525
|
+
let buttons: [SimpleCardButton]
|
|
526
|
+
let handoff: Bool
|
|
527
|
+
}
|
|
528
|
+
|
|
529
|
+
struct PendingAsk: Decodable, Identifiable {
|
|
530
|
+
let historyId: String
|
|
531
|
+
let toolCallId: String
|
|
532
|
+
let title: String?
|
|
533
|
+
let simple: SimpleCard
|
|
534
|
+
|
|
535
|
+
var id: String { toolCallId }
|
|
536
|
+
}
|
|
537
|
+
|
|
538
|
+
struct TurnResult: Decodable {
|
|
539
|
+
let text: String
|
|
540
|
+
let error: String?
|
|
541
|
+
}
|
|
542
|
+
|
|
543
|
+
enum AskClientError: Error {
|
|
544
|
+
case signedOut
|
|
545
|
+
case noLongerPending
|
|
546
|
+
case failed(status: Int)
|
|
547
|
+
}
|
|
548
|
+
|
|
549
|
+
struct AskClient {
|
|
550
|
+
let baseURL: URL
|
|
551
|
+
let token: String
|
|
552
|
+
|
|
553
|
+
func pendingAsks() async throws -> [PendingAsk] {
|
|
554
|
+
try await send("GET", "gpt/histories/pendingAsks")
|
|
555
|
+
}
|
|
556
|
+
|
|
557
|
+
func press(_ button: SimpleCardButton, on ask: PendingAsk) async throws -> TurnResult {
|
|
558
|
+
try await send("POST", "gpt/histories/\(ask.historyId)/turn", body: [
|
|
559
|
+
"buttonId": button.id,
|
|
560
|
+
"surface": "compact",
|
|
561
|
+
"toolCallId": ask.toolCallId,
|
|
562
|
+
])
|
|
563
|
+
}
|
|
564
|
+
|
|
565
|
+
func say(_ prompt: String, in historyId: String) async throws -> TurnResult {
|
|
566
|
+
try await send("POST", "gpt/histories/\(historyId)/turn", body: [
|
|
567
|
+
"prompt": prompt,
|
|
568
|
+
"surface": "compact",
|
|
569
|
+
])
|
|
570
|
+
}
|
|
571
|
+
|
|
572
|
+
private struct Envelope<Value: Decodable>: Decodable {
|
|
573
|
+
let data: Value
|
|
574
|
+
}
|
|
575
|
+
|
|
576
|
+
private func send<Value: Decodable>(
|
|
577
|
+
_ method: String, _ path: String, body: [String: String]? = nil
|
|
578
|
+
) async throws -> Value {
|
|
579
|
+
var request = URLRequest(url: baseURL.appending(path: path))
|
|
580
|
+
request.httpMethod = method
|
|
581
|
+
// A turn responds only when the agent is done, which can take longer than the 60 s default.
|
|
582
|
+
request.timeoutInterval = 120
|
|
583
|
+
request.setValue("Bearer \(token)", forHTTPHeaderField: "Authorization")
|
|
584
|
+
if let body {
|
|
585
|
+
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
|
|
586
|
+
request.httpBody = try JSONEncoder().encode(body)
|
|
587
|
+
}
|
|
588
|
+
let (data, response) = try await URLSession.shared.data(for: request)
|
|
589
|
+
let status = (response as? HTTPURLResponse)?.statusCode ?? 0
|
|
590
|
+
switch status {
|
|
591
|
+
case 200..<300:
|
|
592
|
+
return try JSONDecoder().decode(Envelope<Value>.self, from: data).data
|
|
593
|
+
case 401:
|
|
594
|
+
TokenKeychain.delete()
|
|
595
|
+
throw AskClientError.signedOut
|
|
596
|
+
case 409:
|
|
597
|
+
throw AskClientError.noLongerPending
|
|
598
|
+
default:
|
|
599
|
+
throw AskClientError.failed(status: status)
|
|
600
|
+
}
|
|
601
|
+
}
|
|
602
|
+
}
|
|
603
|
+
```
|
|
604
|
+
|
|
605
|
+
`JSONDecoder` ignores fields the structs leave out, such as each button's `response`: the server
|
|
606
|
+
answers with the stored `response` of the button whose `id` the watch sends. `turn` also returns
|
|
607
|
+
`pendingAsk` when the reply ends in a new ask. This sketch reloads `pendingAsks` after every turn
|
|
608
|
+
instead, which also picks up asks from other devices. To start a conversation from the watch, send
|
|
609
|
+
`POST /gpt/histories` with the body `{}` and use `data._id` as the `historyId`. The
|
|
610
|
+
[JSON Schemas](../reference/agent-ui-asks.md#json-schemas-and-fixtures) in
|
|
611
|
+
`@terreno/blocks/schemas/` describe every field, if you would rather generate `Codable` types.
|
|
612
|
+
|
|
613
|
+
The watch sends `surface: "compact"` on every turn, answers included:
|
|
614
|
+
|
|
615
|
+
- **Every ask fits the watch.** On a compact turn the agent can ask only a `choice` with 2–3
|
|
616
|
+
options whose labels fit a button uncut and differ from each other, or a `confirm`. Its card
|
|
617
|
+
has `handoff: false`, so the user can answer it from the watch. `markdown`, `form`, and `files`
|
|
618
|
+
are never offered.
|
|
619
|
+
- **Replies fit the screen.** The system prompt asks for at most two short sentences.
|
|
620
|
+
- **The surface covers one turn.** The server does not store it. An answer starts a turn that can
|
|
621
|
+
end in a new ask, so the answer sends `compact` too. The phone's chat sends no `surface`, so its
|
|
622
|
+
turns in the same conversation use `full` and offer every ask kind.
|
|
623
|
+
|
|
624
|
+
`pendingAsks` also lists asks the agent made in the phone's chat, whose cards can set `handoff`.
|
|
625
|
+
Show the buttons such a card has, such as `Use "Team"` and Skip, with its "Continue on your phone"
|
|
626
|
+
line. A `markdown` card offers "Approve draft" and Cancel; say "Edit on your phone" for it. A
|
|
627
|
+
`form` card offers "Submit defaults", when every required field has a default, and Cancel; say
|
|
628
|
+
"Fill it in on your phone" for it. A `files` card offers only Skip, when the ask allows it; say
|
|
629
|
+
"Upload on your phone" for it.
|
|
630
|
+
|
|
631
|
+
### 5. Show the card in SwiftUI
|
|
632
|
+
|
|
633
|
+
```swift
|
|
634
|
+
import SwiftUI
|
|
635
|
+
|
|
636
|
+
struct PendingAskView: View {
|
|
637
|
+
let ask: PendingAsk
|
|
638
|
+
let client: AskClient
|
|
639
|
+
/// Shows the reply, then reloads `pendingAsks`. `nil` when the answer did not go through.
|
|
640
|
+
let onFinish: (TurnResult?) -> Void
|
|
641
|
+
|
|
642
|
+
@State private var pressedButtonId: String?
|
|
643
|
+
|
|
644
|
+
var body: some View {
|
|
645
|
+
ScrollView {
|
|
646
|
+
VStack(alignment: .leading, spacing: 8) {
|
|
647
|
+
if let title = ask.simple.title {
|
|
648
|
+
Text(title).font(.headline)
|
|
649
|
+
}
|
|
650
|
+
Text(ask.simple.text)
|
|
651
|
+
if ask.simple.handoff {
|
|
652
|
+
Text("Continue on your phone")
|
|
653
|
+
.font(.footnote)
|
|
654
|
+
.foregroundStyle(.secondary)
|
|
655
|
+
}
|
|
656
|
+
ForEach(ask.simple.buttons) { button in
|
|
657
|
+
Button(button.label, role: role(for: button.style)) {
|
|
658
|
+
press(button)
|
|
659
|
+
}
|
|
660
|
+
.tint(button.style == "primary" ? .accentColor : nil)
|
|
661
|
+
.disabled(pressedButtonId != nil)
|
|
662
|
+
.handGestureShortcut(.primaryAction, isEnabled: button.id == doubleTapButtonId)
|
|
663
|
+
}
|
|
664
|
+
}
|
|
665
|
+
}
|
|
666
|
+
}
|
|
667
|
+
|
|
668
|
+
/// The first button that is not destructive, so Double Tap never approves a destructive action.
|
|
669
|
+
private var doubleTapButtonId: String? {
|
|
670
|
+
ask.simple.buttons.first { $0.style != "destructive" }?.id
|
|
671
|
+
}
|
|
672
|
+
|
|
673
|
+
private func role(for style: String) -> ButtonRole? {
|
|
674
|
+
switch style {
|
|
675
|
+
case "destructive":
|
|
676
|
+
return .destructive
|
|
677
|
+
case "cancel":
|
|
678
|
+
return .cancel
|
|
679
|
+
default:
|
|
680
|
+
return nil
|
|
681
|
+
}
|
|
682
|
+
}
|
|
683
|
+
|
|
684
|
+
private func press(_ button: SimpleCardButton) {
|
|
685
|
+
pressedButtonId = button.id
|
|
686
|
+
Task {
|
|
687
|
+
let result = try? await client.press(button, on: ask)
|
|
688
|
+
pressedButtonId = nil
|
|
689
|
+
onFinish(result)
|
|
690
|
+
}
|
|
691
|
+
}
|
|
692
|
+
}
|
|
693
|
+
```
|
|
694
|
+
|
|
695
|
+
Keep the buttons in the card's order. When the ask suggests an answer, its button comes first with
|
|
696
|
+
style `primary`, and Skip comes last with style `cancel`. A `confirm` card has the approve button
|
|
697
|
+
first and the deny button last. Give Double Tap (`handGestureShortcut`, watchOS 11 and later) to the
|
|
698
|
+
first button that is not `destructive`: on a destructive confirm that is the deny button, so the
|
|
699
|
+
gesture cannot approve an action that cannot be undone. Disable every button while one answer is
|
|
700
|
+
sending, so a double tap sends one answer instead of a second one that gets 409. `SimpleAskCard`
|
|
701
|
+
in `@terreno/ui` draws the same card in React Native ([props](../reference/ui.md#simpleaskcard)).
|
|
702
|
+
|
|
703
|
+
### 6. Handle errors
|
|
704
|
+
|
|
705
|
+
| Response | Handling |
|
|
706
|
+
| --- | --- |
|
|
707
|
+
| 401 | The session ended or the user signed out. The client above deletes the token. Ask the user to open the phone app, which sends a new token after sign-in. |
|
|
708
|
+
| 409 `This ask is no longer pending` | Another device answered or cancelled the ask first. Reload `pendingAsks`. |
|
|
709
|
+
| 400 `UNKNOWN_BUTTON` | The `buttonId` is not on the card. Send only ids from `simple.buttons`; the error's `fix` lists them. |
|
|
710
|
+
| 200 with `error` in `data` | The agent failed after the turn started. The message or answer is kept, with what the agent did before it failed. Show a short error; the next message continues the conversation. |
|
|
711
|
+
| A timeout or a lost connection | The turn still finishes and is saved. Reload `pendingAsks`, or read the conversation with `GET /gpt/histories/:id`. |
|
|
712
|
+
|
|
713
|
+
## Verify
|
|
714
|
+
|
|
715
|
+
| Test | Covers |
|
|
716
|
+
| --- | --- |
|
|
717
|
+
| `example-frontend/e2e/ai-chat.spec.ts` | Against a mocked stream: a quick-reply answer, a radio answer after a rejected one, an answer another tab sent first, a turn that streams only `{done}`, a message that goes through after another tab answered the ask, an ask on a new chat answered before `{done}`, the result card of a tool an approval ran shown live, and a receipt sent as data URLs after a rejected answer and as uploads |
|
|
718
|
+
| `example-frontend/lib/gptAsks.test.ts` | Stream events and saved rows mapped to messages, tool results placed with and without a streamed tool call, image-only replies kept, the conversation an answer goes to, and the upload resolver with its data URL fallback |
|
|
719
|
+
| `example-backend/src/api/demoAgent.test.ts` | The demo agent's ask, answers, and replies on both surfaces, a plan ask answered over HTTP with a button of its simple card, an announcement draft edited in chat and approved as is with its card's Approve draft button, and a receipt upload rejected for a wrong file and then named file by file |
|
|
720
|
+
| `ai/src/routes/gptApprovals.test.ts` | Approval asks: the pause without running the tool, the host and default approval inputs, the compact surface, approve running the tool once, deny, decline, and cancel never running it, a prompt that cancels it, extra approvals denied with `one_ask_at_a_time`, `turn` buttons, and crafted answers refused |
|
|
721
|
+
| `example-backend/src/api/todoTools.test.ts`, `demoAgent.test.ts` (cleanup) | `deleteCompletedTodos` soft-deletes only the caller's completed todos, and the demo agent's approval flow over `/gpt/prompt` and `turn` |
|
|
722
|
+
| `ai/src/routes/gptFiles.test.ts` | `ask_files` answers with data URLs and with uploads: the bytes the model gets, the metadata-only stored answer, and each error code |
|
|
723
|
+
| `ui/src/GPTChat.test.tsx`, `ui/src/asks/AskCard.test.tsx` | Rendering, focus, answers, errors, and summaries |
|
|
724
|
+
| `ai/src/routes/gptHistories.test.ts`, `ai/src/aiApp.test.ts` | `turn` and `pendingAsks`: a pressed button, a full answer, a prompt that cancels the ask, a failed turn, a stream that fails mid-reply, a client that disconnects, the 400, 403, 404, and 409 responses, response bodies that match the published JSON Schemas, and neither endpoint when asks are off |
|
|
725
|
+
| `ai/src/routes/gpt.test.ts` (compact surface) | A compact turn offers only the narrowed `ask_choice` and adds the compact line to the system prompt |
|
|
726
|
+
| `ai/src/routes/gpt.test.ts` (failures and races) | `{error}` then `{done}` when the model fails before its first chunk, mid-reply, or during a resume; a prompt that goes through after an answer resolved the ask; two turns and a rating on one conversation keeping every row |
|
|
727
|
+
| `ui/src/asks/SimpleAskCard.test.tsx`, `demo/stories/SimpleAskCard.stories.test.tsx` | Every fixture's card, button presses, the sending state, and the compact `turn` body the watch-sized demo sends |
|
|
728
|
+
|
|
729
|
+
Related: [UI reference for `GPTChat` asks and `AskCard`](../reference/ui.md#asks),
|
|
730
|
+
[Agent UI Asks explained](../explanation/agent-ui-asks.md).
|