@alexkroman1/aai-cli 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (58) hide show
  1. package/LICENSE +21 -0
  2. package/dist/_build-p1HHkdon.mjs +132 -0
  3. package/dist/_discover-BzlCDVZ6.mjs +161 -0
  4. package/dist/_init-l_uoyFCN.mjs +82 -0
  5. package/dist/_link-BGXGFYWa.mjs +47 -0
  6. package/dist/_server-common-qLA1QU2C.mjs +36 -0
  7. package/dist/_ui-kJIua5L9.mjs +44 -0
  8. package/dist/cli.mjs +318 -0
  9. package/dist/deploy-KyNJaoP5.mjs +86 -0
  10. package/dist/dev-DBFvKyzk.mjs +39 -0
  11. package/dist/init-BWG5OrQa.mjs +65 -0
  12. package/dist/rag-BnCMnccf.mjs +173 -0
  13. package/dist/secret-CzeHIGzE.mjs +50 -0
  14. package/dist/start-C1qkhU4O.mjs +23 -0
  15. package/package.json +39 -0
  16. package/templates/_shared/.env.example +5 -0
  17. package/templates/_shared/CLAUDE.md +1051 -0
  18. package/templates/_shared/biome.json +32 -0
  19. package/templates/_shared/global.d.ts +1 -0
  20. package/templates/_shared/index.html +16 -0
  21. package/templates/_shared/package.json +23 -0
  22. package/templates/_shared/tsconfig.json +15 -0
  23. package/templates/code-interpreter/agent.ts +27 -0
  24. package/templates/code-interpreter/client.tsx +3 -0
  25. package/templates/css.d.ts +1 -0
  26. package/templates/dispatch-center/agent.ts +1227 -0
  27. package/templates/dispatch-center/client.tsx +505 -0
  28. package/templates/embedded-assets/agent.ts +48 -0
  29. package/templates/embedded-assets/client.tsx +3 -0
  30. package/templates/embedded-assets/knowledge.json +20 -0
  31. package/templates/health-assistant/agent.ts +160 -0
  32. package/templates/health-assistant/client.tsx +3 -0
  33. package/templates/infocom-adventure/agent.ts +164 -0
  34. package/templates/infocom-adventure/client.tsx +300 -0
  35. package/templates/math-buddy/agent.ts +21 -0
  36. package/templates/math-buddy/client.tsx +3 -0
  37. package/templates/memory-agent/agent.ts +20 -0
  38. package/templates/memory-agent/client.tsx +3 -0
  39. package/templates/night-owl/agent.ts +98 -0
  40. package/templates/night-owl/client.tsx +12 -0
  41. package/templates/personal-finance/agent.ts +26 -0
  42. package/templates/personal-finance/client.tsx +3 -0
  43. package/templates/pizza-ordering/agent.ts +218 -0
  44. package/templates/pizza-ordering/client.tsx +264 -0
  45. package/templates/simple/agent.ts +6 -0
  46. package/templates/simple/client.tsx +3 -0
  47. package/templates/smart-research/agent.ts +164 -0
  48. package/templates/smart-research/client.tsx +3 -0
  49. package/templates/solo-rpg/agent.ts +1244 -0
  50. package/templates/solo-rpg/client.tsx +698 -0
  51. package/templates/support/README.md +62 -0
  52. package/templates/support/agent.ts +19 -0
  53. package/templates/support/client.tsx +3 -0
  54. package/templates/travel-concierge/agent.ts +29 -0
  55. package/templates/travel-concierge/client.tsx +3 -0
  56. package/templates/tsconfig.json +1 -0
  57. package/templates/web-researcher/agent.ts +17 -0
  58. package/templates/web-researcher/client.tsx +3 -0
@@ -0,0 +1,1051 @@
1
+ # aai Voice Agent Project
2
+
3
+ You are helping a user build a voice agent using the **aai** framework.
4
+
5
+ ## Workflow
6
+
7
+ 1. **Understand** — Restate what the user wants to build. If the request is
8
+ vague, ask a clarifying question before writing code.
9
+ 2. **Check existing work** — Look for a template or built-in tool that already
10
+ does what the user needs before writing custom code.
11
+ 3. **Start minimal** — Scaffold from the closest template, then layer on
12
+ customizations. Don't over-engineer the first version.
13
+ 4. **Verify** — After every change, run `aai build` to validate the bundle and
14
+ catch errors. Fix all errors before presenting work to the user.
15
+ 5. **Iterate** — Make small, focused changes. Verify each change works before
16
+ moving on.
17
+
18
+ ## Key rules
19
+
20
+ - Every agent lives in `agent.ts` and exports a default `defineAgent()` call
21
+ - Custom UI goes in `client.tsx` alongside `agent.ts`
22
+ - Optimize `instructions` for spoken conversation — short sentences, no visual
23
+ formatting, no exclamation points
24
+ - Never hardcode secrets — use `aai secret put` and access via `ctx.env`
25
+ - Tool `execute` return values go into LLM context — filter and truncate large
26
+ API responses
27
+ - Agent code runs in a sandboxed worker — use `fetch` (proxied) for HTTP,
28
+ `ctx.env` for secrets
29
+
30
+ ## CLI commands
31
+
32
+ ```sh
33
+ aai init # Scaffold a new agent (uses simple template)
34
+ aai init -t <template> # Scaffold from a specific template
35
+ aai dev # Start local dev server
36
+ aai build # Bundle and validate (no server or deploy)
37
+ aai deploy # Bundle and deploy to production
38
+ aai deploy -y # Deploy without prompts
39
+ aai deploy --dry-run # Validate and bundle without deploying
40
+ aai secret put <NAME> # Set a secret on the server (prompts for value)
41
+ aai secret delete <NAME> # Remove a secret
42
+ aai secret list # List secret names
43
+ aai secret pull # Pull secret names into .env for local dev
44
+ aai rag <url> # Ingest a site's llms-full.txt into the vector store
45
+ ```
46
+
47
+ ## Templates
48
+
49
+ Before writing an agent from scratch, choose the closest template and scaffold
50
+ with `aai init -t <template_name>`.
51
+
52
+ | Template | Description |
53
+ | ------------------- | ---------------------------------------------------------------------------------- |
54
+ | `simple` | Minimal starter with web_search, visit_webpage, fetch_json, run_code. **Default.** |
55
+ | `web-researcher` | Research assistant — web search + page visits for detailed answers |
56
+ | `smart-research` | Phase-based research (gather → analyze → respond) with dynamic tool filtering |
57
+ | `memory-agent` | Persistent KV storage — remembers facts and preferences across conversations |
58
+ | `code-interpreter` | Writes and runs JavaScript for math, calculations, data processing |
59
+ | `math-buddy` | Calculations, unit conversions, dice rolls via run_code |
60
+ | `health-assistant` | Medication lookup, drug interactions, BMI, symptom guidance |
61
+ | `personal-finance` | Currency conversion, crypto prices, loan calculations, savings projections |
62
+ | `travel-concierge` | Trip planning, weather, flights, hotels, currency conversion |
63
+ | `night-owl` | Movie/music/book recs by mood, sleep calculator. **Has custom UI.** |
64
+ | `pizza-ordering` | Pizza order-taker with dynamic cart sidebar. **Has custom UI.** |
65
+ | `dispatch-center` | 911 dispatch with incident triage and resource assignment. **Has custom UI.** |
66
+ | `infocom-adventure` | Zork-style text adventure with state, puzzles, inventory. **Has custom UI.** |
67
+ | `solo-rpg` | Solo dark-fantasy RPG with dice, oaths, combat, save/load. **Has custom UI.** |
68
+ | `embedded-assets` | FAQ bot using embedded JSON knowledge (no web search) |
69
+ | `support` | RAG-powered support agent using vector_search (AssemblyAI docs example) |
70
+
71
+ ## Minimal agent
72
+
73
+ Every agent lives in `agent.ts` and exports a default `defineAgent()` call:
74
+
75
+ ```ts
76
+ import { defineAgent } from "aai";
77
+
78
+ export default defineAgent({
79
+ name: "My Agent",
80
+ instructions: "You are a helpful assistant that...",
81
+ greeting: "Hey there. What can I help you with?",
82
+ });
83
+ ```
84
+
85
+ ### Imports
86
+
87
+ ```ts
88
+ import { defineAgent, tool } from "aai"; // defineAgent + helpers
89
+ import type { BeforeStepResult, BuiltinTool, HookContext, StepInfo, ToolContext } from "aai";
90
+ import { z } from "zod"; // Tools with typed params (included in package.json)
91
+ ```
92
+
93
+ ## Agent configuration
94
+
95
+ ```ts
96
+ defineAgent({
97
+ // Core
98
+ name: string; // Required: display name
99
+ instructions?: string; // System prompt (default: general voice assistant)
100
+ greeting?: string; // Spoken on connect (default: "Hey, how can I help you?")
101
+ // Speech
102
+ sttPrompt?: string; // STT guidance for jargon, names, acronyms
103
+
104
+ // Tools
105
+ builtinTools?: BuiltinTool[];
106
+ tools?: Record<string, ToolDef>;
107
+ toolChoice?: ToolChoice; // "auto" | "required" | "none" | { type: "tool", toolName }
108
+ activeTools?: string[]; // Default active tools per turn (subset of all tools)
109
+ maxSteps?: number | ((ctx: HookContext) => number);
110
+
111
+ // State
112
+ state?: () => S; // Factory for per-session state
113
+
114
+ // Lifecycle hooks
115
+ onConnect?: (ctx: HookContext) => void | Promise<void>;
116
+ onDisconnect?: (ctx: HookContext) => void | Promise<void>;
117
+ onError?: (error: Error, ctx?: HookContext) => void;
118
+ onTurn?: (text: string, ctx: HookContext) => void | Promise<void>;
119
+ onStep?: (step: StepInfo, ctx: HookContext) => void | Promise<void>;
120
+ onBeforeStep?: (stepNumber: number, ctx: HookContext) =>
121
+ BeforeStepResult | Promise<BeforeStepResult>;
122
+ });
123
+ ```
124
+
125
+ Use `sttPrompt` for domain-specific vocabulary:
126
+
127
+ ```ts
128
+ export default defineAgent({
129
+ name: "Tech Support",
130
+ sttPrompt: "Transcribe technical terms: Kubernetes, gRPC, PostgreSQL",
131
+ });
132
+ ```
133
+
134
+ ### Writing good `instructions`
135
+
136
+ Optimize for spoken conversation:
137
+
138
+ - Short, punchy sentences — optimize for speech, not text
139
+ - Never mention "search results" or "sources" — speak as if knowledge is your
140
+ own
141
+ - No visual formatting ("bullet point", "bold") — use "First", "Next", "Finally"
142
+ - Lead with the most important information
143
+ - Be concise and confident — no hedging ("It seems that", "I believe")
144
+ - No exclamation points — calm, conversational tone
145
+ - Define personality, tone, and specialty
146
+ - Include when and how to use each tool
147
+
148
+ ### Environment variables
149
+
150
+ Secrets are managed on the server via the CLI, like `vercel env`. They are
151
+ injected into agent workers at runtime and available as `ctx.env`. Secrets are
152
+ **never** embedded in the bundled code.
153
+
154
+ ```sh
155
+ # Set secrets on the server (prompts for value)
156
+ aai secret put ASSEMBLYAI_API_KEY
157
+ aai secret put MY_API_KEY
158
+
159
+ # List what's set
160
+ aai secret list
161
+
162
+ # Pull secret names into .env for local dev reference
163
+ aai secret pull
164
+
165
+ # Remove a secret
166
+ aai secret delete MY_API_KEY
167
+ ```
168
+
169
+ Access secrets in tool code via `ctx.env`:
170
+
171
+ ```ts
172
+ import { defineAgent } from "aai";
173
+ import { z } from "zod";
174
+
175
+ export default defineAgent({
176
+ name: "API Agent",
177
+ tools: {
178
+ call_api: {
179
+ description: "Call an external API",
180
+ parameters: z.object({ query: z.string() }),
181
+ execute: async (args, ctx) => {
182
+ const res = await fetch(`https://api.example.com?q=${args.query}`, {
183
+ headers: { Authorization: `Bearer ${ctx.env.MY_API_KEY}` },
184
+ });
185
+ return res.json();
186
+ },
187
+ },
188
+ },
189
+ });
190
+ ```
191
+
192
+ ## Tools
193
+
194
+ ### Custom tools
195
+
196
+ Define tools as plain objects in the `tools` record. The `parameters` field
197
+ takes a Zod schema for type-safe argument inference:
198
+
199
+ ```ts
200
+ import { defineAgent, tool } from "aai";
201
+ import { z } from "zod";
202
+
203
+ export default defineAgent({
204
+ name: "Weather Agent",
205
+ tools: {
206
+ get_weather: tool({
207
+ description: "Get current weather for a city",
208
+ parameters: z.object({
209
+ city: z.string().describe("City name"),
210
+ }),
211
+ execute: async ({ city }, ctx) => {
212
+ // city is typed as string (inferred from Zod schema)
213
+ const data = await fetch(
214
+ `https://api.example.com/weather?q=${city}`,
215
+ );
216
+ return data.json();
217
+ },
218
+ }),
219
+
220
+ // No-parameter tools — omit `parameters` and `tool()` wrapper
221
+ list_items: {
222
+ description: "List all items",
223
+ execute: () => items,
224
+ },
225
+ },
226
+ });
227
+ ```
228
+
229
+ **Important:** Wrap tool definitions in `tool()` to get typed `args` inferred
230
+ from the Zod `parameters` schema. Without `tool()`, args are untyped.
231
+
232
+ Zod schema patterns:
233
+
234
+ ```ts
235
+ parameters: z.object({
236
+ query: z.string().describe("Search query"),
237
+ category: z.enum(["a", "b", "c"]),
238
+ count: z.number().describe("How many"),
239
+ label: z.string().describe("Optional label").optional(),
240
+ }),
241
+ ```
242
+
243
+ ### Built-in tools
244
+
245
+ Enable via `builtinTools`.
246
+
247
+ | Tool | Description | Params |
248
+ | --------------- | ---------------------------------------------- | ----------------------------------- |
249
+ | `web_search` | Search the web (Brave Search) | `query`, `max_results?` (default 5) |
250
+ | `visit_webpage` | Fetch URL → Markdown | `url` |
251
+ | `fetch_json` | HTTP GET a JSON API | `url`, `headers?` |
252
+ | `run_code` | Execute JS in sandbox (no net/fs, 5s timeout) | `code` |
253
+ | `vector_search` | Search the agent's RAG knowledge base | `query`, `topK?` (default 5) |
254
+ | `memory` | Persistent KV memory (4 tools, see below) | — |
255
+
256
+ The agentic loop runs up to `maxSteps` iterations (default 5) and stops when the
257
+ LLM produces a text response.
258
+
259
+ ### Tool context
260
+
261
+ Every `execute` function and lifecycle hook receives a context object:
262
+
263
+ ```ts
264
+ ctx.env; // Record<string, string> — secrets from `aai secret put`
265
+ ctx.state; // per-session state
266
+ ctx.kv; // persistent KV store
267
+ ctx.vector; // VectorStore — vector store for RAG (tools only)
268
+ ctx.messages; // readonly Message[] — conversation history (tools only)
269
+ ```
270
+
271
+ Hooks get `HookContext` (same but without `messages`).
272
+
273
+ **Timeouts:** Tool execution times out after **30 seconds**. Lifecycle hooks
274
+ (`onConnect`, `onTurn`, etc.) time out after **5 seconds**.
275
+
276
+ ### Fetching external APIs
277
+
278
+ Use `fetch` directly in tool execute functions:
279
+
280
+ ```ts
281
+ execute: async (args, ctx) => {
282
+ const resp = await fetch(url, {
283
+ headers: { Authorization: `Bearer ${ctx.env.API_KEY}` },
284
+ });
285
+ if (!resp.ok) return { error: `${resp.status} ${resp.statusText}` };
286
+ return resp.json();
287
+ },
288
+ ```
289
+
290
+ `fetch` is proxied through the host process (the worker has no direct network
291
+ access). All URLs are validated against SSRF rules — only public addresses are
292
+ allowed.
293
+
294
+ ## State and storage
295
+
296
+ ### Per-session state
297
+
298
+ For data that lasts only one connection (games, workflows, multi-step
299
+ processes). Fresh state is created per session and cleaned up on disconnect:
300
+
301
+ ```ts
302
+ export default defineAgent({
303
+ state: () => ({ score: 0, question: 0 }),
304
+ tools: {
305
+ answer: {
306
+ description: "Submit an answer",
307
+ parameters: z.object({ answer: z.string() }),
308
+ execute: (args, ctx) => {
309
+ const state = ctx.state as { score: number; question: number };
310
+ state.question++;
311
+ return state;
312
+ },
313
+ },
314
+ },
315
+ });
316
+ ```
317
+
318
+ ### Persisting state across reconnects
319
+
320
+ Use the KV store to auto-save and auto-load state:
321
+
322
+ ```ts
323
+ export default defineAgent({
324
+ state: () => ({ score: 0, initialized: false }),
325
+ onConnect: async (ctx) => {
326
+ const saved = await ctx.kv.get("save:game");
327
+ if (saved) Object.assign(ctx.state, saved);
328
+ },
329
+ onTurn: async (_text, ctx) => {
330
+ await ctx.kv.set("save:game", ctx.state);
331
+ },
332
+ });
333
+ ```
334
+
335
+ This works for games, workflows,
336
+ or any agent where users expect to resume where they left off.
337
+
338
+ ### Persistent storage (KV)
339
+
340
+ `ctx.kv` is a persistent key-value store scoped per agent. Values are
341
+ auto-serialized as JSON.
342
+
343
+ ```ts
344
+ await ctx.kv.set("user:123", { name: "Alice" }); // save
345
+ await ctx.kv.set("temp:x", value, { expireIn: 60_000 }); // save with TTL (ms)
346
+ const user = await ctx.kv.get<User>("user:123"); // read (or null)
347
+ const notes = await ctx.kv.list("note:", { limit: 10, reverse: true }); // list by prefix
348
+ const allKeys = await ctx.kv.keys(); // all keys
349
+ const userKeys = await ctx.kv.keys("user:*"); // keys matching glob pattern
350
+ await ctx.kv.delete("user:123"); // delete
351
+ ```
352
+
353
+ Keys are strings; use colon-separated prefixes (`"user:123"`). Max value: 64 KB.
354
+
355
+ `kv.list()` returns `KvEntry[]` where each entry has
356
+ `{ key: string, value: T }`.
357
+
358
+ ### Memory tools (pre-built KV tools)
359
+
360
+ Add `"memory"` to `builtinTools` to give the agent four persistent KV tools:
361
+ `save_memory`, `recall_memory`, `list_memories`, and `forget_memory`.
362
+
363
+ ```ts
364
+ import { defineAgent } from "aai";
365
+
366
+ export default defineAgent({
367
+ name: "My Agent",
368
+ builtinTools: ["memory"],
369
+ });
370
+ ```
371
+
372
+ Keys use colon-separated prefixes (`"user:name"`, `"preference:color"`).
373
+
374
+ ## Advanced patterns
375
+
376
+ ### Step hooks
377
+
378
+ `onStep` — called after each LLM step (logging, analytics):
379
+
380
+ ```ts
381
+ onStep: (step, ctx) => {
382
+ console.log(`Step ${step.stepNumber}: ${step.toolCalls.length} tool calls`);
383
+ },
384
+ ```
385
+
386
+ `onBeforeStep` — return `{ activeTools: [...] }` to filter tools per step:
387
+
388
+ ```ts
389
+ state: () => ({ phase: "gather" }),
390
+ onBeforeStep: (stepNumber, ctx) => {
391
+ const state = ctx.state as { phase: string };
392
+ if (state.phase === "gather") {
393
+ return { activeTools: ["search", "lookup"] };
394
+ }
395
+ return { activeTools: ["summarize"] };
396
+ },
397
+ ```
398
+
399
+ ### Tool choice
400
+
401
+ Control when the LLM uses tools:
402
+
403
+ ```ts
404
+ toolChoice: "auto", // Default — LLM decides when to use tools
405
+ toolChoice: "required", // Force a tool call every step (useful for research pipelines)
406
+ toolChoice: "none", // Disable all tool use
407
+ toolChoice: { type: "tool", toolName: "search" }, // Force a specific tool
408
+ ```
409
+
410
+ ### Phase-based tool filtering
411
+
412
+ Combine `state`, `onBeforeStep`, and `activeTools` for multi-phase workflows:
413
+
414
+ ```ts
415
+ state: () => ({ phase: "gather" as "gather" | "analyze" | "respond" }),
416
+ onBeforeStep: (_step, ctx) => {
417
+ const state = ctx.state as { phase: string };
418
+ if (state.phase === "gather") return { activeTools: ["web_search", "advance"] };
419
+ if (state.phase === "analyze") return { activeTools: ["summarize", "advance"] };
420
+ return { activeTools: [] }; // respond phase — LLM speaks freely
421
+ },
422
+ tools: {
423
+ advance: {
424
+ description: "Move to the next phase",
425
+ execute: (_args, ctx) => {
426
+ const state = ctx.state as { phase: string };
427
+ if (state.phase === "gather") state.phase = "analyze";
428
+ else if (state.phase === "analyze") state.phase = "respond";
429
+ return { phase: state.phase };
430
+ },
431
+ },
432
+ },
433
+ ```
434
+
435
+ ### Static `activeTools`
436
+
437
+ Restrict which tools the LLM can use by default, without writing a hook:
438
+
439
+ ```ts
440
+ export default defineAgent({
441
+ builtinTools: ["web_search", "visit_webpage", "run_code"],
442
+ tools: { summarize: {/* ... */} },
443
+ activeTools: ["web_search", "summarize"], // Only these two are available
444
+ });
445
+ ```
446
+
447
+ Use `onBeforeStep` to override `activeTools` dynamically per step.
448
+
449
+ ### Dynamic `maxSteps`
450
+
451
+ ```ts
452
+ maxSteps: (ctx) => {
453
+ const state = ctx.state as { complexity: string };
454
+ return state.complexity === "complex" ? 10 : 5;
455
+ },
456
+ ```
457
+
458
+ ### Conversation history in tools
459
+
460
+ ```ts
461
+ execute: (args, ctx) => {
462
+ const userMessages = ctx.messages.filter(m => m.role === "user");
463
+ return { turns: userMessages.length };
464
+ },
465
+ ```
466
+
467
+ ### Embedded knowledge
468
+
469
+ ```ts
470
+ import knowledge from "./knowledge.json" with { type: "json" };
471
+
472
+ export default defineAgent({
473
+ tools: {
474
+ search_faq: {
475
+ description: "Search the knowledge base",
476
+ parameters: z.object({ query: z.string() }),
477
+ execute: (args) =>
478
+ knowledge.faqs.filter((f: { question: string }) =>
479
+ f.question.toLowerCase().includes(args.query.toLowerCase())
480
+ ),
481
+ },
482
+ },
483
+ });
484
+ ```
485
+
486
+ ### Using npm packages
487
+
488
+ Add packages to `package.json` dependencies:
489
+
490
+ ```sh
491
+ npm install some-package
492
+ ```
493
+
494
+ ## Custom UI (`client.tsx`)
495
+
496
+ Add `client.tsx` alongside `agent.ts`. Define a Preact component and call
497
+ `mount()` to render it. Use JSX syntax:
498
+
499
+ ```tsx
500
+ import "aai-ui/styles.css";
501
+ import { mount, useSession } from "aai-ui";
502
+
503
+ function App() {
504
+ const { session, started, running, start, toggle, reset } = useSession();
505
+ const msgs = session.messages.value;
506
+ const tx = session.userUtterance.value;
507
+ return (
508
+ <div>
509
+ {msgs.map((m, i) => <p key={i}>{m.text}</p>)}
510
+ {tx !== null && <p>{tx || "..."}</p>}
511
+ {!started.value ? <button onClick={start}>Start</button> : (
512
+ <>
513
+ <button onClick={toggle}>{running.value ? "Stop" : "Resume"}</button>
514
+ <button onClick={reset}>Reset</button>
515
+ </>
516
+ )}
517
+ </div>
518
+ );
519
+ }
520
+
521
+ mount(App);
522
+ ```
523
+
524
+ **Rules:**
525
+
526
+ - Always import `"aai-ui/styles.css"` at the top — without it, default styles
527
+ won't load
528
+ - Call `mount(YourComponent)` at the end of the file
529
+ - Use `.tsx` file extension for JSX syntax
530
+ - Import hooks from `preact/hooks` (`useEffect`, `useRef`, `useState`, etc.)
531
+ - Style with `style={{ color: "red" }}` or inject `<style>` for selectors,
532
+ keyframes, media queries
533
+
534
+ ### `mount()` options
535
+
536
+ ```ts
537
+ mount(App, {
538
+ target: "#app", // CSS selector or DOM element (default: "#app")
539
+ platformUrl: "...", // Server URL (auto-derived from location.href)
540
+ title: "My Agent", // Shown in header and start screen
541
+ theme: { // CSS custom property overrides
542
+ bg: "#101010", // Background color
543
+ primary: "#fab283", // Accent color
544
+ text: "#ffffff", // Text color
545
+ surface: "#1a1a1a", // Card/surface color
546
+ border: "#333333", // Border color
547
+ },
548
+ });
549
+ ```
550
+
551
+ `mount()` returns a `MountHandle` with `session`, `signals`, and `dispose()`.
552
+
553
+ ### Built-in components
554
+
555
+ Import from `aai-ui`:
556
+
557
+ **Layout components:**
558
+
559
+ | Component | Description |
560
+ | --------------- | ---------------------------------------------------- |
561
+ | `App` | Default full UI (StartScreen + ChatView) |
562
+ | `StartScreen` | Centered start card; renders children after start |
563
+ | `ChatView` | Chat interface (header + messages + controls) |
564
+ | `SidebarLayout` | Two-column layout with sidebar + main area |
565
+ | `Controls` | Stop/Resume + New Conversation buttons |
566
+ | `MessageList` | Messages with auto-scroll, tool calls, transcript |
567
+
568
+ `StartScreen` props: `{ children, icon?, title?, subtitle?, buttonText? }`
569
+ `SidebarLayout` props: `{ sidebar, children, width?, side? }`
570
+
571
+ **Atomic components:**
572
+
573
+ | Component | Props | Description |
574
+ | ------------------- | --------------------------------------- | ------------------------------- |
575
+ | `MessageBubble` | `{ message: Message }` | Single message bubble |
576
+ | `Transcript` | `{ userUtterance: Signal<str\|null> }` | Live STT text display |
577
+ | `StateIndicator` | `{ state: Signal<AgentState> }` | Colored dot + state label |
578
+ | `ErrorBanner` | `{ error: Signal<SessionError\|null> }` | Red error box with message |
579
+ | `ThinkingIndicator` | none | Animated dots during processing |
580
+ | `ToolCallBlock` | `{ toolCall: ToolCallInfo }` | Collapsible tool call display |
581
+
582
+ **Hooks:**
583
+
584
+ - `useAutoScroll()` — returns a `RefObject<HTMLDivElement>` to attach to a
585
+ sentinel div. Auto-scrolls when messages or utterances change.
586
+ - `useMountConfig()` — returns the `title` and `theme` passed to `mount()`.
587
+
588
+ **Important:** Components that accept `Signal<T>` props (like `StateIndicator`,
589
+ `Transcript`, `ErrorBanner`) expect the Signal object itself, NOT `.value`. Pass
590
+ `session.state`, not `session.state.value`. Passing `.value` compiles but breaks
591
+ reactivity silently.
592
+
593
+ ### Session signals (`useSession()`)
594
+
595
+ `useSession()` returns
596
+ `{ session, started, running, start, toggle, reset, dispose }`. Reactive agent
597
+ data lives on `session` (a `VoiceSession`); UI-only controls are top-level.
598
+
599
+ | Signal / field | Type | Description |
600
+ | ------------------------------ | ---------------------- | --------------------------------------------------------------- |
601
+ | `session.state.value` | `AgentState` | "disconnected", "connecting", "ready", "listening", etc. |
602
+ | `session.messages.value` | `Message[]` | `{ role, text }` objects |
603
+ | `session.toolCalls.value` | `ToolCallInfo[]` | `{ toolCallId, toolName, args, status, result? }` — tool calls |
604
+ | `session.userUtterance.value` | `string \| null` | `null` = not speaking, `""` = speech detected, string = text |
605
+ | `session.agentUtterance.value` | `string \| null` | `null` = not speaking, string = streaming agent response text |
606
+ | `session.error.value` | `SessionError \| null` | `{ code, message }` |
607
+ | `session.disconnected.value` | `object \| null` | `{ intentional: boolean }` when disconnected, `null` otherwise |
608
+ | `started.value` | `boolean` | Whether session has been started |
609
+ | `running.value` | `boolean` | Whether session is active |
610
+
611
+ **Methods:** `start()`, `toggle()`, `reset()`, `dispose()`
612
+
613
+ **Hooks:**
614
+
615
+ | Hook | Description |
616
+ | --------------------------------------------- | -------------------------------------------------------------------------------------------- |
617
+ | `useToolResult((toolName, result, tc) => {})` | Fires once per completed tool call with parsed JSON result. Use for carts, scoreboards, etc. |
618
+
619
+ **Signal semantics for utterances:**
620
+
621
+ - `userUtterance`: `null` = user is not speaking, `""` = speech detected but
622
+ no text yet (show "..."), non-empty string = partial/final transcript
623
+ - `agentUtterance`: `null` = agent is not speaking, non-empty string =
624
+ streaming response text (cleared when final `chat` message arrives)
625
+ - `disconnected`: `null` = connected, `{ intentional: true }` = user
626
+ disconnected, `{ intentional: false }` = unexpected disconnect (show
627
+ reconnect UI)
628
+
629
+ **UI Message type:** `{ role: "user" | "assistant"; content: string }`. Both UI
630
+ and SDK `Message` types use `content`.
631
+
632
+ ### Showing tool calls in custom UI
633
+
634
+ ```tsx
635
+ import "aai-ui/styles.css";
636
+ import { mount, ToolCallBlock, useSession } from "aai-ui";
637
+
638
+ function App() {
639
+ const { session, started, start } = useSession();
640
+ if (!started.value) return <button onClick={start}>Start</button>;
641
+
642
+ const msgs = session.messages.value;
643
+ const toolCalls = session.toolCalls.value;
644
+
645
+ return (
646
+ <div>
647
+ {msgs.map((m, i) => (
648
+ <div key={i}>
649
+ <p>{m.text}</p>
650
+ {toolCalls
651
+ .filter((tc) => tc.afterMessageIndex === i)
652
+ .map((tc) => <ToolCallBlock key={tc.toolCallId} toolCall={tc} />)}
653
+ </div>
654
+ ))}
655
+ </div>
656
+ );
657
+ }
658
+
659
+ mount(App);
660
+ ```
661
+
662
+ ### Building dynamic UI from tool results
663
+
664
+ Use `useToolResult` to update local state (carts, scoreboards, dashboards)
665
+ whenever a tool completes. It fires exactly once per completed tool call with
666
+ the parsed JSON result, handling deduplication internally.
667
+
668
+ ```tsx
669
+ import "aai-ui/styles.css";
670
+ import { useState } from "preact/hooks";
671
+ import { ChatView, SidebarLayout, StartScreen, mount, useToolResult } from "aai-ui";
672
+
673
+ interface CartItem { id: number; name: string; price: number }
674
+
675
+ function ShopAgent() {
676
+ const [cart, setCart] = useState<CartItem[]>([]);
677
+
678
+ useToolResult((toolName, result: any) => {
679
+ switch (toolName) {
680
+ case "add_item":
681
+ setCart((prev) => [...prev, result.item]);
682
+ break;
683
+ case "remove_item":
684
+ setCart((prev) => prev.filter((i) => i.id !== result.removedId));
685
+ break;
686
+ case "clear_cart":
687
+ setCart([]);
688
+ break;
689
+ }
690
+ });
691
+
692
+ const sidebar = (
693
+ <div class="p-4">
694
+ <h3 class="text-aai-text font-bold">Cart ({cart.length})</h3>
695
+ {cart.map((i) => <p key={i.id} class="text-aai-text text-sm">{i.name} — ${i.price}</p>)}
696
+ </div>
697
+ );
698
+
699
+ return (
700
+ <StartScreen title="Shop" buttonText="Start Shopping">
701
+ <SidebarLayout sidebar={sidebar}>
702
+ <ChatView />
703
+ </SidebarLayout>
704
+ </StartScreen>
705
+ );
706
+ }
707
+
708
+ mount(ShopAgent);
709
+ ```
710
+
711
+ **Do NOT use `useEffect` + `session.toolCalls.value` to build derived state.**
712
+ That pattern re-processes every tool call on every signal change, causing
713
+ duplicates (e.g. items added to the cart multiple times). `useToolResult`
714
+ handles this correctly.
715
+
716
+ ### Reacting to agent state
717
+
718
+ ```tsx
719
+ import "aai-ui/styles.css";
720
+ import { useEffect } from "preact/hooks";
721
+ import { mount, StateIndicator, useSession } from "aai-ui";
722
+
723
+ function App() {
724
+ const { session, started, start } = useSession();
725
+
726
+ useEffect(() => {
727
+ // Run side effects when state changes
728
+ if (session.state.value === "speaking") {
729
+ // Agent is speaking — e.g., show animation
730
+ }
731
+ }, [session.state.value]);
732
+
733
+ return (
734
+ <div>
735
+ <StateIndicator />
736
+ {!started.value && <button onClick={start}>Start</button>}
737
+ </div>
738
+ );
739
+ }
740
+
741
+ mount(App);
742
+ ```
743
+
744
+ ### Styling custom UIs
745
+
746
+ The framework uses **Tailwind CSS v4** (compiled at bundle time). Prefer
747
+ Tailwind classes over inline styles — all design tokens work as classes:
748
+ `bg-aai-surface` not `style={{ background: "var(--color-aai-surface)" }}`,
749
+ `border-t border-aai-border` not `style={{ borderTop: "1px solid var(--)" }}`.
750
+
751
+ Three approaches:
752
+
753
+ 1. **Tailwind classes** — `class="flex items-center gap-2 bg-aai-surface"`
754
+ 2. **Inline styles** — only for dynamic values (`style={{ width: pixels }}`)
755
+ 3. **Injected `<style>` tags** — for keyframes, selectors, media queries:
756
+
757
+ ```tsx
758
+ function App() {
759
+ return (
760
+ <>
761
+ <style>
762
+ {`
763
+ @keyframes pulse { 0%, 100% { opacity: 1 } 50% { opacity: 0.5 } }
764
+ .pulse { animation: pulse 2s ease-in-out infinite; }
765
+ @media (max-width: 640px) { .sidebar { display: none; } }
766
+ `}
767
+ </style>
768
+ <div class="pulse">Content</div>
769
+ </>
770
+ );
771
+ }
772
+ ```
773
+
774
+ **Design tokens** — available as CSS custom properties and Tailwind classes
775
+ (e.g. `bg-aai-bg`, `text-aai-text-muted`, `rounded-aai`, `font-aai`):
776
+
777
+ | Token | Tailwind class | Default |
778
+ | ---------------------------- | ------------------------- | ------------------------- |
779
+ | `--color-aai-bg` | `bg-aai-bg` | `#101010` |
780
+ | `--color-aai-surface` | `bg-aai-surface` | `#151515` |
781
+ | `--color-aai-surface-faint` | `bg-aai-surface-faint` | `rgba(255,255,255,0.031)` |
782
+ | `--color-aai-surface-hover` | `bg-aai-surface-hover` | `rgba(255,255,255,0.059)` |
783
+ | `--color-aai-border` | `border-aai-border` | `#282828` |
784
+ | `--color-aai-primary` | `text-aai-primary` | `#fab283` |
785
+ | `--color-aai-text` | `text-aai-text` | `rgba(255,255,255,0.936)` |
786
+ | `--color-aai-text-secondary` | `text-aai-text-secondary` | `rgba(255,255,255,0.618)` |
787
+ | `--color-aai-text-muted` | `text-aai-text-muted` | `rgba(255,255,255,0.284)` |
788
+ | `--color-aai-text-dim` | `text-aai-text-dim` | `rgba(255,255,255,0.422)` |
789
+ | `--color-aai-error` | `text-aai-error` | `#e06c75` |
790
+ | `--color-aai-ring` | `ring-aai-ring` | `#56b6c2` |
791
+ | `--color-aai-state-{state}` | `text-aai-state-{state}` | per-state colors |
792
+ | `--radius-aai` | `rounded-aai` | `6px` |
793
+ | `--font-aai` | `font-aai` | Inter, sans-serif |
794
+ | `--font-aai-mono` | `font-aai-mono` | IBM Plex Mono, mono |
795
+
796
+ The 5 core colors (`bg`, `primary`, `text`, `surface`, `border`) can be
797
+ overridden via `mount()` theme options. All other tokens use fixed defaults.
798
+
799
+ ### Common UI patterns
800
+
801
+ **Auto-scrolling messages** — use `useAutoScroll` for custom message lists:
802
+
803
+ ```tsx
804
+ import { useAutoScroll, useSession } from "aai-ui";
805
+
806
+ function MyChat() {
807
+ const { session } = useSession();
808
+ const bottomRef = useAutoScroll();
809
+
810
+ return (
811
+ <div class="overflow-y-auto">
812
+ {session.messages.value.map((m, i) => <p key={i}>{m.text}</p>)}
813
+ <div ref={bottomRef} />
814
+ </div>
815
+ );
816
+ }
817
+ ```
818
+
819
+ Note: `MessageList` and `ChatView` already include auto-scroll. Only use
820
+ `useAutoScroll` when building a fully custom message list.
821
+
822
+ **Reading signal values in render:** Extract `.value` once at the top of the
823
+ component to avoid redundant signal subscriptions:
824
+
825
+ ```tsx
826
+ function MyComponent() {
827
+ const { session } = useSession();
828
+ const state = session.state.value;
829
+ const msgs = session.messages.value;
830
+ // Use `state` and `msgs` as plain values throughout the render
831
+ }
832
+ ```
833
+
834
+ ## Self-hosting with `createServer()`
835
+
836
+ Agents can run anywhere (Node, Docker) without the managed platform:
837
+
838
+ ```ts
839
+ import { defineAgent } from "aai";
840
+ import { createServer } from "aai/server";
841
+
842
+ const agent = defineAgent({
843
+ name: "My Agent",
844
+ instructions: "You are a helpful assistant.",
845
+ });
846
+
847
+ const server = createServer({
848
+ agent,
849
+ clientDir: "public", // optional: serve static files
850
+ });
851
+
852
+ await server.listen(3000);
853
+ ```
854
+
855
+ Run with `node server.ts` (Node >=22.6 strips types natively) or bundle
856
+ with your preferred tool. The server handles WebSocket connections, STT/TTS,
857
+ and the agentic loop. Set `ASSEMBLYAI_API_KEY` as an environment variable.
858
+
859
+ ## Useful free API endpoints
860
+
861
+ These public APIs require no auth and work well in voice agents:
862
+
863
+ ```text
864
+ Weather (Open-Meteo):
865
+ Geocode: https://geocoding-api.open-meteo.com/v1/search?name={city}&count=1&language=en
866
+ Forecast: https://api.open-meteo.com/v1/forecast?latitude={lat}&longitude={lon}&daily=temperature_2m_max,temperature_2m_min,precipitation_sum,weathercode&timezone=auto&forecast_days=7
867
+
868
+ Currency (ExchangeRate):
869
+ Rates: https://open.er-api.com/v6/latest/{CODE} → { rates: { USD: 1.0, EUR: 0.85, ... } }
870
+
871
+ Crypto (CoinGecko):
872
+ Price: https://api.coingecko.com/api/v3/simple/price?ids={coin}&vs_currencies={cur}&include_24hr_change=true
873
+
874
+ Drug info (FDA):
875
+ Label: https://api.fda.gov/drug/label.json?search=openfda.generic_name:"{name}"&limit=1
876
+
877
+ Drug interactions (RxNorm):
878
+ RxCUI: https://rxnav.nlm.nih.gov/REST/rxcui.json?name={name}
879
+ Interactions: https://rxnav.nlm.nih.gov/REST/interaction/list.json?rxcuis={id1}+{id2}
880
+ ```
881
+
882
+ Use `fetch_json` builtin tool or `fetch` in custom tools to call these.
883
+
884
+ ## Custom start screen with `StartScreen`
885
+
886
+ Use `StartScreen` for a branded start card that transitions to `ChatView`:
887
+
888
+ ```tsx
889
+ import "aai-ui/styles.css";
890
+ import { ChatView, StartScreen, mount } from "aai-ui";
891
+
892
+ function MyAgent() {
893
+ return (
894
+ <StartScreen icon={<span>&#x1F3A4;</span>} title="My Agent" subtitle="Ask me anything">
895
+ <ChatView />
896
+ </StartScreen>
897
+ );
898
+ }
899
+
900
+ mount(MyAgent);
901
+ ```
902
+
903
+ `StartScreen` handles the started/not-started transition automatically. Pass
904
+ `icon`, `title`, `subtitle`, and `buttonText` to customize the card.
905
+
906
+ ## Sidebar layout with `SidebarLayout`
907
+
908
+ Use `SidebarLayout` for apps with a persistent side panel (cart, dashboard):
909
+
910
+ ```tsx
911
+ import "aai-ui/styles.css";
912
+ import { useState } from "preact/hooks";
913
+ import { ChatView, SidebarLayout, StartScreen, mount, useToolResult } from "aai-ui";
914
+
915
+ function ShopAgent() {
916
+ const [cart, setCart] = useState<{ id: number; name: string }[]>([]);
917
+
918
+ useToolResult((toolName, result: any) => {
919
+ if (toolName === "add_item") setCart((prev) => [...prev, result.item]);
920
+ });
921
+
922
+ const sidebar = (
923
+ <div class="p-4">
924
+ <h3 class="text-aai-text font-bold">Cart ({cart.length})</h3>
925
+ {cart.map((i) => <p key={i.id} class="text-aai-text text-sm">{i.name}</p>)}
926
+ </div>
927
+ );
928
+
929
+ return (
930
+ <StartScreen title="Shop" buttonText="Start Shopping">
931
+ <SidebarLayout sidebar={sidebar}>
932
+ <ChatView />
933
+ </SidebarLayout>
934
+ </StartScreen>
935
+ );
936
+ }
937
+
938
+ mount(ShopAgent);
939
+ ```
940
+
941
+ `SidebarLayout` accepts `width` (default `"20rem"`) and `side` (`"left"` or
942
+ `"right"`).
943
+
944
+ ## Project structure
945
+
946
+ After scaffolding, your project directory looks like:
947
+
948
+ ```text
949
+ my-agent/
950
+ agent.ts # Agent definition
951
+ client.tsx # UI component (calls mount() to render into #app)
952
+ styles.css # Tailwind CSS entry point
953
+ package.json # Dependencies, scripts, and config
954
+ tsconfig.json # TypeScript configuration
955
+ .env.example # Reference for env var names
956
+ .env # Local dev secrets (gitignored)
957
+ .gitignore # Ignores node_modules/, .aai/, .env, etc.
958
+ README.md # Getting started guide
959
+ CLAUDE.md # Agent API reference (always loaded by Claude Code)
960
+ .aai/ # Build output (managed by CLI, gitignored)
961
+ project.json # Deploy target (slug, server URL)
962
+ build/ # Bundle output
963
+ ```
964
+
965
+ ## Instructions patterns from templates
966
+
967
+ Good instructions tell the LLM what it is, how to behave, and when to use each
968
+ tool. Study these patterns:
969
+
970
+ **Code execution agent** — force tool use for anything computational:
971
+
972
+ ```text
973
+ You MUST use the run_code tool for ANY question involving math, counting,
974
+ string manipulation, or data processing. NEVER do mental math or estimate.
975
+ Use console.log() to output intermediate steps.
976
+ ```
977
+
978
+ **Research agent** — search before answering:
979
+
980
+ ```text
981
+ Search first. Never guess or rely on memory for factual questions.
982
+ Use visit_webpage when search snippets aren't detailed enough.
983
+ For complex questions, search multiple times with different queries.
984
+ ```
985
+
986
+ **FAQ/support agent** — stay grounded in knowledge:
987
+
988
+ ```text
989
+ Always use vector_search to find relevant documentation before answering.
990
+ Base your answers strictly on the retrieved documentation — don't guess.
991
+ If search results aren't relevant, say the docs don't cover that topic.
992
+ ```
993
+
994
+ **API-calling agent** — tell the LLM which endpoints to use:
995
+
996
+ ```text
997
+ API endpoints (use fetch_json):
998
+ - Currency rates: https://open.er-api.com/v6/latest/{CODE}
999
+ Returns { rates: { USD: 1.0, EUR: 0.85, ... } }
1000
+ - Weather: https://api.open-meteo.com/v1/forecast?latitude={lat}&longitude={lon}...
1001
+ ```
1002
+
1003
+ **Game/interactive agent** — establish world rules and voice style:
1004
+
1005
+ ```text
1006
+ You ARE the game. Maintain world state, describe rooms, handle puzzles.
1007
+ Keep descriptions to two to four sentences. No visual formatting.
1008
+ Use directional words naturally: "To the north you see..." not "N: forest"
1009
+ ```
1010
+
1011
+ ## Common pitfalls
1012
+
1013
+ - **Using `useEffect` to build state from tool calls** — Iterating
1014
+ `session.toolCalls.value` in a `useEffect` re-processes every tool call on
1015
+ every signal change, causing duplicates (e.g. cart items added multiple
1016
+ times). Use the `useToolResult` hook instead — it fires exactly once per
1017
+ completed tool call with proper deduplication.
1018
+ - **Writing `instructions` with visual formatting** — Bullets, bold, numbered
1019
+ lists sound terrible when spoken. Use natural transitions: "First", "Next",
1020
+ "Finally". Write instructions as if you're coaching a human phone operator.
1021
+ - **Returning huge payloads from tools** — Everything a tool returns goes into
1022
+ the LLM context. Filter, summarize, or truncate API responses before
1023
+ returning. Return only what the agent needs to formulate a spoken answer.
1024
+ - **Forgetting sandbox constraints** — Agent code runs in a sandboxed Worker
1025
+ with no direct network or filesystem access. Use `fetch` (proxied through the
1026
+ host) for HTTP. Use `ctx.env` for secrets. Direct network access will fail.
1027
+ - **Hardcoding secrets** — Never put API keys in `agent.ts`. Use
1028
+ `aai secret put MY_KEY` to store them on the server, then access via
1029
+ `ctx.env.MY_KEY`.
1030
+ - **Telling the agent to be verbose** — Voice responses should be 1-3 sentences.
1031
+ If your `instructions` say "provide detailed explanations", the agent will
1032
+ monologue. Instruct it to be brief and let the user ask follow-ups.
1033
+ - **Not setting secrets before deploying** — If your agent needs custom
1034
+ secrets, set them with `aai secret put MY_KEY` before deploying.
1035
+ - **Forgetting SSRF restrictions on `fetch`** — The host validates all proxied
1036
+ fetch URLs. Requests to private/internal IP addresses (localhost, 10.x,
1037
+ 192.168.x, etc.) are blocked.
1038
+
1039
+ ## Troubleshooting
1040
+
1041
+ - **"no agent found"** — Ensure `agent.ts` exists in the current directory
1042
+ - **"bundle failed"** — TypeScript syntax error — check imports, brackets
1043
+ - **"No .aai/project.json found"** — Run `aai deploy` first before using
1044
+ `aai secret`
1045
+ - **Tool returns `undefined`** — Make sure `execute` returns a value. Even
1046
+ `return { ok: true }` is better than an implicit void return.
1047
+ - **Agent doesn't use a tool** — Check `description` is clear about when to use
1048
+ it. The LLM relies on the description to decide. Also check `activeTools`
1049
+ isn't filtering it out.
1050
+ - **KV reads return `null`** — Keys are scoped per agent deployment. A
1051
+ redeployment with a new slug creates a fresh KV namespace.