@convex-dev/agent 0.1.15-alpha.2 → 0.1.16-alpha.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. package/README.md +41 -700
  2. package/dist/client/index.d.ts +23 -20
  3. package/dist/client/index.d.ts.map +1 -1
  4. package/dist/client/index.js +34 -18
  5. package/dist/client/index.js.map +1 -1
  6. package/dist/client/streaming.d.ts +24 -2
  7. package/dist/client/streaming.d.ts.map +1 -1
  8. package/dist/client/streaming.js +35 -4
  9. package/dist/client/streaming.js.map +1 -1
  10. package/dist/client/types.d.ts +2 -20
  11. package/dist/client/types.d.ts.map +1 -1
  12. package/dist/component/_generated/api.d.ts +7 -150
  13. package/dist/component/messages.d.ts +0 -59
  14. package/dist/component/messages.d.ts.map +1 -1
  15. package/dist/component/messages.js +8 -24
  16. package/dist/component/messages.js.map +1 -1
  17. package/dist/component/schema.d.ts +4 -4
  18. package/dist/component/schema.d.ts.map +1 -1
  19. package/dist/component/schema.js +1 -1
  20. package/dist/component/schema.js.map +1 -1
  21. package/dist/component/streams.d.ts +7 -2
  22. package/dist/component/streams.d.ts.map +1 -1
  23. package/dist/component/streams.js +40 -14
  24. package/dist/component/streams.js.map +1 -1
  25. package/dist/component/vector/tables.d.ts +3 -3
  26. package/dist/component/vector/tables.d.ts.map +1 -1
  27. package/dist/mapping.d.ts +5 -0
  28. package/dist/mapping.d.ts.map +1 -1
  29. package/dist/mapping.js +8 -1
  30. package/dist/mapping.js.map +1 -1
  31. package/dist/react/deltas.d.ts.map +1 -1
  32. package/dist/react/deltas.js +20 -1
  33. package/dist/react/deltas.js.map +1 -1
  34. package/dist/react/index.js +1 -1
  35. package/dist/react/index.js.map +1 -1
  36. package/dist/react/toUIMessages.js +2 -2
  37. package/dist/react/toUIMessages.js.map +1 -1
  38. package/dist/validators.d.ts +217 -265
  39. package/dist/validators.d.ts.map +1 -1
  40. package/dist/validators.js +1 -7
  41. package/dist/validators.js.map +1 -1
  42. package/package.json +1 -1
  43. package/src/client/index.ts +41 -29
  44. package/src/client/streaming.ts +55 -4
  45. package/src/client/types.ts +2 -21
  46. package/src/component/_generated/api.d.ts +7 -150
  47. package/src/component/messages.ts +13 -30
  48. package/src/component/schema.ts +1 -1
  49. package/src/component/streams.ts +49 -16
  50. package/src/component/vector/tables.ts +5 -5
  51. package/src/mapping.ts +8 -1
  52. package/src/react/deltas.ts +24 -3
  53. package/src/react/index.ts +1 -1
  54. package/src/react/toUIMessages.ts +2 -2
  55. package/src/validators.ts +1 -26
package/README.md CHANGED
@@ -5,719 +5,60 @@
5
5
  ```sh
6
6
  npm i @convex-dev/agent
7
7
  ```
8
- <!-- START: Include on https://convex.dev/components -->
9
8
 
10
- AI Agent framework built on Convex.
9
+ <!-- START: Include on https://convex.dev/components -->
11
10
 
12
- - Automatic storage of chat history, per-user or per-thread, that can span multiple agents.
13
- - Playground UI for testing, debugging, and development. See [playground/README.md](playground/README.md) for more.
14
- - RAG for chat context, via hybrid text & vector search, with configuration options.
15
- You can also pair it with the [RAG component](https://convex.dev/components/rag)
16
- to search for other context either before the prompt or via tool calls.
17
- See [examples/convex/rag](./example/convex/rag/README.md).
18
- - Opt-in search for messages from other threads (for the same specified user).
19
- - Support for generating / streaming objects and storing them in messages (as JSON).
20
- - Tool calls via the AI SDK, along with Convex-specific tool wrappers.
21
- - Easy integration with the [Workflow component](https://convex.dev/components/workflow).
22
- Enables long-lived, durable workflows defined as code.
23
- - Reactive & realtime updates from asynchronous functions / workflows.
24
- - Support for streaming text and storing the final result.
25
- See [chat/chat-streaming](./example/convex/chat/README.md).
26
- - Rate limiting for chat messages and token usage via the
27
- [Rate Limiter component](https://convex.dev/components/rate-limiter).
28
- See [examples/rate-limiting](./example/convex/rate_limiting/README.md).
29
- - Optionally filter tool calls out of the thread history.
11
+ AI Agents, built on Convex.
12
+ [Check out the docs here](https://docs.convex.dev/agents).
13
+
14
+ - [Agents](./docs/agent-setup.md) provide an abstraction for using LLMs to
15
+ represent units of use-case-specific prompting with associated models,
16
+ prompts, [Tool Calls](./docs/tools.md), and behavior in relation to other
17
+ Agents, functions, APIs, and more.
18
+ - [Threads](./docs/threads.md) persist [messages](./docs/messages.md) and can be
19
+ shared by multiple users and agents (including
20
+ [human agents](./docs/human-agents.md)).
21
+ - Streaming text and objects using deltas over websockets so all clients stay in
22
+ sync efficiently, without http streaming. Enables streaming from async
23
+ functions.
24
+ - [Conversation context](./docs/context.md) is automatically included in each
25
+ LLM call, including built-in hybrid vector/text search for messages in the
26
+ thread and opt-in search for messages from other threads (for the same
27
+ specified user).
28
+ - [RAG](./docs/rag.md) techniques are supported for prompt augmentation from
29
+ other sources, either up front in the prompt or as tool calls. Integrates with
30
+ the [RAG Component](https://www.convex.dev/components/rag), or DIY.
31
+ - [Workflows](./docs/workflows.md) allow building multi-step operations that can
32
+ span agents, users, durably and reliably.
33
+ - [Files](./docs/files.md) are supported in thread history with automatic saving
34
+ to [file storage](https://docs.convex.dev/file-storage) and ref-counting.
35
+ - [Debugging](./docs/debugging.md) is enabled by callbacks, the
36
+ [agent playground](./docs/playground.md) where you can inspect all metadata
37
+ and iterate on prompts and context settings, and inspection in the dashboard.
38
+ - [Usage tracking](./docs/usage-tracking.md) is easy to set up, enabling usage
39
+ attribution per-provider, per-model, per-user, per-agent, for billing & more.
40
+ - [Rate limiting](./docs/rate-limiting.md), powered by the
41
+ [Rate Limiting Component](https://www.convex.dev/components/rate-limiting),
42
+ helps control the rate at which users can interact with agents and keep you
43
+ from exceeding your LLM provider's limits.
30
44
 
31
45
  [Read the associated Stack post here](https://stack.convex.dev/ai-agents).
32
46
 
33
47
  [![Powerful AI Apps Made Easy with the Agent Component](https://thumbs.video-to-markdown.com/b323ac24.jpg)](https://youtu.be/tUKMPUlOCHY)
48
+ **Read the [docs](https://docs.convex.dev/agents) for more details.**
34
49
 
35
- Play with the [example](./example/) by cloning this repo and running:
50
+ Play with the [example](./example/):
36
51
 
37
52
  ```sh
53
+ git clone https://github.com/get-convex/agent.git
54
+ cd agent
38
55
  npm run setup
39
56
  npm run example
40
57
  ```
41
58
 
42
- ## Example usage:
43
-
44
- See example usage in [example/convex](./example/convex).
45
-
46
- ```ts
47
- // Define an agent similarly to the AI SDK
48
- const supportAgent = new Agent(components.agent, {
49
- chat: openai.chat("gpt-4o-mini"),
50
- textEmbedding: openai.embedding("text-embedding-3-small"),
51
- instructions: "You are a helpful assistant.",
52
- tools: { accountLookup, fileTicket, sendEmail },
53
- });
54
-
55
- // Use the agent from within a normal action:
56
- export const createThreadAndPrompt = action({
57
- args: { prompt: v.string() },
58
- handler: async (ctx, { prompt }) => {
59
- const userId = await getUserId(ctx);
60
- // Start a new thread for the user.
61
- const { threadId, thread } = await supportAgent.createThread(ctx, { userId});
62
- // Creates a user message with the prompt, and an assistant reply message.
63
- const result = await thread.generateText({ prompt });
64
- return { threadId, text: result.text };
65
- },
66
- });
67
-
68
- // Pick up where you left off, with the same or a different agent:
69
- export const continueThread = action({
70
- args: { prompt: v.string(), threadId: v.string() },
71
- handler: async (ctx, { prompt, threadId }) => {
72
- // Continue a thread, picking up where you left off.
73
- const { thread } = await anotherAgent.continueThread(ctx, { threadId });
74
- // This includes previous message history from the thread automatically.
75
- const result = await thread.generateText({ prompt });
76
- return result.text;
77
- },
78
- });
79
- ```
80
-
81
- Also see the [Stack article](https://stack.convex.dev/ai-agents).
82
-
83
- Found a bug? Feature request? [File it here](https://github.com/get-convex/agent/issues).
84
-
85
- ## Pre-requisite: Convex
86
-
87
- You'll need an existing Convex project to use the component.
88
- Convex is a hosted backend platform, including a database, serverless functions,
89
- and a ton more you can learn about [here](https://docs.convex.dev/get-started).
90
-
91
- Run `npm create convex` or follow any of the [quickstarts](https://docs.convex.dev/home) to set one up.
92
-
93
- ## Installation
94
-
95
- Install the component package:
96
-
97
- ```ts
98
- npm install @convex-dev/agent
99
- ```
100
-
101
- Create a `convex.config.ts` file in your app's `convex/` folder and install the component by calling `use`:
102
-
103
- ```ts
104
- // convex/convex.config.ts
105
- import { defineApp } from "convex/server";
106
- import agent from "@convex-dev/agent/convex.config";
107
-
108
- const app = defineApp();
109
- app.use(agent);
110
-
111
- export default app;
112
- ```
113
-
114
- ## Usage
115
-
116
- ### Creating the agent
117
-
118
- ```ts
119
- import { tool } from "ai";
120
- import { openai } from "@ai-sdk/openai";
121
- import { z } from "zod";
122
- import { Agent, createTool } from "@convex-dev/agent";
123
- import { components } from "./_generated/api";
124
-
125
- // Define an agent similarly to the AI SDK
126
- const supportAgent = new Agent(components.agent, {
127
- // The chat completions model to use for the agent.
128
- chat: openai.chat("gpt-4o-mini"),
129
- // The default system prompt if not overriden.
130
- instructions: "You are a helpful assistant.",
131
- tools: {
132
- // Convex tool
133
- myConvexTool: createTool({
134
- description: "My Convex tool",
135
- args: z.object({...}),
136
- // Note: annotate the return type of the handler to avoid type cycles.
137
- handler: async (ctx, args): Promise<string> => {
138
- return "Hello, world!";
139
- },
140
- }),
141
- // Standard AI SDK tool
142
- myTool: tool({ description, parameters, execute: () => {}}),
143
- },
144
- // Embedding model to power vector search of message history (RAG).
145
- textEmbedding: openai.embedding("text-embedding-3-small"),
146
- // Used for fetching context messages. See [below](#configuring-the-context-of-messages)
147
- contextOptions,
148
- // Used for storing messages. See [below](#configuring-the-storage-of-messages)
149
- storageOptions,
150
- // Used for limiting the number of steps when tool calls are involved.
151
- // NOTE: if you want tool calls to happen automatically with a single call,
152
- // you need to set this to something greater than 1 (the default).
153
- maxSteps: 1,
154
- // Used for limiting the number of retries when a tool call fails. Default: 3.
155
- maxRetries: 3,
156
- // Used for tracking token usage. See [below](#tracking-token-usage)
157
- usageHandler: async (ctx, { model, usage }) => {
158
- // ... log, save usage to your database, etc.
159
- },
160
- });
161
- ```
162
-
163
- ### Starting a thread
164
-
165
- You can start a thread from either an action or a mutation.
166
- If it's in an action, you can also start sending messages.
167
- The threadId allows you to resume later and maintain message history.
168
- If you specify a userId, the thread will be associated with that user and messages will be saved to the user's history.
169
- You can also search the user's history for relevant messages in this thread.
170
-
171
- ```ts
172
- // Use the agent from within a normal action:
173
- export const createThread = mutation({
174
- args: {},
175
- handler: async (ctx): Promise<{ threadId: string }> => {
176
- const userId = await getUserId(ctx);
177
- // Start a new thread for the user.
178
- const { threadId } = await supportAgent.createThread(ctx, { userId });
179
- return { threadId };
180
- },
181
- });
182
- ```
183
-
184
- Note: if you're in an environment where you don't have access to the Agent, then
185
- you can create the thread more manually:
186
-
187
- ```ts
188
- const { _id: threadId } = await ctx.runMutation(
189
- components.agent.threads.createThread,
190
- { userId, title, summary }
191
- );
192
- ```
193
-
194
- ### Continuing a thread
195
-
196
- If you specify a userId too, you can search the user's history for relevant messages
197
- to include in the prompt context.
198
-
199
- ```ts
200
- // Pick up where you left off:
201
- export const continueThread = action({
202
- args: { prompt: v.string(), threadId: v.string() },
203
- handler: async (ctx, { prompt, threadId }): Promise<string> => {
204
- await authorizeThreadAccess(ctx, threadId);
205
- // This includes previous message history from the thread automatically.
206
- + const { thread } = await supportAgent.continueThread(ctx, { threadId });
207
- const result = await thread.generateText({ prompt });
208
- return result.text;
209
- },
210
- });
211
- ```
212
-
213
- ### Generating text
214
-
215
- The arguments to `generateText` are the same as the AI SDK, except you don't
216
- have to provide a model. By default it will use the agent's chat model.
217
-
218
- ```ts
219
- const { thread } = await supportAgent.createThread(ctx);
220
- // OR
221
- const { thread } = await supportAgent.continueThread(ctx, { threadId });
222
-
223
- const result = await thread.generateText({ prompt });
224
- ```
225
-
226
- ### Generating an object
227
-
228
- Similar to the AI SDK, you can generate or streaman object.
229
- The same arguments apply, except you don't have to provide a model.
230
- It will use the agent's default chat model.
231
-
232
- ```ts
233
- import { z } from "zod";
234
-
235
- const result = await thread.generateObject({
236
- prompt: "Generate a plan based on the conversation so far",
237
- schema: z.object({...}),
238
- });
239
- ```
240
-
241
- ### Showing messages
242
-
243
- Fetch the full messages directly. These will include things like usage, etc.
244
-
245
- Server-side:
246
-
247
- ```ts
248
- import type { MessageDoc } from "@convex-dev/agent";
249
- import { paginationOptsValidator, type PaginationResult } from "convex/server";
250
-
251
- export const listThreadMessages = query({
252
- args: {
253
- threadId: v.string(),
254
- paginationOpts: paginationOptsValidator,
255
- //... other arguments you want
256
- },
257
- handler: async (
258
- ctx, { threadId, paginationOpts },
259
- ): PaginationResult<MessageDoc> => {
260
- // await authorizeThreadAccess(ctx, threadId);
261
- const paginated = await agent.listMessages(ctx, {
262
- threadId,
263
- paginationOpts,
264
- });
265
- // Here you could filter out / modify the documents
266
- return paginated;
267
- },
268
- });
269
- ```
270
-
271
- Client-side:
272
-
273
- ```tsx
274
- import { api } from "../convex/_generated/api";
275
- import { useThreadMessages, toUIMessages } from "@convex-dev/agent/react";
276
-
277
- function MyComponent({ threadId }: { threadId: string }) {
278
- const messages = useThreadMessages(
279
- api.chatBasic.listThreadMessages,
280
- { threadId },
281
- { initialNumItems: 10 },
282
- );
283
- return (
284
- <div>
285
- {toUIMessages(messages.results ?? []).map((message) => (
286
- <div key={message.key}>{message.content}</div>
287
- ))}
288
- </div>
289
- );
290
- }
291
- ```
292
-
293
- See [ChatBasic.tsx](./example/ui/ChatBasic.tsx) for an example, and
294
- [ChatStreaming.tsx](./example/ui/ChatStreaming.tsx) for a streaming example.
295
-
296
- ### Configuring the context of message generation
297
-
298
- You can customize what history is included per-message via `contextOptions`.
299
- These options can be provided to the Agent constructor, or per-message.
300
-
301
- ```ts
302
- const result = await thread.generateText({ prompt }, {
303
- // Values shown are the defaults.
304
- contextOptions: {
305
- // Whether to exclude tool messages in the context.
306
- excludeToolMessages: true,
307
- // How many recent messages to include. These are added after the search
308
- // messages, and do not count against the search limit.
309
- recentMessages: 100,
310
- // Options for searching messages via text and/or vector search.
311
- searchOptions: {
312
- limit: 10, // The maximum number of messages to fetch.
313
- textSearch: false, // Whether to use text search to find messages.
314
- vectorSearch: false, // Whether to use vector search to find messages.
315
- // Note, this is after the limit is applied.
316
- // E.g. this will quadruple the number of messages fetched.
317
- // (two before, and one after each message found in the search)
318
- messageRange: { before: 2, after: 1 },
319
- },
320
- // Whether to search across other threads for relevant messages.
321
- // By default, only the current thread is searched.
322
- searchOtherThreads: false,
323
- },
324
- ```
325
-
326
- ### Configuring the storage of messages
327
-
328
- Generally the defaults are fine, but if you want to pass in multiple messages
329
- and have them all saved (vs. just the last one), or avoid saving any input
330
- or output messages, you can pass in a `storageOptions` object, either to the
331
- Agent constructor or per-message.
332
-
333
- The usecase for passing in multiple messages but not saving them is if you want
334
- to include some extra messages for context to the LLM, but only the last message
335
- is the user's actual request. e.g. `messages = [...messagesFromRag, messageFromUser]`.
336
- The default is to save the prompt and all output messages.
337
-
338
- ```ts
339
- const result = await thread.generateText({ messages }, {
340
- storageOptions: {
341
- saveMessages: "all" | "none" | "promptAndOutput";
342
- },
343
- });
344
- ```
345
-
346
- ### Creating a tool with Convex context
347
-
348
- There are two ways to create a tool that has access to the Convex context.
349
-
350
- 1. Use the `createTool` function, which is a wrapper around the AI SDK's `tool` function.
351
-
352
- ```ts
353
- export const ideaSearch = createTool({
354
- description: "Search for ideas in the database",
355
- args: z.object({ query: z.string() }),
356
- handler: async (ctx, args): Promise<Array<Idea>> => {
357
- // ctx has userId, threadId, messageId, runQuery, runMutation, and runAction
358
- const ideas = await ctx.runQuery(api.ideas.searchIdeas, { query: args.query });
359
- console.log("found ideas", ideas);
360
- return ideas;
361
- },
362
- });
363
- ```
364
-
365
- 2. Define tools at runtime in a context with the variables you want to use.
366
-
367
- ```ts
368
- async function createTool(ctx: ActionCtx, teamId: Id<"teams">) {
369
- const myTool = tool({
370
- description: "My tool",
371
- parameters: z.object({...}),
372
- execute: async (args, options) => {
373
- return await ctx.runQuery(internal.foo.bar, args);
374
- },
375
- });
376
- }
377
- ```
378
-
379
- You can provide tools at different times:
380
-
381
- - Agent contructor: (`new Agent(components.agent, { tools: {...} })`)
382
- - Creating a thread: `createThread(ctx, { tools: {...} })`
383
- - Continuing a thread: `continueThread(ctx, { tools: {...} })`
384
- - On thread functions: `thread.generateText({ tools: {...} })`
385
- - Outside of a thread: `supportAgent.generateText(ctx, {}, { tools: {...} })`
386
-
387
- Specifying tools at each layer will overwrite the defaults.
388
- The tools will be `args.tools ?? thread.tools ?? agent.options.tools`.
389
- This allows you to create tools in a context that is convenient.
390
-
391
- ### Saving messages then generate asynchronously
392
-
393
- You can save messages in a mutation, then do the generation asynchronously.
394
- This is recommended for a few reasons:
395
- 1. You can set up optimistic UI updates on mutations that are transactional, so
396
- the message will be shown optimistically until the message is saved and
397
- present in your message query.
398
-
399
- To do this, you need to first save the message, then pass the `messageId` as
400
- `promptMessageId` to generate / stream text.
401
-
402
- Note: embeddings are usually generated automatically when you save messages from
403
- an action. However, if you're saving messages in a mutation, where calling
404
- an LLM is not possible, you can generate them asynchronously as well.
405
-
406
- ```ts
407
- export const sendMessage = mutation({
408
- args: { threadId: v.id("threads"), prompt: v.string() },
409
- handler: async (ctx, { threadId, prompt }) => {
410
- const userId = await getUserId(ctx);
411
- const { messageId } = await agent.saveMessage(ctx, {
412
- threadId, userId, prompt,
413
- skipEmbeddings: true,
414
- });
415
- await ctx.scheduler.runAfter(0, internal.example.myAsyncAction, {
416
- threadId, promptMessageId: messageId,
417
- });
418
- }
419
- });
420
-
421
- export const myAsyncAction = internalAction({
422
- args: { threadId: v.string(), promptMessageId: v.string() },
423
- handler: async (ctx, { threadId, promptMessageId }) => {
424
- // Generate embeddings for the prompt message
425
- await supportAgent.generateAndSaveEmbeddings(ctx, { messageIds: [promptMessageId] });
426
- const { thread } = await supportAgent.continueThread(ctx, { threadId });
427
- await thread.generateText({ promptMessageId });
428
- },
429
- });
430
- ```
431
-
432
- ### Search for messages
433
-
434
- This is what the agent does automatically, but it can be useful to do manually, e.g. to find custom context to include.
435
-
436
- Fetch Messages for a user and/or thread.
437
- Accepts ContextOptions, e.g. excludeToolMessages, searchOptions, etc.
438
- If you provide a `beforeMessageId`, it will only fetch messages from before that message.
439
-
440
- ```ts
441
- import type { MessageDoc } from "@convex-dev/agent";
442
-
443
- const messages: MessageDoc[] = await supportAgent.fetchContextMessages(ctx, {
444
- threadId, messages: [{ role, content }], contextOptions
445
- });
446
- ```
447
-
448
- ### Get and update thread information
449
-
450
- List threads for a user:
451
-
452
- ```ts
453
- const threads = await ctx.runQuery(components.agent.threads.listThreadsByUserId, {
454
- userId,
455
- order: "desc",
456
- paginationOpts: { cursor: null, numItems: 10 }
457
- });
458
- ```
459
-
460
- Get a thread by id:
461
-
462
- ```ts
463
- const thread = await ctx.runQuery(components.agent.threads.getThread, {
464
- threadId,
465
- });
466
- ```
467
-
468
- Update a thread's metadata:
469
-
470
- ```ts
471
- await ctx.runMutation(components.agent.threads.updateThread, {
472
- threadId,
473
- { title, summary, status }
474
- });
475
- ```
476
-
477
- ## Using the Playground UI
478
-
479
- The Playground UI is a simple way to test, debug, and develop with the agent.
480
- - First configure it with instructions [here](./playground/README.md).
481
- - Then you can use the [hosted version on GitHub pages](https://get-convex.github.io/agent/)
482
- or run it locally with `npx @convex-dev/agent-playground`.
483
-
484
- ![Playground UI Screenshot](./playground/public/screenshot.png)
485
-
486
- ## Using the Workflow component for long-lived durable workflows
487
-
488
- The [Workflow component](https://convex.dev/components/workflow) is a great way to build long-lived, durable workflows.
489
- It handles retries and guarantees of eventually completing, surviving server restarts, and more.
490
- Read more about durable workflows in [this Stack post](https://stack.convex.dev/durable-workflows-and-strong-guarantees).
491
-
492
- To use the agent alongside workflows, you can run indivdual idempotent steps
493
- that the workflow can run, each with configurable retries, with guarantees that
494
- the workflow will eventually complete. Even if the server crashes mid-workflow,
495
- the workflow will pick up from where it left off and run the next step. If a
496
- step fails and isn't caught by the workflow, the workflow's onComplete handler
497
- will get the error result.
498
-
499
- ### Exposing the agent as Convex actions
500
-
501
- You can expose the agent's capabilities as Convex functions to be used as steps
502
- in a workflow.
503
-
504
- To create a thread as a standalone mutation, similar to `agent.createThread`:
505
-
506
- ```ts
507
- export const createThread = supportAgent.createThreadMutation();
508
- ```
509
-
510
- For an action that generates text in a thread, similar to `thread.generateText`:
511
-
512
- ```ts
513
- export const getSupport = supportAgent.asTextAction({
514
- maxSteps: 10,
515
- });
516
- ```
517
-
518
- You can also expose a standalone action that generates an object.
519
-
520
- ```ts
521
- export const getStructuredSupport = supportAgent.asObjectAction({
522
- schema: z.object({
523
- analysis: z.string().describe("A detailed analysis of the user's request."),
524
- suggestion: z.string().describe("A suggested action to take.")
525
- }),
526
- });
527
- ```
528
-
529
- To save messages explicitly as a mutation, similar to `agent.saveMessages`:
530
-
531
- ```ts
532
- export const saveMessages = supportAgent.asSaveMessagesMutation();
533
- ```
534
-
535
- This is useful for idempotency, as you can first create the user's message,
536
- then generate a response in an unreliable action with retries, passing in the
537
- existing messageId instead of a prompt.
538
-
539
- ### Using the agent actions within a workflow
540
-
541
- You can use the [Workflow component](https://convex.dev/components/workflow)
542
- to run agent flows. It handles retries and guarantees of eventually completing,
543
- surviving server restarts, and more. Read more about durable workflows
544
- [in this Stack post](https://stack.convex.dev/durable-workflows-and-strong-guarantees).
545
-
546
- ```ts
547
- const workflow = new WorkflowManager(components.workflow);
548
-
549
- export const supportAgentWorkflow = workflow.define({
550
- args: { prompt: v.string(), userId: v.string() },
551
- handler: async (step, { prompt, userId }) => {
552
- const { threadId } = await step.runMutation(internal.example.createThread, {
553
- userId, title: "Support Request",
554
- });
555
- const suggestion = await step.runAction(internal.example.getSupport, {
556
- threadId, userId, prompt,
557
- });
558
- const { object } = await step.runAction(internal.example.getStructuredSupport, {
559
- userId, message: suggestion,
560
- });
561
- await step.runMutation(internal.example.sendUserMessage, {
562
- userId, message: object.suggestion,
563
- });
564
- },
565
- });
566
- ```
567
-
568
- See another example in [workflows/chaining.ts](./example/convex/workflows/chaining.ts).
569
-
570
- ## Extra control: how to do more things yourself
571
-
572
- ### Generating text for a user without an associated thread
573
-
574
- ```ts
575
- const result = await supportAgent.generateText(ctx, { userId }, { prompt });
576
- ```
577
-
578
- ### Saving messages manually
579
-
580
- Save messages to the database.
581
-
582
- ```ts
583
- const { lastMessageId, messageIds} = await agent.saveMessages(ctx, {
584
- threadId, userId,
585
- messages: [{ role, content }],
586
- metadata: [{ reasoning, usage, ... }] // See MessageWithMetadata type
587
- });
588
- ```
589
-
590
- ### Manage embeddings
591
-
592
- Generate embeddings for a set of messages.
593
-
594
- ```ts
595
- const embeddings = await supportAgent.generateEmbeddings([
596
- { role: "user", content: "What is love?" },
597
- ]);
598
- ```
599
-
600
- Get and update embeddings, e.g. for a migration to a new model.
601
-
602
- ```ts
603
- const messages = await ctx.runQuery(
604
- components.agent.vector.index.paginate,
605
- { vectorDimension: 1536, cursor: null, limit: 10 }
606
- );
607
- ```
608
-
609
- Note: If the dimension changes, you need to delete the old and insert the new.
610
-
611
- ```ts
612
- const messages = await ctx.runQuery(components.agent.vector.index.updateBatch, {
613
- vectors: [
614
- { model: "gpt-4o-mini", vector: embedding, id: msg.embeddingId },
615
- ],
616
- });
617
- ```
618
-
619
- Delete embeddings
620
-
621
- ```ts
622
- await ctx.runMutation(components.agent.vector.index.deleteBatch, {
623
- ids: [embeddingId1, embeddingId2],
624
- });
625
- ```
626
-
627
- Insert embeddings
628
-
629
- ```ts
630
- const ids = await ctx.runMutation(
631
- components.agent.vector.index.insertBatch, {
632
- vectorDimension: 1536,
633
- vectors: [
634
- {
635
- model: "gpt-4o-mini",
636
- table: "messages",
637
- userId: "123",
638
- threadId: "123",
639
- vector: embedding,
640
- // Optional, if you want to update the message with the embeddingId
641
- messageId: messageId,
642
- },
643
- ],
644
- }
645
- );
646
- ```
647
-
648
-
649
- ### Tracking token usage
650
-
651
- You can provide a `usageHandler` to the agent to track token usage.
652
- See an example in
653
- [this demo](https://github.com/ianmacartney/ai-agent-chat/blob/main/convex/chat.ts)
654
- that captures usage to a table, then scans it to generate per-user invoices.
655
-
656
- You can provide a `usageHandler` to the agent, per-thread, or per-message.
657
-
658
- ```ts
659
- const supportAgent = new Agent(components.agent, {
660
- ...
661
- usageHandler: async (ctx, args) => {
662
- const {
663
- // Who used the tokens
664
- userId, threadId, agentName,
665
- // What LLM was used
666
- model, provider,
667
- // How many tokens were used (extra info is available in providerMetadata)
668
- usage, providerMetadata
669
- } = args;
670
- // ... log, save usage to your database, etc.
671
- },
672
- });
673
- ```
674
-
675
- Tip: Define the `usageHandler` within a function where you have more variables
676
- available to attribute the usage to a different user, team, project, etc.
677
-
678
- ### Logging the raw request and response
679
-
680
- You can provide a `rawRequestResponseHandler` to the agent to log the raw request and response from the LLM.
681
-
682
- You could use this to log the request and response to a table, or use console logs with
683
- [Log Streaming](https://docs.convex.dev/production/integrations/log-streams/)
684
- to allow debugging and searching through Axiom or another logging service.
685
-
686
- ```ts
687
- const supportAgent = new Agent(components.agent, {
688
- ...
689
- rawRequestResponseHandler: async (ctx, { request, response }) => {
690
- console.log("request", request);
691
- console.log("response", response);
692
- },
693
- });
694
- ```
695
-
696
- ## Troubleshooting
697
-
698
- ### Circular dependencies
699
-
700
- Having the return value of workflows depend on other Convex functions can lead to circular dependencies due to the
701
- `internal.foo.bar` way of specifying functions. The way to fix this is to explicitly type the return value of the
702
- workflow. When in doubt, add return types to more `handler` functions, like this:
703
-
704
- ```diff
705
- export const supportAgentWorkflow = workflow.define({
706
- args: { prompt: v.string(), userId: v.string(), threadId: v.string() },
707
- + handler: async (step, { prompt, userId, threadId }): Promise<string> => {
708
- // ...
709
- },
710
- });
711
-
712
- // And regular functions too:
713
- export const myFunction = action({
714
- args: { prompt: v.string() },
715
- + handler: async (ctx, { prompt }): Promise<string> => {
716
- // ...
717
- },
718
- });
719
- ```
720
-
59
+ Found a bug? Feature request?
60
+ [File it here](https://github.com/get-convex/agent/issues).
721
61
 
722
62
  <!-- END: Include on https://convex.dev/components -->
63
+
723
64
  [![DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/get-convex/agent)