@mastra/mcp-docs-server 1.2.5-alpha.5 → 1.2.5-alpha.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.docs/docs/agents/agent-approval.md +2 -0
- package/.docs/docs/long-running-agents/schedules.md +227 -0
- package/.docs/docs/observability/feedback.md +136 -0
- package/.docs/docs/observability/overview.md +8 -4
- package/.docs/docs/workflows/scheduled-workflows.md +5 -4
- package/.docs/guides/build-your-ui/openui.md +12 -20
- package/.docs/models/environment-variables.md +1 -0
- package/.docs/models/gateways/vercel.md +1 -1
- package/.docs/models/index.md +1 -1
- package/.docs/models/providers/anthropic.md +3 -9
- package/.docs/models/providers/deepinfra.md +17 -4
- package/.docs/models/providers/digitalocean.md +3 -1
- package/.docs/models/providers/longcat.md +73 -0
- package/.docs/models/providers/mistral.md +1 -1
- package/.docs/models/providers/stackit.md +6 -6
- package/.docs/models/providers.md +1 -0
- package/.docs/reference/client-js/observability.md +88 -1
- package/.docs/reference/index.md +1 -0
- package/.docs/reference/observability/feedback.md +391 -0
- package/.docs/reference/workflows/workflow.md +2 -0
- package/CHANGELOG.md +14 -0
- package/package.json +5 -5
- package/.docs/docs/long-running-agents/heartbeats.md +0 -213
|
@@ -49,6 +49,8 @@ for await (const chunk of stream.fullStream) {
|
|
|
49
49
|
```
|
|
50
50
|
|
|
51
51
|
> **Note:** Agent approval uses snapshots to capture request state. Configure a [storage provider](https://mastra.ai/docs/memory/storage) on your Mastra instance or you'll see a "snapshot not found" error.
|
|
52
|
+
>
|
|
53
|
+
> Snapshots for agent runs are minimal resume artifacts: they hold only what's needed to resume the suspended run and are deleted once the run finishes. Use [tracing](https://mastra.ai/docs/observability/overview) for the execution record and [memory](https://mastra.ai/docs/memory/overview) for the conversation history.
|
|
52
54
|
|
|
53
55
|
## How approval works
|
|
54
56
|
|
|
@@ -0,0 +1,227 @@
|
|
|
1
|
+
> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
|
|
2
|
+
|
|
3
|
+
# Schedules
|
|
4
|
+
|
|
5
|
+
**Added in:** `@mastra/core@1.50.0`
|
|
6
|
+
|
|
7
|
+
> **Beta:** This feature is in beta. Breaking changes may occur without a major version bump until the API is stable.
|
|
8
|
+
|
|
9
|
+
A schedule runs an agent on a cron cadence. On each fire, Mastra sends a prompt to the agent, either as a [signal](https://mastra.ai/docs/long-running-agents/signals) into a thread or as a threadless `agent.generate()` run. Use schedules for recurring agent work such as daily summaries, periodic checks, or scheduled nudges into a conversation.
|
|
10
|
+
|
|
11
|
+
Schedules are persisted, so they survive restarts and redeploys. Manage them at runtime through `mastra.schedules`, the canonical create, read, update, and delete (CRUD) surface. The same surface also manages [workflow schedules](https://mastra.ai/docs/workflows/scheduled-workflows) — pass `workflowId` instead of `agentId` to schedule a workflow.
|
|
12
|
+
|
|
13
|
+
## Prerequisites
|
|
14
|
+
|
|
15
|
+
Schedules require a [storage](https://mastra.ai/docs/memory/storage) adapter that implements the schedules domain, for example `@mastra/libsql`. Without one, `mastra.schedules.create()` throws.
|
|
16
|
+
|
|
17
|
+
## Quickstart
|
|
18
|
+
|
|
19
|
+
The following schedule runs the `pinger` agent every hour. It has no thread, so each fire is an isolated `agent.generate()` run.
|
|
20
|
+
|
|
21
|
+
```typescript
|
|
22
|
+
import { Mastra } from '@mastra/core'
|
|
23
|
+
import { Agent } from '@mastra/core/agent'
|
|
24
|
+
import { LibSQLStore } from '@mastra/libsql'
|
|
25
|
+
|
|
26
|
+
const pinger = new Agent({
|
|
27
|
+
id: 'pinger',
|
|
28
|
+
name: 'Pinger',
|
|
29
|
+
instructions: 'Report the current system status in one sentence.',
|
|
30
|
+
model: 'openai/gpt-5.5',
|
|
31
|
+
})
|
|
32
|
+
|
|
33
|
+
const mastra = new Mastra({
|
|
34
|
+
agents: { pinger },
|
|
35
|
+
storage: new LibSQLStore({ url: 'file:./mastra.db' }),
|
|
36
|
+
})
|
|
37
|
+
|
|
38
|
+
await mastra.schedules.create({
|
|
39
|
+
agentId: 'pinger',
|
|
40
|
+
cron: '0 * * * *',
|
|
41
|
+
prompt: 'Give me a status update.',
|
|
42
|
+
})
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
Mastra starts the scheduler the first time a schedule is created, then fires the agent on the cron you specify.
|
|
46
|
+
|
|
47
|
+
## Cadence
|
|
48
|
+
|
|
49
|
+
Schedules fire on a cron expression. The `cron` field accepts a standard 5-, 6-, or 7-part cron expression, and it's validated when you create or update the schedule.
|
|
50
|
+
|
|
51
|
+
Croner nicknames also work, for example `@hourly`, `@daily`, `@weekly`, `@monthly`, and `@midnight`. For day-and-time combinations, write the cron field directly:
|
|
52
|
+
|
|
53
|
+
```typescript
|
|
54
|
+
// Every weekday at 9am
|
|
55
|
+
await mastra.schedules.create({
|
|
56
|
+
agentId: 'pinger',
|
|
57
|
+
cron: '0 9 * * 1-5',
|
|
58
|
+
prompt: 'Start-of-day check.',
|
|
59
|
+
})
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Set `timezone` to an IANA timezone, for example `America/New_York`, so fire times don't depend on the host's locale. When omitted, the cron resolves against the host's local timezone.
|
|
63
|
+
|
|
64
|
+
For more readable cron construction, you can use a userland builder such as [`cron-time-generator`](https://www.npmjs.com/package/cron-time-generator) and pass its output to `cron`.
|
|
65
|
+
|
|
66
|
+
## Threadless and threaded schedules
|
|
67
|
+
|
|
68
|
+
An agent schedule fires in one of two modes, decided by whether you pass a `threadId`.
|
|
69
|
+
|
|
70
|
+
### Threadless
|
|
71
|
+
|
|
72
|
+
Without a `threadId`, each fire is an isolated `agent.generate()` run. Nothing is written to a conversation thread. This is the simplest mode and suits status checks, reports, and other work that doesn't need conversation context.
|
|
73
|
+
|
|
74
|
+
### Threaded
|
|
75
|
+
|
|
76
|
+
With a `threadId`, the schedule sends a [signal](https://mastra.ai/docs/long-running-agents/signals) into that thread, so the prompt joins the agent's conversation. Threaded schedules require a `resourceId` alongside the `threadId`.
|
|
77
|
+
|
|
78
|
+
```typescript
|
|
79
|
+
await mastra.schedules.create({
|
|
80
|
+
agentId: 'pinger',
|
|
81
|
+
cron: '0 9 * * *',
|
|
82
|
+
prompt: 'Summarize anything new since yesterday.',
|
|
83
|
+
threadId: 'thread-123',
|
|
84
|
+
resourceId: 'user-456',
|
|
85
|
+
})
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Threaded schedules accept extra fields that control how the signal behaves. They mirror the options [`agent.sendSignal`](https://mastra.ai/docs/long-running-agents/signals) accepts and stay JSON-serializable so they persist with the schedule.
|
|
89
|
+
|
|
90
|
+
- `signalType`: the [signal type](https://mastra.ai/docs/long-running-agents/signals) to send, for example `notification` or `system-reminder`. Defaults to `notification`.
|
|
91
|
+
- `tagName`: the XML tag the signal renders as. Defaults to `schedule`, so a fire surfaces to the agent as `<schedule>…</schedule>`.
|
|
92
|
+
- `attributes`: values rendered onto the signal's XML tag.
|
|
93
|
+
- `ifActive`: behavior when the thread is already streaming, as `{ behavior, attributes }`. `behavior` is one of `deliver`, `persist`, or `discard`.
|
|
94
|
+
- `ifIdle`: behavior when the thread is idle, as `{ behavior, attributes, streamOptions }`. `behavior` is one of `wake`, `persist`, or `discard`. `streamOptions.requestContext` is applied to the woken run.
|
|
95
|
+
|
|
96
|
+
These fields require a `threadId`. Passing them on a threadless schedule throws.
|
|
97
|
+
|
|
98
|
+
```typescript
|
|
99
|
+
await mastra.schedules.create({
|
|
100
|
+
agentId: 'pinger',
|
|
101
|
+
cron: '0 9 * * *',
|
|
102
|
+
prompt: 'Summarize anything new since yesterday.',
|
|
103
|
+
threadId: 'thread-123',
|
|
104
|
+
resourceId: 'user-456',
|
|
105
|
+
tagName: 'check-in', // renders as <check-in>…</check-in>
|
|
106
|
+
attributes: { source: 'cron' },
|
|
107
|
+
ifActive: { behavior: 'discard' }, // skip if the thread is mid-stream
|
|
108
|
+
ifIdle: {
|
|
109
|
+
behavior: 'wake', // wake the agent if the thread is idle
|
|
110
|
+
streamOptions: { requestContext: { locale: 'en-US' } },
|
|
111
|
+
},
|
|
112
|
+
})
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
`providerOptions` are merged into the signal payload on every fire and apply to both threaded and threadless schedules.
|
|
116
|
+
|
|
117
|
+
## Managing schedules
|
|
118
|
+
|
|
119
|
+
Use `mastra.schedules` for all schedule operations. To scope to a single agent, pass `agentId` to `create` or `list`.
|
|
120
|
+
|
|
121
|
+
```typescript
|
|
122
|
+
// Create
|
|
123
|
+
const schedule = await mastra.schedules.create({
|
|
124
|
+
agentId: 'pinger',
|
|
125
|
+
cron: '0 * * * *',
|
|
126
|
+
prompt: 'Status check.',
|
|
127
|
+
})
|
|
128
|
+
|
|
129
|
+
// Read
|
|
130
|
+
await mastra.schedules.get(schedule.id)
|
|
131
|
+
await mastra.schedules.list({ agentId: 'pinger' })
|
|
132
|
+
|
|
133
|
+
// Update — changing cron or timezone recomputes the next fire time
|
|
134
|
+
await mastra.schedules.update(schedule.id, { cron: '*/30 * * * *' })
|
|
135
|
+
|
|
136
|
+
// Pause and resume
|
|
137
|
+
await mastra.schedules.pause(schedule.id)
|
|
138
|
+
await mastra.schedules.resume(schedule.id)
|
|
139
|
+
|
|
140
|
+
// Fire once now, off-schedule
|
|
141
|
+
await mastra.schedules.run(schedule.id)
|
|
142
|
+
|
|
143
|
+
// Delete
|
|
144
|
+
await mastra.schedules.delete(schedule.id)
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
A few rules worth knowing:
|
|
148
|
+
|
|
149
|
+
- `pause` and `resume` are durable and idempotent. A paused schedule survives restarts, and `resume` recomputes the next fire time from now rather than firing backlogged runs.
|
|
150
|
+
- `run` fires the schedule once immediately without affecting its cadence.
|
|
151
|
+
- `list` filters by `agentId`, `workflowId`, `threadId`, `resourceId`, `name`, and `status`. Without a filter it returns every schedule, agent and workflow alike.
|
|
152
|
+
|
|
153
|
+
### Workflow schedules
|
|
154
|
+
|
|
155
|
+
The same service creates schedules that run a workflow instead of an agent. Pass `workflowId` and the workflow-shaped fields:
|
|
156
|
+
|
|
157
|
+
```typescript
|
|
158
|
+
await mastra.schedules.create({
|
|
159
|
+
workflowId: 'daily-report',
|
|
160
|
+
cron: '0 9 * * *',
|
|
161
|
+
inputData: { userId: 'system' },
|
|
162
|
+
})
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
Workflow schedules created this way are independent of the declarative `schedule` field on `createWorkflow` — see [scheduled workflows](https://mastra.ai/docs/workflows/scheduled-workflows) for the declarative form and Studio views.
|
|
166
|
+
|
|
167
|
+
### Custom IDs
|
|
168
|
+
|
|
169
|
+
By default `create` generates a random `agent_<uuid>` id. Pass `id` to choose a stable one, for example when you want a predictable handle to look up, update, or delete later:
|
|
170
|
+
|
|
171
|
+
```typescript
|
|
172
|
+
await mastra.schedules.create({
|
|
173
|
+
id: 'nightly-summary',
|
|
174
|
+
agentId: 'pinger',
|
|
175
|
+
cron: '0 9 * * *',
|
|
176
|
+
prompt: 'Summarize anything new since yesterday.',
|
|
177
|
+
})
|
|
178
|
+
// stored as `agent_nightly-summary`
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
The id is normalized to `agent_<slug>`: the `agent_` prefix is added if missing and the rest is slugified. Workflow schedules normalize to `schedule_<slug>` instead. Creating a schedule with an id that already exists throws, so use `update` to change an existing one.
|
|
182
|
+
|
|
183
|
+
### From the client
|
|
184
|
+
|
|
185
|
+
The same operations are available from `@mastra/client-js` over the `/api/schedules` routes (`client.createSchedule`, `client.listSchedules`, `client.runSchedule`, and so on), so you can manage schedules from a separate process or a UI.
|
|
186
|
+
|
|
187
|
+
## Lifecycle hooks
|
|
188
|
+
|
|
189
|
+
Hooks let you run code at key points in an agent schedule's lifecycle, for example to compute fire-time parameters or react to the outcome. Configure them on the `Mastra` constructor under `schedules`. The hooks are a single flat bundle that runs for every agent's schedules; each hook context carries the firing `agentId`, so branch on it when you need per-agent behavior. Hooks live at the `Mastra` level so they apply to both code-defined and stored agents.
|
|
190
|
+
|
|
191
|
+
```typescript
|
|
192
|
+
const mastra = new Mastra({
|
|
193
|
+
agents: { pinger },
|
|
194
|
+
storage: new LibSQLStore({ url: 'file:./mastra.db' }),
|
|
195
|
+
schedules: {
|
|
196
|
+
prepare: async ({ agentId, schedule, trigger }) => {
|
|
197
|
+
// Return overrides, null to skip this fire, or undefined for defaults
|
|
198
|
+
return { prompt: `Status as of ${trigger.firedAt.toISOString()}` }
|
|
199
|
+
},
|
|
200
|
+
onFinish: async ({ agentId, outcome, runId }) => {
|
|
201
|
+
// Runs on any non-error, non-abort outcome
|
|
202
|
+
},
|
|
203
|
+
onError: async ({ agentId, phase, error }) => {
|
|
204
|
+
// Runs when prepare, the signal, or the agent run threw
|
|
205
|
+
},
|
|
206
|
+
onAbort: async ({ agentId, runId }) => {
|
|
207
|
+
// Runs when the run was aborted mid-stream
|
|
208
|
+
},
|
|
209
|
+
},
|
|
210
|
+
})
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
The hooks are:
|
|
214
|
+
|
|
215
|
+
- `prepare`: runs before the fire. Return an object to override fire-time parameters such as `prompt` or `threadId`, `null` to skip the fire, or `undefined` to use the stored defaults.
|
|
216
|
+
- `onFinish`: runs once per trigger that reached a non-error, non-abort terminal state.
|
|
217
|
+
- `onError`: runs when `prepare`, the signal, or the agent run threw.
|
|
218
|
+
- `onAbort`: runs when the run was aborted mid-stream.
|
|
219
|
+
|
|
220
|
+
Every hook context includes `agentId` (the agent the schedule fired for) alongside `schedule` and `trigger`.
|
|
221
|
+
|
|
222
|
+
Hook exceptions are caught and logged. They never re-route the worker or trigger another hook.
|
|
223
|
+
|
|
224
|
+
## Related
|
|
225
|
+
|
|
226
|
+
- [Signals](https://mastra.ai/docs/long-running-agents/signals): the delivery mechanism behind threaded schedules.
|
|
227
|
+
- [Scheduled workflows](https://mastra.ai/docs/workflows/scheduled-workflows): declare a cron schedule on a workflow definition.
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
|
|
2
|
+
|
|
3
|
+
# Feedback
|
|
4
|
+
|
|
5
|
+
Feedback records capture human-in-the-loop signals such as thumbs, ratings, comments, and corrections. Use feedback when you need to attach user, QA, Studio, or system review data to a trace or span and query that data alongside other observability signals.
|
|
6
|
+
|
|
7
|
+
Unlike metrics and scores, feedback is usually supplied by a person or review workflow. Numeric feedback values can be aggregated, grouped, charted over time, and queried for percentiles.
|
|
8
|
+
|
|
9
|
+
## When to use feedback
|
|
10
|
+
|
|
11
|
+
- Collect user satisfaction ratings for agent responses.
|
|
12
|
+
- Store QA comments or corrections next to the trace they review.
|
|
13
|
+
- Build dashboards for ratings by agent, environment, or experiment.
|
|
14
|
+
|
|
15
|
+
## Add feedback
|
|
16
|
+
|
|
17
|
+
Use `mastra.observability.addFeedback()` when you want to annotate a persisted trace or span from app code. The helper is optional on the observability entrypoint, so check that the active observability implementation supports it. See the [client SDK observability reference](https://mastra.ai/reference/client-js/observability) for all client methods.
|
|
18
|
+
|
|
19
|
+
```typescript
|
|
20
|
+
if (!mastra.observability.addFeedback) {
|
|
21
|
+
throw new Error('Feedback is not supported by the active observability implementation')
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
await mastra.observability.addFeedback({
|
|
25
|
+
traceId: 'trace-123',
|
|
26
|
+
spanId: 'span-456',
|
|
27
|
+
feedback: {
|
|
28
|
+
feedbackSource: 'user',
|
|
29
|
+
feedbackType: 'rating',
|
|
30
|
+
value: 1,
|
|
31
|
+
comment: 'The answer solved my issue.',
|
|
32
|
+
},
|
|
33
|
+
})
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
## Create feedback
|
|
37
|
+
|
|
38
|
+
Every `createFeedback()` requires `feedbackType` and `value`. Add `traceId` or `spanId` when the feedback should be anchored to a trace or a specific span. Use `feedbackSource` as optional string metadata, such as `user`, `qa`, `studio`, or `system`.
|
|
39
|
+
|
|
40
|
+
For storage-level writes, include `timestamp` because the method writes directly to the store.
|
|
41
|
+
|
|
42
|
+
```typescript
|
|
43
|
+
const observability = await mastra.getStorage()!.getStore('observability')
|
|
44
|
+
|
|
45
|
+
await observability!.createFeedback({
|
|
46
|
+
feedback: {
|
|
47
|
+
feedbackId: 'feedback-rating-1',
|
|
48
|
+
timestamp: new Date(),
|
|
49
|
+
traceId: 'trace-123',
|
|
50
|
+
spanId: 'span-456',
|
|
51
|
+
feedbackSource: 'user',
|
|
52
|
+
feedbackType: 'rating',
|
|
53
|
+
value: 1,
|
|
54
|
+
comment: 'The answer solved my issue.',
|
|
55
|
+
tags: ['production'],
|
|
56
|
+
},
|
|
57
|
+
})
|
|
58
|
+
|
|
59
|
+
await observability!.createFeedback({
|
|
60
|
+
feedback: {
|
|
61
|
+
feedbackId: 'feedback-comment-1',
|
|
62
|
+
timestamp: new Date(),
|
|
63
|
+
feedbackSource: 'qa',
|
|
64
|
+
feedbackType: 'comment',
|
|
65
|
+
value: 'Needs a citation before shipping.',
|
|
66
|
+
experimentId: 'support-agent-eval',
|
|
67
|
+
},
|
|
68
|
+
})
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
## List feedback
|
|
72
|
+
|
|
73
|
+
Use `listFeedback()` to page through raw records or poll with delta mode.
|
|
74
|
+
|
|
75
|
+
```typescript
|
|
76
|
+
const result = await observability!.listFeedback({
|
|
77
|
+
filters: {
|
|
78
|
+
feedbackType: 'rating',
|
|
79
|
+
feedbackSource: 'user',
|
|
80
|
+
timestamp: { start: new Date(Date.now() - 7 * 24 * 60 * 60 * 1000) },
|
|
81
|
+
},
|
|
82
|
+
pagination: { page: 0, perPage: 20 },
|
|
83
|
+
orderBy: { field: 'timestamp', direction: 'DESC' },
|
|
84
|
+
})
|
|
85
|
+
|
|
86
|
+
console.log(result.feedback, result.pagination?.hasMore)
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
Filters include target fields such as `traceId` and `spanId`, feedback fields such as `feedbackType`, `feedbackSource`, and `feedbackUserId`, and shared context fields such as `entityName`, `environment`, `experimentId`, and `tags`.
|
|
90
|
+
|
|
91
|
+
```typescript
|
|
92
|
+
await observability!.listFeedback({
|
|
93
|
+
filters: {
|
|
94
|
+
traceId: 'trace-123',
|
|
95
|
+
feedbackType: ['rating', 'thumbs'],
|
|
96
|
+
tags: ['production'],
|
|
97
|
+
},
|
|
98
|
+
})
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
## Query feedback analytics
|
|
102
|
+
|
|
103
|
+
OLAP feedback queries operate on numeric `value` fields. Use them for ratings, thumbs encoded as `1` and `-1`, numeric QA scores, or other numeric feedback types.
|
|
104
|
+
|
|
105
|
+
```typescript
|
|
106
|
+
const rating = await observability!.getFeedbackAggregate({
|
|
107
|
+
feedbackType: 'rating',
|
|
108
|
+
feedbackSource: 'user',
|
|
109
|
+
aggregation: 'avg',
|
|
110
|
+
comparePeriod: 'previous_day',
|
|
111
|
+
})
|
|
112
|
+
|
|
113
|
+
const byAgent = await observability!.getFeedbackBreakdown({
|
|
114
|
+
feedbackType: 'rating',
|
|
115
|
+
groupBy: ['entityName'],
|
|
116
|
+
aggregation: 'avg',
|
|
117
|
+
filters: { environment: 'production' },
|
|
118
|
+
})
|
|
119
|
+
|
|
120
|
+
const ratingsOverTime = await observability!.getFeedbackTimeSeries({
|
|
121
|
+
feedbackType: 'rating',
|
|
122
|
+
aggregation: 'avg',
|
|
123
|
+
interval: '1h',
|
|
124
|
+
groupBy: ['feedbackSource'],
|
|
125
|
+
})
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
See the [feedback reference](https://mastra.ai/reference/observability/feedback) for all fields, filters, return types, and percentile query parameters.
|
|
129
|
+
|
|
130
|
+
## Related
|
|
131
|
+
|
|
132
|
+
- [Observability overview](https://mastra.ai/docs/observability/overview)
|
|
133
|
+
- [Tracing overview](https://mastra.ai/docs/observability/tracing/overview)
|
|
134
|
+
- [Metrics overview](https://mastra.ai/docs/observability/metrics/overview)
|
|
135
|
+
- [Client SDK observability reference](https://mastra.ai/reference/client-js/observability)
|
|
136
|
+
- [Feedback reference](https://mastra.ai/reference/observability/feedback)
|
|
@@ -2,13 +2,14 @@
|
|
|
2
2
|
|
|
3
3
|
# Observability overview
|
|
4
4
|
|
|
5
|
-
Mastra's observability system gives you visibility into every agent run, workflow step, tool call, and model interaction. Agent behavior depends on model responses, prompts, tools, memory, and workflow state, so observability helps you inspect runtime decisions from day one. It captures
|
|
5
|
+
Mastra's observability system gives you visibility into every agent run, workflow step, tool call, and model interaction. Agent behavior depends on model responses, prompts, tools, memory, and workflow state, so observability helps you inspect runtime decisions from day one. It captures complementary signals that work together to help you understand what your application is doing and why.
|
|
6
6
|
|
|
7
|
-
- [**Configuration**](https://mastra.ai/docs/observability/config): Configure observability once for traces, logs, and
|
|
8
|
-
- [**Storage**](https://mastra.ai/docs/observability/storage): Choose storage backends for persisted traces, logs, and
|
|
7
|
+
- [**Configuration**](https://mastra.ai/docs/observability/config): Configure observability once for traces, logs, metrics, and feedback.
|
|
8
|
+
- [**Storage**](https://mastra.ai/docs/observability/storage): Choose storage backends for persisted traces, logs, metrics aggregation, and feedback queries.
|
|
9
9
|
- [**Tracing**](https://mastra.ai/docs/observability/tracing/overview): Records every operation as a hierarchical timeline of spans, capturing inputs, outputs, token usage, and timing.
|
|
10
10
|
- [**Logging**](https://mastra.ai/docs/observability/logging): Forwards structured log entries from your application and Mastra internals to observability storage, correlated to traces automatically.
|
|
11
11
|
- [**Metrics**](https://mastra.ai/docs/observability/metrics/overview): Extracts duration, token usage, and cost data from traces automatically, with no additional instrumentation required.
|
|
12
|
+
- [**Feedback**](https://mastra.ai/docs/observability/feedback): Stores ratings, comments, corrections, and other review signals linked to traces and spans.
|
|
12
13
|
- [**Integrations**](https://mastra.ai/docs/observability/integrations/overview): Choose exporters, bridges, and span processors for Studio, hosted, or external observability workflows.
|
|
13
14
|
|
|
14
15
|
## When to use observability
|
|
@@ -27,7 +28,9 @@ Metrics are derived from traces automatically. When a span ends, Mastra extracts
|
|
|
27
28
|
|
|
28
29
|
Logs are correlated to traces automatically. Every `logger.info()`, `logger.warn()`, or `logger.error()` call within a traced context is tagged with the current trace and span IDs. You can navigate from a log entry directly to the trace that produced it.
|
|
29
30
|
|
|
30
|
-
|
|
31
|
+
Feedback records human review signals such as ratings, comments, and corrections. Feedback can be linked to traces and spans, then queried with the same observability store used for metrics.
|
|
32
|
+
|
|
33
|
+
These signals share correlation IDs (trace ID, span ID, entity type, entity name), so you can jump between a metric spike, the traces behind it, logs within those traces, and related feedback.
|
|
31
34
|
|
|
32
35
|
## Quickstart
|
|
33
36
|
|
|
@@ -114,6 +117,7 @@ Not all storage backends support every signal. Traces work with most backends, b
|
|
|
114
117
|
- [Tracing](https://mastra.ai/docs/observability/tracing/overview)
|
|
115
118
|
- [Logging](https://mastra.ai/docs/observability/logging)
|
|
116
119
|
- [Metrics](https://mastra.ai/docs/observability/metrics/overview)
|
|
120
|
+
- [Feedback](https://mastra.ai/docs/observability/feedback)
|
|
117
121
|
- [Configuration](https://mastra.ai/docs/observability/config)
|
|
118
122
|
- [Storage](https://mastra.ai/docs/observability/storage)
|
|
119
123
|
- [Integrations overview](https://mastra.ai/docs/observability/integrations/overview)
|
|
@@ -74,8 +74,8 @@ Fields:
|
|
|
74
74
|
Pass an array to fire the same workflow on multiple cadences. Each entry needs a unique stable `id`:
|
|
75
75
|
|
|
76
76
|
```typescript
|
|
77
|
-
const
|
|
78
|
-
id: '
|
|
77
|
+
const statusCheck = createWorkflow({
|
|
78
|
+
id: 'status-check',
|
|
79
79
|
schedule: [
|
|
80
80
|
{ id: 'morning', cron: '0 9 * * *', inputData: { window: 'morning' } },
|
|
81
81
|
{ id: 'evening', cron: '0 18 * * *', inputData: { window: 'evening' } },
|
|
@@ -137,7 +137,7 @@ A few rules worth knowing:
|
|
|
137
137
|
- Resume recomputes `nextFireAt` from now. A schedule paused for a week doesn't fire seven backlogged runs the moment you resume it. It fires on the next regular cron tick.
|
|
138
138
|
- The only way to unpause is `resumeSchedule` (or the **Resume** button in Studio). Editing the workflow's `schedule` config doesn't unpause a paused row.
|
|
139
139
|
- Pause and resume are idempotent. Calling pause on an already-paused schedule is a no-op.
|
|
140
|
-
- This is an operational override, not a way to author schedules.
|
|
140
|
+
- This is an operational override, not a way to author schedules. Declarative schedules are created, deleted, and edited in code via the `schedule` field on `createWorkflow`. To create schedules imperatively at runtime instead, use the unified [`mastra.schedules`](https://mastra.ai/docs/long-running-agents/schedules) service with a `workflowId`.
|
|
141
141
|
|
|
142
142
|
The underlying HTTP routes are `POST /api/schedules/:scheduleId/pause` and `POST /api/schedules/:scheduleId/resume`. Both require the `schedules:write` permission.
|
|
143
143
|
|
|
@@ -180,4 +180,5 @@ Manage Inngest schedules from the [Inngest dashboard](https://www.inngest.com/do
|
|
|
180
180
|
## Related
|
|
181
181
|
|
|
182
182
|
- [Workflow overview](https://mastra.ai/docs/workflows/overview)
|
|
183
|
-
- [Suspend and resume](https://mastra.ai/docs/workflows/suspend-and-resume)
|
|
183
|
+
- [Suspend and resume](https://mastra.ai/docs/workflows/suspend-and-resume)
|
|
184
|
+
- [Agent schedules](https://mastra.ai/docs/long-running-agents/schedules): run an agent, rather than a workflow, on a cron schedule, and manage both kinds at runtime through `mastra.schedules`.
|
|
@@ -10,7 +10,7 @@ OpenUI connects to Mastra through the [AG-UI protocol](https://docs.ag-ui.com).
|
|
|
10
10
|
|
|
11
11
|
## Integration guide
|
|
12
12
|
|
|
13
|
-
Embed Mastra in your Next.js API route and connect an OpenUI `<
|
|
13
|
+
Embed Mastra in your Next.js API route and connect an OpenUI `<AgentInterface />` chat surface to it through the AG-UI protocol.
|
|
14
14
|
|
|
15
15
|
1. Scaffold a new OpenUI app:
|
|
16
16
|
|
|
@@ -184,33 +184,25 @@ Embed Mastra in your Next.js API route and connect an OpenUI `<FullScreen />` ch
|
|
|
184
184
|
}
|
|
185
185
|
```
|
|
186
186
|
|
|
187
|
-
4. Wire the OpenUI `<
|
|
187
|
+
4. Wire the OpenUI `<AgentInterface />` chat surface to the route. Build an `llm` adapter with `fetchLLM()` and set `streamAdapter` to `agUIAdapter()` so OpenUI knows to parse AG-UI events.
|
|
188
188
|
|
|
189
189
|
```tsx
|
|
190
190
|
'use client'
|
|
191
191
|
|
|
192
192
|
import '@openuidev/react-ui/components.css'
|
|
193
193
|
|
|
194
|
-
import { agUIAdapter } from '@openuidev/react-
|
|
195
|
-
import { FullScreen } from '@openuidev/react-ui'
|
|
194
|
+
import { AgentInterface, agUIAdapter, fetchLLM } from '@openuidev/react-ui'
|
|
196
195
|
import { openuiChatLibrary } from '@openuidev/react-ui/genui-lib'
|
|
197
196
|
|
|
197
|
+
const llm = fetchLLM({
|
|
198
|
+
url: '/api/chat',
|
|
199
|
+
streamAdapter: agUIAdapter(),
|
|
200
|
+
})
|
|
201
|
+
|
|
198
202
|
export default function Page() {
|
|
199
203
|
return (
|
|
200
204
|
<div className="relative h-screen w-screen overflow-hidden">
|
|
201
|
-
<
|
|
202
|
-
processMessage={async ({ messages, threadId, abortController }) => {
|
|
203
|
-
return fetch('/api/chat', {
|
|
204
|
-
method: 'POST',
|
|
205
|
-
headers: { 'Content-Type': 'application/json' },
|
|
206
|
-
body: JSON.stringify({ messages, threadId }),
|
|
207
|
-
signal: abortController.signal,
|
|
208
|
-
})
|
|
209
|
-
}}
|
|
210
|
-
streamProtocol={agUIAdapter()}
|
|
211
|
-
componentLibrary={openuiChatLibrary}
|
|
212
|
-
agentName="OpenUI + Mastra Chat"
|
|
213
|
-
/>
|
|
205
|
+
<AgentInterface llm={llm} componentLibrary={openuiChatLibrary} />
|
|
214
206
|
</div>
|
|
215
207
|
)
|
|
216
208
|
}
|
|
@@ -251,7 +243,7 @@ Embed Mastra in your Next.js API route and connect an OpenUI `<FullScreen />` ch
|
|
|
251
243
|
OpenUI consumes the [AG-UI protocol](https://docs.ag-ui.com), a transport-agnostic stream of typed events for AI agents. `@ag-ui/mastra` translates a Mastra `Agent` into this protocol:
|
|
252
244
|
|
|
253
245
|
- The server calls `agent.run({ messages, threadId, runId, ... })` and serializes each emitted event as an SSE message.
|
|
254
|
-
- The client passes `
|
|
246
|
+
- The client passes `fetchLLM({ streamAdapter: agUIAdapter() })` to `<AgentInterface />`, which parses the SSE stream into the internal events that drive OpenUI Lang rendering.
|
|
255
247
|
|
|
256
248
|
`threadId` ties a conversation together across requests, and `runId` identifies a single execution. Generate a fresh `runId` per request and persist `threadId` on the client.
|
|
257
249
|
|
|
@@ -266,11 +258,11 @@ OpenUI generates UI from a component library. The library defines which componen
|
|
|
266
258
|
- `openuiChatLibrary`: components for chat interfaces (cards, forms, tables, charts).
|
|
267
259
|
- `openuiDashboardLibrary`: components for dashboards and data-heavy surfaces.
|
|
268
260
|
|
|
269
|
-
Pass the library to `<
|
|
261
|
+
Pass the library to `<AgentInterface componentLibrary={...} />` to make its components available to the model.
|
|
270
262
|
|
|
271
263
|
### Customizing the library
|
|
272
264
|
|
|
273
|
-
To restrict or extend the output, define your own library in `src/library.ts` and export a subset of components. Pass that library to `<
|
|
265
|
+
To restrict or extend the output, define your own library in `src/library.ts` and export a subset of components. Pass that library to `<AgentInterface />` and regenerate the system prompt whenever the library changes:
|
|
274
266
|
|
|
275
267
|
**npm**:
|
|
276
268
|
|
|
@@ -65,6 +65,7 @@ List of required environment variables for each model provider and gateway suppo
|
|
|
65
65
|
| [LLM Gateway](https://mastra.ai/models/providers/llmgateway) | `llmgateway/*` | `LLMGATEWAY_API_KEY` |
|
|
66
66
|
| [LLMTR](https://mastra.ai/models/providers/llmtr) | `llmtr/*` | `LLMTR_API_KEY` |
|
|
67
67
|
| [LMStudio](https://mastra.ai/models/providers/lmstudio) | `lmstudio/*` | `LMSTUDIO_API_KEY` |
|
|
68
|
+
| [LongCat](https://mastra.ai/models/providers/longcat) | `longcat/*` | `LONGCAT_API_KEY` |
|
|
68
69
|
| [LucidQuery](https://mastra.ai/models/providers/lucidquery) | `lucidquery/*` | `LUCIDQUERY_API_KEY` |
|
|
69
70
|
| [Meganova](https://mastra.ai/models/providers/meganova) | `meganova/*` | `MEGANOVA_API_KEY` |
|
|
70
71
|
| [MiniMax (minimax.io)](https://mastra.ai/models/providers/minimax) | `minimax/*` | `MINIMAX_API_KEY` |
|
|
@@ -74,11 +74,11 @@ ANTHROPIC_API_KEY=ant-...
|
|
|
74
74
|
| `amazon/nova-micro` |
|
|
75
75
|
| `amazon/nova-pro` |
|
|
76
76
|
| `amazon/titan-embed-text-v2` |
|
|
77
|
+
| `anthropic/claude-3-haiku` |
|
|
77
78
|
| `anthropic/claude-3.5-haiku` |
|
|
78
79
|
| `anthropic/claude-fable-5` |
|
|
79
80
|
| `anthropic/claude-haiku-4.5` |
|
|
80
81
|
| `anthropic/claude-opus-4` |
|
|
81
|
-
| `anthropic/claude-opus-4.1` |
|
|
82
82
|
| `anthropic/claude-opus-4.5` |
|
|
83
83
|
| `anthropic/claude-opus-4.6` |
|
|
84
84
|
| `anthropic/claude-opus-4.7` |
|
package/.docs/models/index.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# Model Providers
|
|
4
4
|
|
|
5
|
-
Mastra provides a unified interface for working with LLMs across multiple providers, giving you access to
|
|
5
|
+
Mastra provides a unified interface for working with LLMs across multiple providers, giving you access to 4555 models from 140 providers through a single API.
|
|
6
6
|
|
|
7
7
|
## Features
|
|
8
8
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# Anthropic
|
|
4
4
|
|
|
5
|
-
Access
|
|
5
|
+
Access 12 Anthropic models through Mastra's model router. Authentication is handled automatically using the `ANTHROPIC_API_KEY` environment variable.
|
|
6
6
|
|
|
7
7
|
Learn more in the [Anthropic documentation](https://docs.anthropic.com/en/docs/about-claude/models).
|
|
8
8
|
|
|
@@ -37,19 +37,13 @@ for await (const chunk of stream) {
|
|
|
37
37
|
| `anthropic/claude-fable-5` | 1.0M | | | | | | $10 | $50 |
|
|
38
38
|
| `anthropic/claude-haiku-4-5` | 200K | | | | | | $1 | $5 |
|
|
39
39
|
| `anthropic/claude-haiku-4-5-20251001` | 200K | | | | | | $1 | $5 |
|
|
40
|
-
| `anthropic/claude-opus-4-0` | 200K | | | | | | $15 | $75 |
|
|
41
|
-
| `anthropic/claude-opus-4-1` | 200K | | | | | | $15 | $75 |
|
|
42
|
-
| `anthropic/claude-opus-4-1-20250805` | 200K | | | | | | $15 | $75 |
|
|
43
|
-
| `anthropic/claude-opus-4-20250514` | 200K | | | | | | $15 | $75 |
|
|
44
40
|
| `anthropic/claude-opus-4-5` | 200K | | | | | | $5 | $25 |
|
|
45
41
|
| `anthropic/claude-opus-4-5-20251101` | 200K | | | | | | $5 | $25 |
|
|
46
42
|
| `anthropic/claude-opus-4-6` | 1.0M | | | | | | $5 | $25 |
|
|
47
43
|
| `anthropic/claude-opus-4-7` | 1.0M | | | | | | $5 | $25 |
|
|
48
44
|
| `anthropic/claude-opus-4-8` | 1.0M | | | | | | $5 | $25 |
|
|
49
|
-
| `anthropic/claude-sonnet-4-
|
|
50
|
-
| `anthropic/claude-sonnet-4-
|
|
51
|
-
| `anthropic/claude-sonnet-4-5` | 200K | | | | | | $3 | $15 |
|
|
52
|
-
| `anthropic/claude-sonnet-4-5-20250929` | 200K | | | | | | $3 | $15 |
|
|
45
|
+
| `anthropic/claude-sonnet-4-5` | 1.0M | | | | | | $3 | $15 |
|
|
46
|
+
| `anthropic/claude-sonnet-4-5-20250929` | 1.0M | | | | | | $3 | $15 |
|
|
53
47
|
| `anthropic/claude-sonnet-4-6` | 1.0M | | | | | | $3 | $15 |
|
|
54
48
|
| `anthropic/claude-sonnet-5` | 1.0M | | | | | | $2 | $10 |
|
|
55
49
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# Deep Infra
|
|
4
4
|
|
|
5
|
-
Access
|
|
5
|
+
Access 40 Deep Infra models through Mastra's model router. Authentication is handled automatically using the `DEEPINFRA_API_KEY` environment variable.
|
|
6
6
|
|
|
7
7
|
Learn more in the [Deep Infra documentation](https://deepinfra.com/models).
|
|
8
8
|
|
|
@@ -36,7 +36,7 @@ for await (const chunk of stream) {
|
|
|
36
36
|
| ------------------------------------------------------------- | ------- | ----- | --------- | ----- | ----- | ----- | ---------- | ----------- |
|
|
37
37
|
| `deepinfra/deepseek-ai/DeepSeek-R1-0528` | 164K | | | | | | $0.50 | $2 |
|
|
38
38
|
| `deepinfra/deepseek-ai/DeepSeek-V3.2` | 164K | | | | | | $0.26 | $0.38 |
|
|
39
|
-
| `deepinfra/deepseek-ai/DeepSeek-V4-Flash` | 1.0M | | | | | | $0.
|
|
39
|
+
| `deepinfra/deepseek-ai/DeepSeek-V4-Flash` | 1.0M | | | | | | $0.09 | $0.18 |
|
|
40
40
|
| `deepinfra/deepseek-ai/DeepSeek-V4-Pro` | 1.0M | | | | | | $1 | $3 |
|
|
41
41
|
| `deepinfra/google/gemma-4-26B-A4B-it` | 262K | | | | | | $0.07 | $0.34 |
|
|
42
42
|
| `deepinfra/google/gemma-4-31B-it` | 262K | | | | | | $0.13 | $0.38 |
|
|
@@ -44,15 +44,28 @@ for await (const chunk of stream) {
|
|
|
44
44
|
| `deepinfra/meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8` | 1.0M | | | | | | $0.15 | $0.60 |
|
|
45
45
|
| `deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct` | 328K | | | | | | $0.10 | $0.30 |
|
|
46
46
|
| `deepinfra/MiniMaxAI/MiniMax-M2.5` | 197K | | | | | | $0.15 | $1 |
|
|
47
|
+
| `deepinfra/MiniMaxAI/MiniMax-M2.7` | 197K | | | | | | $0.25 | $1 |
|
|
48
|
+
| `deepinfra/MiniMaxAI/MiniMax-M3` | 524K | | | | | | $0.30 | $1 |
|
|
47
49
|
| `deepinfra/moonshotai/Kimi-K2.5` | 262K | | | | | | $0.45 | $2 |
|
|
48
50
|
| `deepinfra/moonshotai/Kimi-K2.6` | 262K | | | | | | $0.75 | $4 |
|
|
49
51
|
| `deepinfra/moonshotai/Kimi-K2.7-Code` | 262K | | | | | | $0.74 | $4 |
|
|
50
|
-
| `deepinfra/
|
|
52
|
+
| `deepinfra/nvidia/Llama-3.3-Nemotron-Super-49B-v1.5` | 131K | | | | | | $0.40 | $0.40 |
|
|
53
|
+
| `deepinfra/nvidia/Nemotron-3-Nano-30B-A3B` | 262K | | | | | | $0.05 | $0.20 |
|
|
54
|
+
| `deepinfra/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning` | 262K | | | | | | $0.20 | $0.80 |
|
|
55
|
+
| `deepinfra/openai/gpt-oss-120b` | 131K | | | | | | $0.04 | $0.17 |
|
|
51
56
|
| `deepinfra/openai/gpt-oss-20b` | 131K | | | | | | $0.03 | $0.14 |
|
|
57
|
+
| `deepinfra/Qwen/Qwen3-32B` | 41K | | | | | | $0.08 | $0.28 |
|
|
52
58
|
| `deepinfra/Qwen/Qwen3-Coder-480B-A35B-Instruct-Turbo` | 262K | | | | | | $0.30 | $1 |
|
|
59
|
+
| `deepinfra/Qwen/Qwen3-Max` | 256K | | | | | | $1 | $6 |
|
|
60
|
+
| `deepinfra/Qwen/Qwen3-Next-80B-A3B-Instruct` | 262K | | | | | | $0.09 | $1 |
|
|
61
|
+
| `deepinfra/Qwen/Qwen3.5-122B-A10B` | 16K | | | | | | $0.29 | $2 |
|
|
62
|
+
| `deepinfra/Qwen/Qwen3.5-27B` | 262K | | | | | | $0.26 | $3 |
|
|
53
63
|
| `deepinfra/Qwen/Qwen3.5-35B-A3B` | 262K | | | | | | $0.14 | $1 |
|
|
54
64
|
| `deepinfra/Qwen/Qwen3.5-397B-A17B` | 262K | | | | | | $0.45 | $3 |
|
|
65
|
+
| `deepinfra/Qwen/Qwen3.5-9B` | 262K | | | | | | $0.10 | $0.15 |
|
|
66
|
+
| `deepinfra/Qwen/Qwen3.6-27B` | 262K | | | | | | $0.32 | $3 |
|
|
55
67
|
| `deepinfra/Qwen/Qwen3.6-35B-A3B` | 262K | | | | | | $0.15 | $0.95 |
|
|
68
|
+
| `deepinfra/Qwen/Qwen3.7-Max` | 256K | | | | | | $3 | $8 |
|
|
56
69
|
| `deepinfra/XiaomiMiMo/MiMo-V2.5` | 262K | | | | | | $0.40 | $2 |
|
|
57
70
|
| `deepinfra/XiaomiMiMo/MiMo-V2.5-Pro` | 1.0M | | | | | | $1 | $3 |
|
|
58
71
|
| `deepinfra/zai-org/GLM-4.6` | 203K | | | | | | $0.43 | $2 |
|
|
@@ -60,7 +73,7 @@ for await (const chunk of stream) {
|
|
|
60
73
|
| `deepinfra/zai-org/GLM-4.7-Flash` | 203K | | | | | | $0.06 | $0.40 |
|
|
61
74
|
| `deepinfra/zai-org/GLM-5` | 203K | | | | | | $0.60 | $2 |
|
|
62
75
|
| `deepinfra/zai-org/GLM-5.1` | 203K | | | | | | $1 | $4 |
|
|
63
|
-
| `deepinfra/zai-org/GLM-5.2` | 1.0M | | | | | | $0.
|
|
76
|
+
| `deepinfra/zai-org/GLM-5.2` | 1.0M | | | | | | $0.93 | $3 |
|
|
64
77
|
|
|
65
78
|
## Advanced configuration
|
|
66
79
|
|