@mastra/mcp-docs-server 1.2.16-alpha.4 → 1.2.16-alpha.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.docs/docs/agents/guardrails.md +11 -10
- package/.docs/docs/agents/processors.md +4 -4
- package/.docs/docs/agents/using-tools.md +36 -1
- package/.docs/models/gateways/netlify.md +2 -1
- package/.docs/models/gateways/openrouter.md +2 -1
- package/.docs/models/index.md +1 -1
- package/.docs/models/providers/crossmodel.md +2 -1
- package/.docs/models/providers/google.md +2 -1
- package/.docs/models/providers/hyper.md +4 -4
- package/.docs/models/providers/kilo.md +9 -8
- package/.docs/models/providers/llmgateway.md +3 -1
- package/.docs/models/providers/nano-gpt.md +1 -4
- package/.docs/models/providers/opencode.md +0 -1
- package/.docs/reference/agent-controller/agent-controller-class.md +18 -0
- package/.docs/reference/index.md +1 -1
- package/.docs/reference/processors/token-cost-control.md +154 -0
- package/CHANGELOG.md +14 -0
- package/package.json +4 -4
- package/.docs/reference/processors/cost-guard-processor.md +0 -115
|
@@ -188,16 +188,16 @@ Visit [`PIIDetector()`](https://mastra.ai/reference/processors/pii-detector) ref
|
|
|
188
188
|
|
|
189
189
|
### Enforce cost limits
|
|
190
190
|
|
|
191
|
-
The `
|
|
191
|
+
The `TokenCostControl()` monitors cumulative estimated cost across the agentic loop, blocking or warning when a monetary limit is exceeded. It queries cost data from observability storage before each LLM call. Cost checks are approximate, metrics are persisted asynchronously, so fast-running agents may briefly exceed the configured limit before the guard triggers.
|
|
192
192
|
|
|
193
193
|
```typescript
|
|
194
|
-
import {
|
|
194
|
+
import { TokenCostControl } from '@mastra/core/processors'
|
|
195
195
|
|
|
196
196
|
export const budgetedAgent = new Agent({
|
|
197
197
|
id: 'budgeted-agent',
|
|
198
198
|
name: 'Budgeted Agent',
|
|
199
199
|
inputProcessors: [
|
|
200
|
-
new
|
|
200
|
+
new TokenCostControl({
|
|
201
201
|
maxCost: 5.0,
|
|
202
202
|
scope: 'thread',
|
|
203
203
|
window: '24h',
|
|
@@ -206,7 +206,7 @@ export const budgetedAgent = new Agent({
|
|
|
206
206
|
})
|
|
207
207
|
```
|
|
208
208
|
|
|
209
|
-
Visit [`
|
|
209
|
+
Visit [`TokenCostControl()`](https://mastra.ai/reference/processors/token-cost-control) reference for scoping modes, time windows, metric persistence delays, and the `onViolation` callback. Requires observability storage with `getMetricAggregate` support.
|
|
210
210
|
|
|
211
211
|
## Processor strategies
|
|
212
212
|
|
|
@@ -232,18 +232,19 @@ All processors support an `onViolation` callback that fires when a policy violat
|
|
|
232
232
|
The callback receives a `ProcessorViolation` object with `processorId`, `message`, and `detail` (processor-specific metadata).
|
|
233
233
|
|
|
234
234
|
```typescript
|
|
235
|
-
import {
|
|
235
|
+
import { TokenCostControl, ModerationProcessor, PIIDetector } from '@mastra/core/processors'
|
|
236
236
|
|
|
237
237
|
// Alert when cost limits are exceeded
|
|
238
|
-
const
|
|
238
|
+
const tokenCostControl = new TokenCostControl({
|
|
239
239
|
maxCost: 10.0,
|
|
240
240
|
scope: 'resource',
|
|
241
241
|
window: '30d',
|
|
242
|
+
strategy: 'warn',
|
|
242
243
|
})
|
|
243
244
|
|
|
244
|
-
|
|
245
|
+
tokenCostControl.onViolation = ({ processorId, message, detail }) => {
|
|
245
246
|
alertSystem.notify(`[${processorId}] ${message}`)
|
|
246
|
-
// detail contains: { usage, limit, totalUsage, scope, scopeKey }
|
|
247
|
+
// detail contains: { usage, limit, totalUsage, scope, scopeKey, threshold }
|
|
247
248
|
}
|
|
248
249
|
|
|
249
250
|
// Log moderation violations
|
|
@@ -257,9 +258,9 @@ moderation.onViolation = ({ processorId, message, detail }) => {
|
|
|
257
258
|
}
|
|
258
259
|
```
|
|
259
260
|
|
|
260
|
-
The `onViolation` property is part of the base [`Processor` interface](https://mastra.ai/reference/processors/processor-interface), so any processor, including custom ones, can use it. The runner automatically invokes `onViolation` when any processor calls `abort()`. For processors using a `warn` strategy (like `
|
|
261
|
+
The `onViolation` property is part of the base [`Processor` interface](https://mastra.ai/reference/processors/processor-interface), so any processor, including custom ones, can use it. The runner automatically invokes `onViolation` when any processor calls `abort()`. For processors using a `warn` strategy (like `TokenCostControl`), the callback also fires on warnings without blocking the request.
|
|
261
262
|
|
|
262
|
-
Errors thrown by
|
|
263
|
+
Errors thrown by runner-invoked callbacks after `abort()` are silently caught to prevent interfering with the processor's main logic. On the `TokenCostControl` warning path, callback errors are caught and logged through the Mastra logger.
|
|
263
264
|
|
|
264
265
|
For more on how violation callbacks integrate with the processor pipeline, see [Violation callbacks](https://mastra.ai/docs/agents/processors) in the Processors documentation.
|
|
265
266
|
|
|
@@ -892,7 +892,7 @@ For `StreamErrorRetryProcessor`, also set its `maxRetries` to the same value. It
|
|
|
892
892
|
All processors expose an `onViolation` property that fires whenever a policy violation is detected, both when `abort()` is called (block strategy) and when a processor issues a warning (warn strategy). Use it for alerting, logging, or side effects without affecting the processor's main logic:
|
|
893
893
|
|
|
894
894
|
```typescript
|
|
895
|
-
import { ModerationProcessor,
|
|
895
|
+
import { ModerationProcessor, TokenCostControl } from '@mastra/core/processors'
|
|
896
896
|
|
|
897
897
|
const moderation = new ModerationProcessor({
|
|
898
898
|
model: 'openai/gpt-5-nano',
|
|
@@ -904,13 +904,13 @@ moderation.onViolation = ({ processorId, message, detail }) => {
|
|
|
904
904
|
monitor.track('processor_violation', { processorId, message, detail })
|
|
905
905
|
}
|
|
906
906
|
|
|
907
|
-
const
|
|
907
|
+
const tokenCostControl = new TokenCostControl({
|
|
908
908
|
maxCost: 10.0,
|
|
909
909
|
scope: 'resource',
|
|
910
910
|
window: '30d',
|
|
911
911
|
})
|
|
912
912
|
|
|
913
|
-
|
|
913
|
+
tokenCostControl.onViolation = ({ processorId, message, detail }) => {
|
|
914
914
|
alertSystem.notify(`[${processorId}] ${message}`)
|
|
915
915
|
}
|
|
916
916
|
```
|
|
@@ -921,7 +921,7 @@ The callback receives a `ProcessorViolation` object with:
|
|
|
921
921
|
- `message`: A human-readable description of what was violated
|
|
922
922
|
- `detail`: Processor-specific metadata (e.g. cost usage, detected PII types, moderation categories)
|
|
923
923
|
|
|
924
|
-
`onViolation` is part of the base [`Processor` interface](https://mastra.ai/reference/processors/processor-interface), so any custom processor can use it too. The runner automatically invokes it when any processor calls `abort()`. Errors thrown inside the callback are
|
|
924
|
+
`onViolation` is part of the base [`Processor` interface](https://mastra.ai/reference/processors/processor-interface), so any custom processor can use it too. The runner automatically invokes it when any processor calls `abort()`. Errors thrown inside the callback are caught to prevent interfering with the processor pipeline.
|
|
925
925
|
|
|
926
926
|
### Abort and tripwire chunks
|
|
927
927
|
|
|
@@ -287,7 +287,42 @@ Use `transform` when a tool returns raw data your application needs, but browser
|
|
|
287
287
|
|
|
288
288
|
If a transform is configured and it fails, Mastra doesn't fall back to the raw payload for display or transcript targets. Input deltas are suppressed when no safe `inputDelta` transform is available.
|
|
289
289
|
|
|
290
|
-
|
|
290
|
+
The following example redacts a secret from both targets:
|
|
291
|
+
|
|
292
|
+
```typescript
|
|
293
|
+
import { createTool } from '@mastra/core/tools'
|
|
294
|
+
import { z } from 'zod'
|
|
295
|
+
|
|
296
|
+
export const customerTool = createTool({
|
|
297
|
+
id: 'lookup-customer',
|
|
298
|
+
description: 'Looks up a customer by id',
|
|
299
|
+
inputSchema: z.object({
|
|
300
|
+
customerId: z.string(),
|
|
301
|
+
}),
|
|
302
|
+
outputSchema: z.object({
|
|
303
|
+
ssn: z.string(),
|
|
304
|
+
}),
|
|
305
|
+
execute: async ({ customerId }) => {
|
|
306
|
+
// const response = await fetch(`https://your-crm.example.com/customers/${customerId}`)
|
|
307
|
+
// const { ssn } = await response.json()
|
|
308
|
+
return {
|
|
309
|
+
ssn: '123-45-6789',
|
|
310
|
+
}
|
|
311
|
+
},
|
|
312
|
+
transform: {
|
|
313
|
+
display: {
|
|
314
|
+
output: () => ({ ssn: '***-**-****' }),
|
|
315
|
+
},
|
|
316
|
+
transcript: {
|
|
317
|
+
output: () => ({ ssn: '***-**-****' }),
|
|
318
|
+
},
|
|
319
|
+
},
|
|
320
|
+
})
|
|
321
|
+
```
|
|
322
|
+
|
|
323
|
+
Your application code still receives the raw `ssn` from the tool. Browser-facing streams get the `display` output, and user-visible transcript messages get the `transcript` output.
|
|
324
|
+
|
|
325
|
+
For more phases, including `input`, `inputDelta`, `error`, `approval`, `suspend`, and `resume`, see the [`createTool()` reference](https://mastra.ai/reference/tools/create-tool). For shared rules across several tools, configure the agent-level `transform` policy in the [`Agent` constructor](https://mastra.ai/reference/agents/agent).
|
|
291
326
|
|
|
292
327
|
## Run logic around tool calls
|
|
293
328
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# Netlify
|
|
4
4
|
|
|
5
|
-
Netlify AI Gateway provides unified access to multiple providers with built-in caching and observability. Access
|
|
5
|
+
Netlify AI Gateway provides unified access to multiple providers with built-in caching and observability. Access 67 models through Mastra's model router.
|
|
6
6
|
|
|
7
7
|
Learn more in the [Netlify documentation](https://docs.netlify.com/build/ai-gateway/overview/).
|
|
8
8
|
|
|
@@ -64,6 +64,7 @@ ANTHROPIC_API_KEY=ant-...
|
|
|
64
64
|
| `gemini/gemini-3.5-flash` |
|
|
65
65
|
| `gemini/gemini-3.5-flash-lite` |
|
|
66
66
|
| `gemini/gemini-3.6-flash` |
|
|
67
|
+
| `gemini/gemini-3.7-flash` |
|
|
67
68
|
| `gemini/gemini-flash-latest` |
|
|
68
69
|
| `gemini/gemini-flash-lite-latest` |
|
|
69
70
|
| `openai/chat-latest` |
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# OpenRouter
|
|
4
4
|
|
|
5
|
-
OpenRouter aggregates models from multiple providers with enhanced features like rate limiting and failover. Access
|
|
5
|
+
OpenRouter aggregates models from multiple providers with enhanced features like rate limiting and failover. Access 349 models through Mastra's model router.
|
|
6
6
|
|
|
7
7
|
Learn more in the [OpenRouter documentation](https://openrouter.ai/models).
|
|
8
8
|
|
|
@@ -125,6 +125,7 @@ ANTHROPIC_API_KEY=ant-...
|
|
|
125
125
|
| `google/gemini-3.5-flash` |
|
|
126
126
|
| `google/gemini-3.5-flash-lite` |
|
|
127
127
|
| `google/gemini-3.6-flash` |
|
|
128
|
+
| `google/gemini-3.7-flash` |
|
|
128
129
|
| `google/gemma-2-27b-it` |
|
|
129
130
|
| `google/gemma-3-12b-it` |
|
|
130
131
|
| `google/gemma-3-27b-it` |
|
package/.docs/models/index.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# Model Providers
|
|
4
4
|
|
|
5
|
-
Mastra provides a unified interface for working with LLMs across multiple providers, giving you access to
|
|
5
|
+
Mastra provides a unified interface for working with LLMs across multiple providers, giving you access to 5661 models from 171 providers through a single API.
|
|
6
6
|
|
|
7
7
|
## Features
|
|
8
8
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# CrossModel
|
|
4
4
|
|
|
5
|
-
Access
|
|
5
|
+
Access 50 CrossModel models through Mastra's model router. Authentication is handled automatically using the `CROSSMODEL_API_KEY` environment variable.
|
|
6
6
|
|
|
7
7
|
Learn more in the [CrossModel documentation](https://www.crossmodel.ai/docs).
|
|
8
8
|
|
|
@@ -77,6 +77,7 @@ for await (const chunk of stream) {
|
|
|
77
77
|
| `crossmodel/tencent/hy3` | 262K | | | | | | $0.16 | $0.64 |
|
|
78
78
|
| `crossmodel/x-ai/grok-4.3` | 1.0M | | | | | | $1 | $3 |
|
|
79
79
|
| `crossmodel/x-ai/grok-4.5` | 500K | | | | | | $2 | $6 |
|
|
80
|
+
| `crossmodel/x-ai/grok-4.6` | 500K | | | | | | $2 | $6 |
|
|
80
81
|
| `crossmodel/x-ai/grok-build-0.1` | 256K | | | | | | $1 | $2 |
|
|
81
82
|
| `crossmodel/xiaomi/mimo-v2.5` | 1.0M | | | | | | $0.16 | $0.32 |
|
|
82
83
|
| `crossmodel/xiaomi/mimo-v2.5-pro` | 1.0M | | | | | | $0.47 | $0.94 |
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# Google
|
|
4
4
|
|
|
5
|
-
Access
|
|
5
|
+
Access 39 Google models through Mastra's model router. Authentication is handled automatically using one of the following environment variables: `GOOGLE_API_KEY`, `GOOGLE_GENERATIVE_AI_API_KEY`.
|
|
6
6
|
|
|
7
7
|
Learn more in the [Google documentation](https://ai.google.dev/gemini-api/docs/models).
|
|
8
8
|
|
|
@@ -59,6 +59,7 @@ for await (const chunk of stream) {
|
|
|
59
59
|
| `google/gemini-3.5-flash-lite` | 1.0M | | | | | | $0.30 | $3 |
|
|
60
60
|
| `google/gemini-3.5-live-translate-preview` | 16K | | | | | | $4 | $21 |
|
|
61
61
|
| `google/gemini-3.6-flash` | 1.0M | | | | | | $2 | $8 |
|
|
62
|
+
| `google/gemini-3.7-flash` | 1.0M | | | | | | $0.75 | $4 |
|
|
62
63
|
| `google/gemini-embedding-001` | 2K | | | | | | $0.15 | — |
|
|
63
64
|
| `google/gemini-embedding-2` | 8K | | | | | | $0.20 | — |
|
|
64
65
|
| `google/gemini-flash-latest` | 1.0M | | | | | | $2 | $9 |
|
|
@@ -39,17 +39,17 @@ for await (const chunk of stream) {
|
|
|
39
39
|
| `hyper/deepseek-v4-flash` | 1.0M | | | | | | $0.20 | $0.40 |
|
|
40
40
|
| `hyper/deepseek-v4-flash-0731` | 1.0M | | | | | | $0.15 | $0.30 |
|
|
41
41
|
| `hyper/deepseek-v4-pro` | 1.0M | | | | | | $2 | $5 |
|
|
42
|
-
| `hyper/gemma-4-26b-a4b-it` | 256K | | | | | | $0.
|
|
42
|
+
| `hyper/gemma-4-26b-a4b-it` | 256K | | | | | | $0.13 | $0.42 |
|
|
43
43
|
| `hyper/glm-5.1` | 203K | | | | | | $2 | $5 |
|
|
44
44
|
| `hyper/glm-5.2` | 1.0M | | | | | | $1 | $4 |
|
|
45
|
-
| `hyper/gpt-oss-120b` | 131K | | | | | | $0.19 | $0.
|
|
46
|
-
| `hyper/kimi-k2.5` | 262K | | | | | | $0.
|
|
45
|
+
| `hyper/gpt-oss-120b` | 131K | | | | | | $0.19 | $0.70 |
|
|
46
|
+
| `hyper/kimi-k2.5` | 262K | | | | | | $0.53 | $3 |
|
|
47
47
|
| `hyper/kimi-k2.6` | 262K | | | | | | $0.95 | $4 |
|
|
48
48
|
| `hyper/kimi-k2.7-code` | 256K | | | | | | $0.95 | $4 |
|
|
49
49
|
| `hyper/kimi-k3` | 1.0M | | | | | | $3 | $16 |
|
|
50
50
|
| `hyper/llama-3.3-70b-instruct` | 128K | | | | | | $0.60 | $0.74 |
|
|
51
51
|
| `hyper/llama-4-maverick-17b-128e-instruct-fp8` | 430K | | | | | | $0.27 | $0.90 |
|
|
52
|
-
| `hyper/minimax-m2.7` | 262K | | | | | | $0.
|
|
52
|
+
| `hyper/minimax-m2.7` | 262K | | | | | | $0.47 | $2 |
|
|
53
53
|
| `hyper/minimax-m3` | 512K | | | | | | $0.33 | $1 |
|
|
54
54
|
| `hyper/qwen3-coder-480b-a35b-instruct-int4-mixed-ar` | 106K | | | | | | $0.45 | $2 |
|
|
55
55
|
| `hyper/qwen3-next-80b-a3b-instruct` | 262K | | | | | | $0.12 | $1 |
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# Kilo Gateway
|
|
4
4
|
|
|
5
|
-
Access
|
|
5
|
+
Access 357 Kilo Gateway models through Mastra's model router. Authentication is handled automatically using the `KILO_API_KEY` environment variable.
|
|
6
6
|
|
|
7
7
|
Learn more in the [Kilo Gateway documentation](https://kilo.ai).
|
|
8
8
|
|
|
@@ -41,7 +41,7 @@ for await (const chunk of stream) {
|
|
|
41
41
|
| `kilo/~anthropic/claude-opus-latest` | 1.0M | | | | | | $5 | $25 |
|
|
42
42
|
| `kilo/~anthropic/claude-sonnet-latest` | 1.0M | | | | | | $2 | $10 |
|
|
43
43
|
| `kilo/~deepseek/deepseek-v4-flash-latest` | 1.0M | | | | | | $0.08 | $0.25 |
|
|
44
|
-
| `kilo/~google/gemini-flash-latest` | 1.0M | | | | | | $
|
|
44
|
+
| `kilo/~google/gemini-flash-latest` | 1.0M | | | | | | $0.38 | $2 |
|
|
45
45
|
| `kilo/~google/gemini-pro-latest` | 1.0M | | | | | | $2 | $12 |
|
|
46
46
|
| `kilo/~moonshotai/kimi-latest` | 1.0M | | | | | | $3 | $14 |
|
|
47
47
|
| `kilo/~openai/gpt-latest` | 1.1M | | | | | | $5 | $30 |
|
|
@@ -127,6 +127,7 @@ for await (const chunk of stream) {
|
|
|
127
127
|
| `kilo/google/gemini-3.5-flash` | 1.0M | | | | | | $2 | $9 |
|
|
128
128
|
| `kilo/google/gemini-3.5-flash-lite` | 1.0M | | | | | | $0.30 | $3 |
|
|
129
129
|
| `kilo/google/gemini-3.6-flash` | 1.0M | | | | | | $2 | $8 |
|
|
130
|
+
| `kilo/google/gemini-3.7-flash` | 1.0M | | | | | | $0.75 | $4 |
|
|
130
131
|
| `kilo/google/gemma-2-27b-it` | 8K | | | | | | $0.65 | $0.65 |
|
|
131
132
|
| `kilo/google/gemma-3-12b-it` | 131K | | | | | | $0.05 | $0.15 |
|
|
132
133
|
| `kilo/google/gemma-3-27b-it` | 131K | | | | | | $0.08 | $0.16 |
|
|
@@ -338,7 +339,7 @@ for await (const chunk of stream) {
|
|
|
338
339
|
| `kilo/qwen/qwen3.7-flash` | 1.0M | | | | | | $0.03 | $0.13 |
|
|
339
340
|
| `kilo/qwen/qwen3.7-max` | 1.0M | | | | | | $1 | $4 |
|
|
340
341
|
| `kilo/qwen/qwen3.7-plus` | 1.0M | | | | | | $0.32 | $1 |
|
|
341
|
-
| `kilo/qwen/qwen3.8-2.4t-a95b` |
|
|
342
|
+
| `kilo/qwen/qwen3.8-2.4t-a95b` | 1.0M | | | | | | $2 | $6 |
|
|
342
343
|
| `kilo/qwen/qwen3.8-max` | 1.0M | | | | | | $2 | $6 |
|
|
343
344
|
| `kilo/rekaai/reka-edge` | 16K | | | | | | $0.10 | $0.10 |
|
|
344
345
|
| `kilo/rekaai/reka-flash-3` | 66K | | | | | | $0.10 | $0.20 |
|
|
@@ -383,14 +384,14 @@ for await (const chunk of stream) {
|
|
|
383
384
|
| `kilo/z-ai/glm-4.5` | 131K | | | | | | $0.60 | $2 |
|
|
384
385
|
| `kilo/z-ai/glm-4.5-air` | 131K | | | | | | $0.13 | $0.85 |
|
|
385
386
|
| `kilo/z-ai/glm-4.5v` | 66K | | | | | | $0.60 | $2 |
|
|
386
|
-
| `kilo/z-ai/glm-4.6` | 203K | | | | | | $0.
|
|
387
|
+
| `kilo/z-ai/glm-4.6` | 203K | | | | | | $0.50 | $2 |
|
|
387
388
|
| `kilo/z-ai/glm-4.6v` | 131K | | | | | | $0.30 | $0.90 |
|
|
388
|
-
| `kilo/z-ai/glm-4.7` | 203K | | | | | | $0.
|
|
389
|
-
| `kilo/z-ai/glm-4.7-flash` | 203K | | | | | | $0.
|
|
390
|
-
| `kilo/z-ai/glm-5` | 205K | | | | | | $
|
|
389
|
+
| `kilo/z-ai/glm-4.7` | 203K | | | | | | $0.40 | $2 |
|
|
390
|
+
| `kilo/z-ai/glm-4.7-flash` | 203K | | | | | | $0.06 | $0.40 |
|
|
391
|
+
| `kilo/z-ai/glm-5` | 205K | | | | | | $0.95 | $3 |
|
|
391
392
|
| `kilo/z-ai/glm-5-turbo` | 203K | | | | | | $1 | $4 |
|
|
392
393
|
| `kilo/z-ai/glm-5.1` | 203K | | | | | | $1 | $4 |
|
|
393
|
-
| `kilo/z-ai/glm-5.2` |
|
|
394
|
+
| `kilo/z-ai/glm-5.2` | 1.0M | | | | | | $1 | $4 |
|
|
394
395
|
| `kilo/z-ai/glm-5v-turbo` | 203K | | | | | | $1 | $4 |
|
|
395
396
|
|
|
396
397
|
## Advanced configuration
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# LLM Gateway
|
|
4
4
|
|
|
5
|
-
Access
|
|
5
|
+
Access 188 LLM Gateway models through Mastra's model router. Authentication is handled automatically using the `LLMGATEWAY_API_KEY` environment variable.
|
|
6
6
|
|
|
7
7
|
Learn more in the [LLM Gateway documentation](https://llmgateway.io/docs).
|
|
8
8
|
|
|
@@ -57,6 +57,7 @@ for await (const chunk of stream) {
|
|
|
57
57
|
| `llmgateway/deepseek-v3.2` | 164K | | | | | | $0.26 | $0.38 |
|
|
58
58
|
| `llmgateway/deepseek-v4-flash` | 1.1M | | | | | | $0.08 | $0.15 |
|
|
59
59
|
| `llmgateway/deepseek-v4-pro` | 1.1M | | | | | | $0.43 | $0.87 |
|
|
60
|
+
| `llmgateway/ernie-4.5-vl-424b-a47b` | 123K | | | | | | $0.42 | $1 |
|
|
60
61
|
| `llmgateway/fugu-ultra` | 1.0M | | | | | | $5 | $30 |
|
|
61
62
|
| `llmgateway/gemini-2.5-flash` | 1.0M | | | | | | $0.30 | $3 |
|
|
62
63
|
| `llmgateway/gemini-2.5-flash-lite` | 1.0M | | | | | | $0.10 | $0.40 |
|
|
@@ -218,6 +219,7 @@ for await (const chunk of stream) {
|
|
|
218
219
|
| `llmgateway/seed-1-6-250915` | 256K | | | | | | $0.25 | $2 |
|
|
219
220
|
| `llmgateway/seed-1-6-flash-250715` | 256K | | | | | | $0.07 | $0.30 |
|
|
220
221
|
| `llmgateway/seed-1-8-251228` | 256K | | | | | | $0.25 | $2 |
|
|
222
|
+
| `llmgateway/seed-2-1-turbo` | 262K | | | | | | $0.50 | $3 |
|
|
221
223
|
| `llmgateway/sonar` | 130K | | | | | | $1 | $1 |
|
|
222
224
|
| `llmgateway/sonar-pro` | 200K | | | | | | $3 | $15 |
|
|
223
225
|
| `llmgateway/sonar-reasoning-pro` | 128K | | | | | | $2 | $8 |
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# NanoGPT
|
|
4
4
|
|
|
5
|
-
Access
|
|
5
|
+
Access 596 NanoGPT models through Mastra's model router. Authentication is handled automatically using the `NANO_GPT_API_KEY` environment variable.
|
|
6
6
|
|
|
7
7
|
Learn more in the [NanoGPT documentation](https://docs.nano-gpt.com).
|
|
8
8
|
|
|
@@ -266,8 +266,6 @@ for await (const chunk of stream) {
|
|
|
266
266
|
| `nano-gpt/inclusionai/ling-2.6-flash` | 262K | | | | | | $0.10 | $0.30 |
|
|
267
267
|
| `nano-gpt/inclusionai/ling-3.0-flash` | 262K | | | | | | $0.07 | $0.22 |
|
|
268
268
|
| `nano-gpt/inclusionai/ling-3.0-flash:thinking` | 262K | | | | | | $0.07 | $0.22 |
|
|
269
|
-
| `nano-gpt/inclusionai/ling-3.0-tiny` | 262K | | | | | | $0.10 | $0.20 |
|
|
270
|
-
| `nano-gpt/inclusionai/ling-3.0-tiny:thinking` | 262K | | | | | | $0.10 | $0.20 |
|
|
271
269
|
| `nano-gpt/inclusionai/ring-2.6-1t` | 262K | | | | | | $0.30 | $3 |
|
|
272
270
|
| `nano-gpt/inflatebot/MN-12B-Mag-Mell-R1` | 16K | | | | | | $0.49 | $0.49 |
|
|
273
271
|
| `nano-gpt/inflection/inflection-3-pi` | 8K | | | | | | $2 | $10 |
|
|
@@ -546,7 +544,6 @@ for await (const chunk of stream) {
|
|
|
546
544
|
| `nano-gpt/TEE/kimi-k2.6` | 262K | | | | | | $2 | $5 |
|
|
547
545
|
| `nano-gpt/TEE/kimi-k3` | 1.0M | | | | | | $3 | $15 |
|
|
548
546
|
| `nano-gpt/TEE/llama3-3-70b` | 128K | | | | | | $2 | $3 |
|
|
549
|
-
| `nano-gpt/TEE/minimax-m2.5` | 197K | | | | | | $0.20 | $1 |
|
|
550
547
|
| `nano-gpt/TEE/muse-glimmer-30b` | 131K | | | | | | $0.35 | $2 |
|
|
551
548
|
| `nano-gpt/TEE/qwen2.5-vl-72b-instruct` | 66K | | | | | | $0.70 | $0.70 |
|
|
552
549
|
| `nano-gpt/TEE/qwen3.5-122b-a10b` | 262K | | | | | | $0.46 | $4 |
|
|
@@ -88,7 +88,6 @@ for await (const chunk of stream) {
|
|
|
88
88
|
| `opencode/kimi-k2.7-code` | 262K | | | | | | $0.95 | $4 |
|
|
89
89
|
| `opencode/kimi-k3` | 1.0M | | | | | | $3 | $15 |
|
|
90
90
|
| `opencode/laguna-s-2.1-free` | 256K | | | | | | — | — |
|
|
91
|
-
| `opencode/ling-3.0-tiny-free` | 262K | | | | | | — | — |
|
|
92
91
|
| `opencode/mimo-v2.5-free` | 200K | | | | | | — | — |
|
|
93
92
|
| `opencode/minimax-m2.5` | 205K | | | | | | $0.30 | $1 |
|
|
94
93
|
| `opencode/minimax-m2.7` | 205K | | | | | | $0.30 | $1 |
|
|
@@ -245,6 +245,24 @@ const resourceIds = await controller.getKnownResourceIds(session)
|
|
|
245
245
|
|
|
246
246
|
Returns: `Promise<string[]>`
|
|
247
247
|
|
|
248
|
+
#### `listActiveThreadRuns()`
|
|
249
|
+
|
|
250
|
+
List every run in flight on this controller, across all resources and threads. The list comes from in-process run tracking, so it covers runs started by the current process and doesn't create or touch a session.
|
|
251
|
+
|
|
252
|
+
```typescript
|
|
253
|
+
const runs = controller.listActiveThreadRuns()
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
Returns: `ActiveThreadRun[]`
|
|
257
|
+
|
|
258
|
+
```typescript
|
|
259
|
+
interface ActiveThreadRun {
|
|
260
|
+
runId: string
|
|
261
|
+
resourceId?: string
|
|
262
|
+
threadId: string
|
|
263
|
+
}
|
|
264
|
+
```
|
|
265
|
+
|
|
248
266
|
### Lifecycle
|
|
249
267
|
|
|
250
268
|
#### `init()`
|
package/.docs/reference/index.md
CHANGED
|
@@ -243,7 +243,6 @@ The Reference section provides documentation of Mastra's API, including paramete
|
|
|
243
243
|
- [Span filtering](https://mastra.ai/reference/observability/tracing/span-filtering)
|
|
244
244
|
- [Spans](https://mastra.ai/reference/observability/tracing/spans)
|
|
245
245
|
- [BatchPartsProcessor](https://mastra.ai/reference/processors/batch-parts-processor)
|
|
246
|
-
- [CostGuardProcessor](https://mastra.ai/reference/processors/cost-guard-processor)
|
|
247
246
|
- [LanguageDetector](https://mastra.ai/reference/processors/language-detector)
|
|
248
247
|
- [MessageHistory](https://mastra.ai/reference/processors/message-history-processor)
|
|
249
248
|
- [ModerationProcessor](https://mastra.ai/reference/processors/moderation-processor)
|
|
@@ -258,6 +257,7 @@ The Reference section provides documentation of Mastra's API, including paramete
|
|
|
258
257
|
- [SkillSearchProcessor](https://mastra.ai/reference/processors/skill-search-processor)
|
|
259
258
|
- [StreamErrorRetryProcessor](https://mastra.ai/reference/processors/stream-error-retry-processor)
|
|
260
259
|
- [SystemPromptScrubber](https://mastra.ai/reference/processors/system-prompt-scrubber)
|
|
260
|
+
- [TokenCostControl](https://mastra.ai/reference/processors/token-cost-control)
|
|
261
261
|
- [TokenLimiterProcessor](https://mastra.ai/reference/processors/token-limiter-processor)
|
|
262
262
|
- [ToolCallFilter](https://mastra.ai/reference/processors/tool-call-filter)
|
|
263
263
|
- [ToolSearchProcessor](https://mastra.ai/reference/processors/tool-search-processor)
|
|
@@ -0,0 +1,154 @@
|
|
|
1
|
+
> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
|
|
2
|
+
|
|
3
|
+
# TokenCostControl
|
|
4
|
+
|
|
5
|
+
The `TokenCostControl` enforces monetary cost limits across the agentic loop, blocking or warning when a configurable cost threshold is exceeded.
|
|
6
|
+
|
|
7
|
+
It uses `processInputStep` to check the cost limit before each LLM call. Cost data is queried from the observability storage APIs (`getMetricAggregate`) for all scopes. For all scopes except `run`, it aggregates cost across runs within a configurable time window (defaults to 7 days). For `run` scope, it queries cost for the current trace.
|
|
8
|
+
|
|
9
|
+
For token-based limits, use `TokenLimiterProcessor` instead.
|
|
10
|
+
|
|
11
|
+
> **Renamed from `CostGuardProcessor`.** The `CostGuardProcessor` export (and its `CostGuard*` option and detail types) remains available as a deprecated alias for the same class, including the `'token-cost-control'` processor id. Migrate imports to `TokenCostControl`.
|
|
12
|
+
|
|
13
|
+
Supports six scoping modes:
|
|
14
|
+
|
|
15
|
+
- **Run scope**: Tracks cost within a single agent run via trace ID
|
|
16
|
+
- **Resource scope** (default): Tracks cumulative cost per `resourceId` across runs
|
|
17
|
+
- **Thread scope**: Tracks cumulative cost per `threadId` across runs
|
|
18
|
+
- **User scope**: Tracks cumulative cost per `userId` across runs
|
|
19
|
+
- **Organization scope**: Tracks cumulative cost per `organizationId` across runs
|
|
20
|
+
- **Session scope**: Tracks cumulative cost per `sessionId` across runs
|
|
21
|
+
|
|
22
|
+
> **Approximate cost control.** Cost data is persisted asynchronously via buffered exporters in the observability pipeline. Fast-running agents may exceed the configured limit before metrics are available for query. Treat `maxCost` as an approximate threshold that fast-running agents may exceed.
|
|
23
|
+
|
|
24
|
+
> **Agent attribution only.** Cost is attributed via the `entityType: 'agent'` metric filter. Model calls made outside an agent run (for example, direct model usage in workflow steps) have no agent parent span and aren't counted by this guard.
|
|
25
|
+
|
|
26
|
+
## Usage example
|
|
27
|
+
|
|
28
|
+
Track cumulative cost per resource (default scope):
|
|
29
|
+
|
|
30
|
+
```typescript
|
|
31
|
+
import { TokenCostControl } from '@mastra/core/processors'
|
|
32
|
+
|
|
33
|
+
const tokenCostControl = new TokenCostControl({
|
|
34
|
+
maxCost: 1.0,
|
|
35
|
+
})
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Track cumulative cost per thread with a 24-hour window and a soft warning at 80% of the limit:
|
|
39
|
+
|
|
40
|
+
```typescript
|
|
41
|
+
import { TokenCostControl } from '@mastra/core/processors'
|
|
42
|
+
|
|
43
|
+
const tokenCostControl = new TokenCostControl({
|
|
44
|
+
maxCost: 5.0,
|
|
45
|
+
scope: 'thread',
|
|
46
|
+
window: '24h',
|
|
47
|
+
warnAtPercent: 80,
|
|
48
|
+
})
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Use a per-tier budget by passing a function as `maxCost`:
|
|
52
|
+
|
|
53
|
+
```typescript
|
|
54
|
+
import { TokenCostControl } from '@mastra/core/processors'
|
|
55
|
+
|
|
56
|
+
const tokenCostControl = new TokenCostControl({
|
|
57
|
+
maxCost: requestContext => (requestContext?.get('tier') === 'pro' ? 10.0 : 1.0),
|
|
58
|
+
scope: 'user',
|
|
59
|
+
})
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Attach to an agent with an `onViolation` callback and a per-provider/model breakdown:
|
|
63
|
+
|
|
64
|
+
```typescript
|
|
65
|
+
import { Agent } from '@mastra/core/agent'
|
|
66
|
+
import { TokenCostControl } from '@mastra/core/processors'
|
|
67
|
+
|
|
68
|
+
const tokenCostControl = new TokenCostControl({
|
|
69
|
+
maxCost: 5.0,
|
|
70
|
+
scope: 'resource',
|
|
71
|
+
window: '30d',
|
|
72
|
+
strategy: 'warn',
|
|
73
|
+
includeBreakdown: true,
|
|
74
|
+
})
|
|
75
|
+
|
|
76
|
+
tokenCostControl.onViolation = ({ detail }) => {
|
|
77
|
+
console.log(
|
|
78
|
+
`Cost ${detail.threshold} threshold for ${detail.scopeKey}: $${detail.usage}/$${detail.limit}`,
|
|
79
|
+
)
|
|
80
|
+
for (const entry of detail.breakdown ?? []) {
|
|
81
|
+
console.log(` ${entry.provider}/${entry.model}: $${entry.estimatedCost}`)
|
|
82
|
+
}
|
|
83
|
+
}
|
|
84
|
+
|
|
85
|
+
const agent = new Agent({
|
|
86
|
+
id: 'my-agent',
|
|
87
|
+
name: 'my-agent',
|
|
88
|
+
model: 'openai/gpt-5-nano',
|
|
89
|
+
inputProcessors: [tokenCostControl],
|
|
90
|
+
})
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
## Constructor parameters
|
|
94
|
+
|
|
95
|
+
**maxCost** (`number | ((requestContext?: RequestContext) => number)`): Maximum estimated cost allowed (e.g. 0.50 for $0.50 USD). A number must be finite and positive. A function is called with the request's RequestContext on every check, enabling per-tier or per-user budgets; if it returns anything other than a finite positive number, the check is skipped for that request (fail-open) and a warning is logged. This is an approximate limit due to metric persistence delays.
|
|
96
|
+
|
|
97
|
+
**scope** (`'run' | 'resource' | 'thread' | 'user' | 'organization' | 'session'`): Scope for cost tracking. 'run' tracks cost within the current agent run via trace ID. 'resource' tracks cumulative cost per resourceId across runs (default). 'thread' tracks cumulative cost per threadId across runs. 'user', 'organization', and 'session' track cumulative cost per userId, organizationId, and sessionId respectively, read from the plain RequestContext keys 'userId', 'organizationId', and 'sessionId'. All scopes require observability storage with getMetricAggregate support. (Default: `'resource'`)
|
|
98
|
+
|
|
99
|
+
**window** (`'1h' | '6h' | '24h' | '7d' | '30d' | '365d'`): Time window for cost aggregation for all scopes except 'run'. (Default: `'7d'`)
|
|
100
|
+
|
|
101
|
+
**strategy** (`'block' | 'warn'`): Strategy when the cost limit is exceeded. 'block' aborts with a TripWire error. 'warn' logs a warning and calls onViolation at most once per request, then allows the step to proceed. (Default: `'block'`)
|
|
102
|
+
|
|
103
|
+
**message** (`string`): Custom message template for the abort reason. Supports {usage} and {limit} placeholders. (Default: `'Cost control: estimated cost limit exceeded ({usage}/{limit})'`)
|
|
104
|
+
|
|
105
|
+
**warnAtPercent** (`number`): Optional soft threshold as a percentage of maxCost (exclusive 0-100, e.g. 80). When the estimated cost reaches this percentage of the limit but is still below it, a warning is logged and onViolation is called once per request with threshold: "soft", regardless of strategy. Never aborts the step.
|
|
106
|
+
|
|
107
|
+
**includeBreakdown** (`boolean`): When true, violations (soft and hard) include a per-provider/model cost breakdown queried via getMetricBreakdown. The breakdown is capped at the top 10 provider/model groups ranked by aggregated token volume, not by spend. The breakdown query runs only when a violation trips, never on the happy path. If the configured store does not support breakdown queries or the query fails, the violation fires without the breakdown field. (Default: `false`)
|
|
108
|
+
|
|
109
|
+
## Instance properties
|
|
110
|
+
|
|
111
|
+
**id** (`'token-cost-control'`): Processor identifier.
|
|
112
|
+
|
|
113
|
+
**name** (`'Token Cost Control'`): Processor display name.
|
|
114
|
+
|
|
115
|
+
**onViolation** (`(violation: ProcessorViolation) => void | Promise<void>`): Callback invoked when a cost violation is detected, regardless of strategy. For the warn strategy and for soft thresholds, the guard calls it with a TokenCostControlViolationDetail (usage, limit, threshold, and optional breakdown) at most once per request per threshold level. Errors thrown by the callback on this path are caught and logged through the Mastra logger. For the block strategy, the processor runner invokes it with the TripWire metadata as the detail (see Error behavior) and silently catches callback errors. Use for side effects like alerting, logging to external systems, or emailing users.
|
|
116
|
+
|
|
117
|
+
**processInputStep** (`(args: ProcessInputStepArgs) => Promise<void>`): Checks cumulative estimated cost against the resolved maxCost before each LLM call. Queries observability storage for cost data: run scope filters by trace ID, all other scopes filter by their respective IDs with a time window. Calls abort() when the limit is exceeded (block strategy) or logs a warning (warn strategy). Cost checks are approximate due to metric persistence delays.
|
|
118
|
+
|
|
119
|
+
## Error behavior
|
|
120
|
+
|
|
121
|
+
When the `block` strategy is active (default), `TokenCostControl` calls `abort()` with `retry: false` when the cost limit is exceeded. The TripWire metadata includes:
|
|
122
|
+
|
|
123
|
+
- `processorId`: `'token-cost-control'`
|
|
124
|
+
- `usage`: Current cumulative usage (`estimatedCost`, `costUnit`)
|
|
125
|
+
- `maxCost`: The resolved cost limit for the request
|
|
126
|
+
- `scope`: The active scope
|
|
127
|
+
- `scopeKey`: The scope identifier for non-run scopes (if applicable)
|
|
128
|
+
- `threshold`: Always `'hard'`, since only the hard limit aborts
|
|
129
|
+
- `breakdown`: Per-provider/model cost entries (only when `includeBreakdown` is enabled and the breakdown query succeeds)
|
|
130
|
+
|
|
131
|
+
With the `warn` strategy, the hard-limit warning and `onViolation` callback fire at most once per request. Subsequent steps in the same request proceed without repeating the warning.
|
|
132
|
+
|
|
133
|
+
Numbers interpolated into violation messages are normalized to at most 6 decimal places, so messages never contain float precision artifacts.
|
|
134
|
+
|
|
135
|
+
## Scoping behavior
|
|
136
|
+
|
|
137
|
+
| Scope | Tracks across runs | Filter | Requires context |
|
|
138
|
+
| -------------- | ------------------ | ------------------------------ | ---------------------------------------- |
|
|
139
|
+
| `run` | No | `traceId` from current span | Tracing context (automatic) |
|
|
140
|
+
| `resource` | Yes | `resourceId` + time window | `resourceId` in `RequestContext` |
|
|
141
|
+
| `thread` | Yes | `threadId` + time window | `threadId` in `RequestContext` |
|
|
142
|
+
| `user` | Yes | `userId` + time window | `userId` key in `RequestContext` |
|
|
143
|
+
| `organization` | Yes | `organizationId` + time window | `organizationId` key in `RequestContext` |
|
|
144
|
+
| `session` | Yes | `sessionId` + time window | `sessionId` key in `RequestContext` |
|
|
145
|
+
|
|
146
|
+
All scopes require observability storage with `getMetricAggregate` support. If the Mastra instance doesn't have observability storage configured, an error is thrown at registration time.
|
|
147
|
+
|
|
148
|
+
For `run` scope, the processor reads the trace ID from the current span's tracing context. If no tracing context is available, the check is skipped (fail-open).
|
|
149
|
+
|
|
150
|
+
For all other scopes, if the required context ID is missing at runtime, the check is skipped. Observability query failures are handled with a fail-open strategy: if a query fails, a warning is logged through the Mastra logger and the step proceeds.
|
|
151
|
+
|
|
152
|
+
> **The `user`, `organization`, and `session` scopes require annotated traces.** These scopes match metric records by their `userId`, `organizationId`, and `sessionId` fields, which are populated from span metadata on the trace (for example, via tracing options metadata). If your traces don't carry the matching metadata, these scopes match zero records and the guard never trips. Setting the RequestContext key alone isn't enough: both the RequestContext key (for scope resolution) and the span metadata (for cost attribution) must be present.
|
|
153
|
+
|
|
154
|
+
> **Note on metric persistence delay.** The observability pipeline uses buffered exporters that flush metrics asynchronously. A short delay exists between when an LLM call completes and when its cost metrics are available for query. During high-frequency agent execution, the cost control may not detect a limit breach until one or more steps after the actual cost exceeded the threshold.
|
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
1
|
# @mastra/mcp-docs-server
|
|
2
2
|
|
|
3
|
+
## 1.2.16-alpha.7
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- Updated dependencies [[`79dd7c2`](https://github.com/mastra-ai/mastra/commit/79dd7c261ee6be1fafedd4651959394db21d2cba), [`b9a28ec`](https://github.com/mastra-ai/mastra/commit/b9a28ecf7acdc0cb7a543d5b660f9fbee301df9a), [`be31796`](https://github.com/mastra-ai/mastra/commit/be3179624ad5f77cff5fa342cd08046bf7605283)]:
|
|
8
|
+
- @mastra/core@1.59.0-alpha.4
|
|
9
|
+
|
|
10
|
+
## 1.2.16-alpha.5
|
|
11
|
+
|
|
12
|
+
### Patch Changes
|
|
13
|
+
|
|
14
|
+
- Updated dependencies [[`d118873`](https://github.com/mastra-ai/mastra/commit/d118873cfd5074b1f814a1c169a97ca7a3a29174), [`161258b`](https://github.com/mastra-ai/mastra/commit/161258b3473a6d0fce00a43cab59d119a49a232f), [`8ea8038`](https://github.com/mastra-ai/mastra/commit/8ea80386fde53d26e2c0b2060c53bc9bd9be10f3)]:
|
|
15
|
+
- @mastra/core@1.59.0-alpha.3
|
|
16
|
+
|
|
3
17
|
## 1.2.16-alpha.4
|
|
4
18
|
|
|
5
19
|
### Patch Changes
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@mastra/mcp-docs-server",
|
|
3
|
-
"version": "1.2.16-alpha.
|
|
3
|
+
"version": "1.2.16-alpha.7",
|
|
4
4
|
"description": "MCP server for accessing Mastra.ai documentation, changelogs, and news.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
@@ -28,7 +28,7 @@
|
|
|
28
28
|
"jsdom": "^26.1.0",
|
|
29
29
|
"local-pkg": "^1.1.2",
|
|
30
30
|
"zod": "^4.4.3",
|
|
31
|
-
"@mastra/core": "1.59.0-alpha.
|
|
31
|
+
"@mastra/core": "1.59.0-alpha.4",
|
|
32
32
|
"@mastra/mcp": "^1.16.0"
|
|
33
33
|
},
|
|
34
34
|
"devDependencies": {
|
|
@@ -45,9 +45,9 @@
|
|
|
45
45
|
"tsx": "^4.23.1",
|
|
46
46
|
"typescript": "^6.0.3",
|
|
47
47
|
"vitest": "4.1.10",
|
|
48
|
+
"@internal/types-builder": "0.0.97",
|
|
48
49
|
"@internal/lint": "0.0.122",
|
|
49
|
-
"@mastra/core": "1.59.0-alpha.
|
|
50
|
-
"@internal/types-builder": "0.0.97"
|
|
50
|
+
"@mastra/core": "1.59.0-alpha.4"
|
|
51
51
|
},
|
|
52
52
|
"homepage": "https://mastra.ai",
|
|
53
53
|
"repository": {
|
|
@@ -1,115 +0,0 @@
|
|
|
1
|
-
> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
|
|
2
|
-
|
|
3
|
-
# CostGuardProcessor
|
|
4
|
-
|
|
5
|
-
The `CostGuardProcessor` enforces monetary cost limits across the agentic loop, blocking or warning when a configurable cost threshold is exceeded.
|
|
6
|
-
|
|
7
|
-
It uses `processInputStep` to check the cost limit before each LLM call. Cost data is queried from the observability storage APIs (`getMetricAggregate`) for all scopes. For `resource` and `thread` scopes, it aggregates cost across runs within a configurable time window (defaults to 7 days). For `run` scope, it queries cost for the current trace.
|
|
8
|
-
|
|
9
|
-
For token-based limits, use `TokenLimiterProcessor` instead.
|
|
10
|
-
|
|
11
|
-
Supports three scoping modes:
|
|
12
|
-
|
|
13
|
-
- **Run scope**: Tracks cost within a single agent run via trace ID
|
|
14
|
-
- **Resource scope** (default): Tracks cumulative cost per `resourceId` across runs
|
|
15
|
-
- **Thread scope**: Tracks cumulative cost per `threadId` across runs
|
|
16
|
-
|
|
17
|
-
> **Approximate cost guard.** Cost data is persisted asynchronously via buffered exporters in the observability pipeline. Fast-running agents may exceed the configured limit before metrics are available for query. Treat `maxCost` as an approximate threshold that fast-running agents may exceed.
|
|
18
|
-
|
|
19
|
-
## Usage example
|
|
20
|
-
|
|
21
|
-
Track cumulative cost per resource (default scope):
|
|
22
|
-
|
|
23
|
-
```typescript
|
|
24
|
-
import { CostGuardProcessor } from '@mastra/core/processors'
|
|
25
|
-
|
|
26
|
-
const costGuard = new CostGuardProcessor({
|
|
27
|
-
maxCost: 1.0,
|
|
28
|
-
})
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
Track cumulative cost per thread with a 24-hour window:
|
|
32
|
-
|
|
33
|
-
```typescript
|
|
34
|
-
import { CostGuardProcessor } from '@mastra/core/processors'
|
|
35
|
-
|
|
36
|
-
const costGuard = new CostGuardProcessor({
|
|
37
|
-
maxCost: 5.0,
|
|
38
|
-
scope: 'thread',
|
|
39
|
-
window: '24h',
|
|
40
|
-
})
|
|
41
|
-
```
|
|
42
|
-
|
|
43
|
-
Attach to an agent with an `onViolation` callback:
|
|
44
|
-
|
|
45
|
-
```typescript
|
|
46
|
-
import { Agent } from '@mastra/core/agent'
|
|
47
|
-
import { CostGuardProcessor } from '@mastra/core/processors'
|
|
48
|
-
|
|
49
|
-
const costGuard = new CostGuardProcessor({
|
|
50
|
-
maxCost: 5.0,
|
|
51
|
-
scope: 'resource',
|
|
52
|
-
window: '30d',
|
|
53
|
-
})
|
|
54
|
-
|
|
55
|
-
costGuard.onViolation = ({ detail }) => {
|
|
56
|
-
console.log(`Cost exceeded for ${detail.scopeKey}: $${detail.usage}/$${detail.limit}`)
|
|
57
|
-
}
|
|
58
|
-
|
|
59
|
-
const agent = new Agent({
|
|
60
|
-
id: 'my-agent',
|
|
61
|
-
name: 'my-agent',
|
|
62
|
-
model: 'openai/gpt-5-nano',
|
|
63
|
-
processors: {
|
|
64
|
-
input: [costGuard],
|
|
65
|
-
},
|
|
66
|
-
})
|
|
67
|
-
```
|
|
68
|
-
|
|
69
|
-
## Constructor parameters
|
|
70
|
-
|
|
71
|
-
**maxCost** (`number`): Maximum estimated cost allowed (e.g. 0.50 for $0.50 USD). Must be a positive number. Uses cost data from observability metrics. This is an approximate limit due to metric persistence delays.
|
|
72
|
-
|
|
73
|
-
**scope** (`'run' | 'resource' | 'thread'`): Scope for cost tracking. 'run' tracks cost within the current agent run via trace ID. 'resource' tracks cumulative cost per resourceId across runs (default). 'thread' tracks cumulative cost per threadId across runs. All scopes require observability storage with getMetricAggregate support. (Default: `'resource'`)
|
|
74
|
-
|
|
75
|
-
**window** (`'1h' | '6h' | '24h' | '7d' | '30d' | '365d'`): Time window for cost aggregation when using 'resource' or 'thread' scope. Only applicable to non-run scopes. (Default: `'7d'`)
|
|
76
|
-
|
|
77
|
-
**strategy** (`'block' | 'warn'`): Strategy when the cost limit is exceeded. 'block' aborts with a TripWire error. 'warn' logs a warning but allows the step to proceed. (Default: `'block'`)
|
|
78
|
-
|
|
79
|
-
**message** (`string`): Custom message template for the abort reason. Supports {usage} and {limit} placeholders. (Default: `'Cost guard: cost limit exceeded ({usage}/{limit})'`)
|
|
80
|
-
|
|
81
|
-
## Instance properties
|
|
82
|
-
|
|
83
|
-
**id** (`'cost-guard'`): Processor identifier.
|
|
84
|
-
|
|
85
|
-
**name** (`'Cost Guard'`): Processor display name.
|
|
86
|
-
|
|
87
|
-
**onViolation** (`(violation: ProcessorViolation) => void | Promise<void>`): Callback invoked when a cost violation is detected, regardless of strategy. Part of the generalized Processor interface. Use for side effects like alerting, logging to external systems, or emailing users. Errors thrown by this callback are silently caught.
|
|
88
|
-
|
|
89
|
-
**processInputStep** (`(args: ProcessInputStepArgs) => Promise<void>`): Checks cumulative estimated cost against maxCost before each LLM call. Queries observability storage for cost data: run scope filters by trace ID, resource/thread scopes filter by their respective IDs with a time window. Calls abort() when the limit is exceeded (block strategy) or logs a warning (warn strategy). Cost checks are approximate due to metric persistence delays.
|
|
90
|
-
|
|
91
|
-
## Error behavior
|
|
92
|
-
|
|
93
|
-
When the `block` strategy is active (default), `CostGuardProcessor` calls `abort()` with `retry: false` when the cost limit is exceeded. The TripWire metadata includes:
|
|
94
|
-
|
|
95
|
-
- `processorId`: `'cost-guard'`
|
|
96
|
-
- `usage`: Current cumulative usage (`estimatedCost`, `costUnit`)
|
|
97
|
-
- `maxCost`: The configured cost limit
|
|
98
|
-
- `scope`: The active scope (`'run'`, `'resource'`, or `'thread'`)
|
|
99
|
-
- `scopeKey`: The scope identifier for resource/thread scopes (if applicable)
|
|
100
|
-
|
|
101
|
-
## Scoping behavior
|
|
102
|
-
|
|
103
|
-
| Scope | Tracks across runs | Filter | Requires context |
|
|
104
|
-
| ---------- | ------------------ | --------------------------- | -------------------------------- |
|
|
105
|
-
| `run` | No | `traceId` from current span | Tracing context (automatic) |
|
|
106
|
-
| `resource` | Yes | `resourceId` + time window | `resourceId` in `RequestContext` |
|
|
107
|
-
| `thread` | Yes | `threadId` + time window | `threadId` in `RequestContext` |
|
|
108
|
-
|
|
109
|
-
All scopes require observability storage with `getMetricAggregate` support. If the Mastra instance doesn't have observability storage configured, an error is thrown at registration time.
|
|
110
|
-
|
|
111
|
-
For `run` scope, the processor reads the trace ID from the current span's tracing context. If no tracing context is available, the check is skipped (fail-open).
|
|
112
|
-
|
|
113
|
-
For `resource` and `thread` scopes, if the required context ID is missing at runtime, the check is skipped. Observability query failures are handled with a fail-open strategy: if a query fails, cost is treated as zero.
|
|
114
|
-
|
|
115
|
-
> **Note on metric persistence delay.** The observability pipeline uses buffered exporters that flush metrics asynchronously. A short delay exists between when an LLM call completes and when its cost metrics are available for query. During high-frequency agent execution, the cost guard may not detect a limit breach until one or more steps after the actual cost exceeded the threshold.
|