@mastra/mcp-docs-server 1.2.2-alpha.2 → 1.2.2-alpha.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.docs/docs/agents/background-tasks.md +5 -5
- package/.docs/docs/evals/gates-and-verdicts.md +140 -0
- package/.docs/docs/evals/overview.md +1 -0
- package/.docs/models/gateways/netlify.md +1 -3
- package/.docs/models/index.md +1 -1
- package/.docs/models/providers/baseten.md +1 -1
- package/.docs/models/providers/huggingface.md +4 -1
- package/.docs/models/providers/llmgateway.md +184 -199
- package/.docs/reference/agents/channels.md +4 -0
- package/.docs/reference/ai-sdk/workflow-snapshot-to-stream.md +51 -0
- package/.docs/reference/cli/mastra.md +8 -0
- package/.docs/reference/evals/run-evals.md +71 -1
- package/.docs/reference/index.md +1 -0
- package/CHANGELOG.md +14 -0
- package/package.json +4 -4
|
@@ -49,7 +49,7 @@ Once a tool has opted in, the LLM can optionally include a `_background` field i
|
|
|
49
49
|
|
|
50
50
|
### Tool-level
|
|
51
51
|
|
|
52
|
-
Set `
|
|
52
|
+
Set `background.enabled: true` on the tool definition. Tools opted in at this layer run in the background whenever called by an agent that has the manager enabled.
|
|
53
53
|
|
|
54
54
|
```typescript
|
|
55
55
|
import { createTool } from '@mastra/core/tools'
|
|
@@ -59,7 +59,7 @@ export const researchTool = createTool({
|
|
|
59
59
|
id: 'research',
|
|
60
60
|
description: 'Run a long research job',
|
|
61
61
|
inputSchema: z.object({ topic: z.string() }),
|
|
62
|
-
|
|
62
|
+
background: {
|
|
63
63
|
enabled: true,
|
|
64
64
|
timeoutMs: 600_000,
|
|
65
65
|
maxRetries: 1,
|
|
@@ -111,7 +111,7 @@ The `_background` override is a _modifier_ on tools the developer has already op
|
|
|
111
111
|
When a tool call is dispatched, the resolved background config is computed in this priority order:
|
|
112
112
|
|
|
113
113
|
1. Agent-level `backgroundTasks.tools` entry for the tool.
|
|
114
|
-
2. Tool-level `
|
|
114
|
+
2. Tool-level `background` config.
|
|
115
115
|
3. LLM `_background.enabled` override (only used to enable background dispatch when the tool was opted in at one of the layers above).
|
|
116
116
|
4. Manager defaults (`defaultTimeoutMs`, `defaultRetries`).
|
|
117
117
|
|
|
@@ -200,7 +200,7 @@ const stream = await supervisor.stream('Research AI in education and write an ar
|
|
|
200
200
|
|
|
201
201
|
### Inheriting from the subagent
|
|
202
202
|
|
|
203
|
-
If a subagent isn't listed under the supervisor's `backgroundTasks.tools` but has background-eligible tools of its own (either via tool-level `
|
|
203
|
+
If a subagent isn't listed under the supervisor's `backgroundTasks.tools` but has background-eligible tools of its own (either via tool-level `background.enabled: true` or its own `backgroundTasks.tools` entry) the framework still dispatches the entire subagent invocation as a background task. The supervisor inherits the subagent's intent: the subagent itself becomes the background task, and its inner tools run in the foreground inside the subagent's loop.
|
|
204
204
|
|
|
205
205
|
The background config used for the inherited dispatch (for example `waitTimeoutMs`) is derived from the subagent's own `backgroundTasks` config.
|
|
206
206
|
|
|
@@ -285,7 +285,7 @@ The manager keeps tool executors in process memory. If the process restarts whil
|
|
|
285
285
|
|
|
286
286
|
Each layer can register terminal-state callbacks. They don't replace one another, and success/failure hooks fire for their respective outcomes:
|
|
287
287
|
|
|
288
|
-
- Tool-level `
|
|
288
|
+
- Tool-level `background.onComplete` / `onFailed`: scoped to one tool.
|
|
289
289
|
- Agent-level `backgroundTasks.onTaskComplete` / `onTaskFailed`: scoped to all tasks dispatched by this agent.
|
|
290
290
|
- Manager-level `onTaskComplete` / `onTaskFailed`: scoped globally.
|
|
291
291
|
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
# Gates and verdicts
|
|
2
|
+
|
|
3
|
+
Gates and verdicts add severity semantics to `runEvals`. Gates are scorers that must score 1.0 — hard requirements that block a run. Thresholds are minimum acceptable scores on tracked metrics. The verdict summarizes the outcome as `passed`, `scored`, or `failed`.
|
|
4
|
+
|
|
5
|
+
## When to use gates and verdicts
|
|
6
|
+
|
|
7
|
+
- Enforce hard requirements in CI (e.g., "agent must call the right tool")
|
|
8
|
+
- Track quality metrics with minimum thresholds (e.g., "faithfulness above 0.7")
|
|
9
|
+
- Get a single verdict signal (`passed`, `scored`, or `failed`) from an eval run without writing custom assertion logic
|
|
10
|
+
- Separate "must pass" gates from "nice to have" tracked metrics
|
|
11
|
+
|
|
12
|
+
## Quickstart
|
|
13
|
+
|
|
14
|
+
```typescript
|
|
15
|
+
import { runEvals } from '@mastra/core/evals'
|
|
16
|
+
import { checks } from '@mastra/evals/checks'
|
|
17
|
+
import { weatherAgent } from '../agents'
|
|
18
|
+
import { faithfulnessScorer } from '../scorers'
|
|
19
|
+
|
|
20
|
+
const result = await runEvals({
|
|
21
|
+
data: [{ input: 'What is the weather in Brooklyn?' }],
|
|
22
|
+
target: weatherAgent,
|
|
23
|
+
|
|
24
|
+
// Gates: must all score 1.0 or the run fails
|
|
25
|
+
gates: [checks.calledTool('get_weather'), checks.noToolErrors()],
|
|
26
|
+
|
|
27
|
+
// Scorers: tracked with optional thresholds
|
|
28
|
+
scorers: [
|
|
29
|
+
{ scorer: faithfulnessScorer, threshold: 0.7 },
|
|
30
|
+
checks.includes('Brooklyn'), // no threshold = tracked only
|
|
31
|
+
],
|
|
32
|
+
})
|
|
33
|
+
|
|
34
|
+
console.log(result.verdict) // 'passed' | 'scored' | 'failed'
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
## How verdicts work
|
|
38
|
+
|
|
39
|
+
The verdict is computed from gates and thresholds after all data items are processed:
|
|
40
|
+
|
|
41
|
+
- `failed`: At least one gate averaged below 1.0 across data items
|
|
42
|
+
- `scored`: All gates passed, but at least one threshold scorer missed its threshold
|
|
43
|
+
- `passed`: All gates scored 1.0 and all thresholds were met
|
|
44
|
+
|
|
45
|
+
When no gates or threshold-bearing scorers are provided, the verdict field is omitted and `runEvals` behaves exactly as before.
|
|
46
|
+
|
|
47
|
+
## Gates
|
|
48
|
+
|
|
49
|
+
Gates are scorers passed via the `gates` field. They run before regular scorers on each data item. A gate must average a score of 1.0 across all data items to pass.
|
|
50
|
+
|
|
51
|
+
```typescript
|
|
52
|
+
import { runEvals } from '@mastra/core/evals'
|
|
53
|
+
import { checks } from '@mastra/evals/checks'
|
|
54
|
+
|
|
55
|
+
const result = await runEvals({
|
|
56
|
+
data: [{ input: 'What is the weather?' }],
|
|
57
|
+
target: weatherAgent,
|
|
58
|
+
gates: [checks.calledTool('get_weather')],
|
|
59
|
+
scorers: [qualityScorer],
|
|
60
|
+
})
|
|
61
|
+
|
|
62
|
+
// result.gateResults: [{ id: 'check-called-tool', passed: true, score: 1 }]
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
Any scorer works as a gate. Quick Checks are a natural fit because they return binary 1/0 scores.
|
|
66
|
+
|
|
67
|
+
> **Note:** Visit [runEvals() reference](https://mastra.ai/reference/evals/run-evals) for the full parameter and return type documentation.
|
|
68
|
+
|
|
69
|
+
## Thresholds
|
|
70
|
+
|
|
71
|
+
Wrap a scorer in `{ scorer, threshold }` to set pass/fail bounds. The threshold is compared against the scorer's average score across all data items.
|
|
72
|
+
|
|
73
|
+
A `threshold` can be:
|
|
74
|
+
|
|
75
|
+
- **A number** — implies minimum (score at or above passes): `{ scorer, threshold: 0.7 }`
|
|
76
|
+
- **An object with `min` and/or `max`** — for range-based checks: `{ scorer, threshold: { max: 0.3 } }`
|
|
77
|
+
|
|
78
|
+
Use `max` for scorers where a high score is bad (e.g., hallucination, toxicity). Use `{ min, max }` when the score should fall within a specific band.
|
|
79
|
+
|
|
80
|
+
```typescript
|
|
81
|
+
import { runEvals } from '@mastra/core/evals'
|
|
82
|
+
|
|
83
|
+
const result = await runEvals({
|
|
84
|
+
data: [{ input: 'Explain quantum computing' }],
|
|
85
|
+
target: myAgent,
|
|
86
|
+
scorers: [
|
|
87
|
+
{ scorer: faithfulnessScorer, threshold: 0.7 }, // min threshold (number shorthand)
|
|
88
|
+
{ scorer: hallucinationScorer, threshold: { max: 0.3 } }, // max threshold — high score = bad
|
|
89
|
+
{ scorer: verbosityScorer, threshold: { min: 0.3, max: 0.8 } }, // range threshold
|
|
90
|
+
toneScorer, // bare scorer, no threshold — tracked only
|
|
91
|
+
],
|
|
92
|
+
})
|
|
93
|
+
|
|
94
|
+
// result.thresholdResults:
|
|
95
|
+
// [
|
|
96
|
+
// { id: 'faithfulness', passed: true, averageScore: 0.85, threshold: 0.7 },
|
|
97
|
+
// { id: 'hallucination', passed: true, averageScore: 0.1, threshold: { max: 0.3 } },
|
|
98
|
+
// { id: 'verbosity', passed: false, averageScore: 0.9, threshold: { min: 0.3, max: 0.8 } },
|
|
99
|
+
// ]
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
A bare scorer (no threshold) still appears in `result.scores` but does not affect the verdict.
|
|
103
|
+
|
|
104
|
+
## Using verdicts in CI
|
|
105
|
+
|
|
106
|
+
The verdict gives a single signal for CI pipelines:
|
|
107
|
+
|
|
108
|
+
```typescript
|
|
109
|
+
import { runEvals } from '@mastra/core/evals'
|
|
110
|
+
import { checks } from '@mastra/evals/checks'
|
|
111
|
+
|
|
112
|
+
const result = await runEvals({
|
|
113
|
+
data: testDataset,
|
|
114
|
+
target: myAgent,
|
|
115
|
+
gates: [checks.calledTool('search'), checks.noToolErrors()],
|
|
116
|
+
scorers: [{ scorer: faithfulnessScorer, threshold: 0.7 }],
|
|
117
|
+
})
|
|
118
|
+
|
|
119
|
+
if (result.verdict === 'failed') {
|
|
120
|
+
console.error(
|
|
121
|
+
'Gate failures:',
|
|
122
|
+
result.gateResults?.filter(g => !g.passed),
|
|
123
|
+
)
|
|
124
|
+
process.exit(1)
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
if (result.verdict === 'scored') {
|
|
128
|
+
console.warn(
|
|
129
|
+
'Threshold misses:',
|
|
130
|
+
result.thresholdResults?.filter(t => !t.passed),
|
|
131
|
+
)
|
|
132
|
+
}
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
## Related
|
|
136
|
+
|
|
137
|
+
- [Quick Checks](https://mastra.ai/docs/evals/quick-checks): Zero-LLM micro-scorers that work well as gates
|
|
138
|
+
- [runEvals() reference](https://mastra.ai/reference/evals/run-evals): Full API documentation
|
|
139
|
+
- [Built-in scorers](https://mastra.ai/docs/evals/built-in-scorers): LLM-based and code-based scorers
|
|
140
|
+
- [Running evals in CI](https://mastra.ai/docs/evals/running-in-ci): CI integration patterns
|
|
@@ -136,6 +136,7 @@ Once registered, you can score traces interactively within Studio under the **Ob
|
|
|
136
136
|
## Next steps
|
|
137
137
|
|
|
138
138
|
- Use [Quick Checks](https://mastra.ai/docs/evals/quick-checks) for fast, deterministic assertions on text and tool usage
|
|
139
|
+
- Add [gates and verdicts](https://mastra.ai/docs/evals/gates-and-verdicts) to enforce hard requirements and quality thresholds
|
|
139
140
|
- Learn how to create your own scorers in the [Creating Custom Scorers](https://mastra.ai/docs/evals/custom-scorers) guide
|
|
140
141
|
- Explore built-in scorers in the [Built-in Scorers](https://mastra.ai/docs/evals/built-in-scorers) section
|
|
141
142
|
- Test scorers with [Studio](https://mastra.ai/docs/studio/overview)
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Netlify
|
|
2
2
|
|
|
3
|
-
Netlify AI Gateway provides unified access to multiple providers with built-in caching and observability. Access
|
|
3
|
+
Netlify AI Gateway provides unified access to multiple providers with built-in caching and observability. Access 64 models through Mastra's model router.
|
|
4
4
|
|
|
5
5
|
Learn more in the [Netlify documentation](https://docs.netlify.com/build/ai-gateway/overview/).
|
|
6
6
|
|
|
@@ -52,9 +52,7 @@ ANTHROPIC_API_KEY=ant-...
|
|
|
52
52
|
| `gemini/gemini-2.5-pro` |
|
|
53
53
|
| `gemini/gemini-3-flash-preview` |
|
|
54
54
|
| `gemini/gemini-3-pro-image` |
|
|
55
|
-
| `gemini/gemini-3-pro-image-preview` |
|
|
56
55
|
| `gemini/gemini-3.1-flash-image` |
|
|
57
|
-
| `gemini/gemini-3.1-flash-image-preview` |
|
|
58
56
|
| `gemini/gemini-3.1-flash-lite` |
|
|
59
57
|
| `gemini/gemini-3.1-pro-preview` |
|
|
60
58
|
| `gemini/gemini-3.1-pro-preview-customtools` |
|
package/.docs/models/index.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Model Providers
|
|
2
2
|
|
|
3
|
-
Mastra provides a unified interface for working with LLMs across multiple providers, giving you access to
|
|
3
|
+
Mastra provides a unified interface for working with LLMs across multiple providers, giving you access to 4536 models from 133 providers through a single API.
|
|
4
4
|
|
|
5
5
|
## Features
|
|
6
6
|
|
|
@@ -44,7 +44,7 @@ for await (const chunk of stream) {
|
|
|
44
44
|
| `baseten/zai-org/GLM-4.7` | 200K | | | | | | $0.60 | $2 |
|
|
45
45
|
| `baseten/zai-org/GLM-5` | 203K | | | | | | $0.95 | $3 |
|
|
46
46
|
| `baseten/zai-org/GLM-5.1` | 203K | | | | | | $1 | $4 |
|
|
47
|
-
| `baseten/zai-org/GLM-5.2` |
|
|
47
|
+
| `baseten/zai-org/GLM-5.2` | 203K | | | | | | $1 | $4 |
|
|
48
48
|
|
|
49
49
|
## Advanced configuration
|
|
50
50
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Hugging Face
|
|
2
2
|
|
|
3
|
-
Access
|
|
3
|
+
Access 49 Hugging Face models through Mastra's model router. Authentication is handled automatically using the `HF_TOKEN` environment variable.
|
|
4
4
|
|
|
5
5
|
Learn more in the [Hugging Face documentation](https://huggingface.co).
|
|
6
6
|
|
|
@@ -68,9 +68,12 @@ for await (const chunk of stream) {
|
|
|
68
68
|
| `huggingface/Qwen/Qwen3.5-35B-A3B` | 262K | | | | | | $0.25 | $2 |
|
|
69
69
|
| `huggingface/Qwen/Qwen3.5-397B-A17B` | 262K | | | | | | $0.60 | $4 |
|
|
70
70
|
| `huggingface/Qwen/Qwen3.5-9B` | 262K | | | | | | $0.17 | $0.25 |
|
|
71
|
+
| `huggingface/Qwen/Qwen3.6-27B` | 262K | | | | | | $0.47 | $3 |
|
|
71
72
|
| `huggingface/Qwen/Qwen3.6-35B-A3B` | 262K | | | | | | $0.15 | $0.95 |
|
|
72
73
|
| `huggingface/stepfun-ai/Step-3.5-Flash` | 262K | | | | | | $0.10 | $0.30 |
|
|
74
|
+
| `huggingface/stepfun-ai/Step-3.7-Flash` | 262K | | | | | | $0.20 | $1 |
|
|
73
75
|
| `huggingface/XiaomiMiMo/MiMo-V2-Flash` | 262K | | | | | | $0.10 | $0.30 |
|
|
76
|
+
| `huggingface/XiaomiMiMo/MiMo-V2.5-Pro` | 1.0M | | | | | | $1 | $3 |
|
|
74
77
|
| `huggingface/zai-org/GLM-4.5` | 131K | | | | | | $0.60 | $2 |
|
|
75
78
|
| `huggingface/zai-org/GLM-4.5-Air` | 131K | | | | | | $0.13 | $0.85 |
|
|
76
79
|
| `huggingface/zai-org/GLM-4.5V` | 66K | | | | | | $0.60 | $2 |
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# LLM Gateway
|
|
2
2
|
|
|
3
|
-
Access
|
|
3
|
+
Access 181 LLM Gateway models through Mastra's model router. Authentication is handled automatically using the `LLMGATEWAY_API_KEY` environment variable.
|
|
4
4
|
|
|
5
5
|
Learn more in the [LLM Gateway documentation](https://llmgateway.io/docs).
|
|
6
6
|
|
|
@@ -32,204 +32,189 @@ for await (const chunk of stream) {
|
|
|
32
32
|
|
|
33
33
|
## Models
|
|
34
34
|
|
|
35
|
-
| Model
|
|
36
|
-
|
|
|
37
|
-
| `llmgateway/auto`
|
|
38
|
-
| `llmgateway/claude-3-
|
|
39
|
-
| `llmgateway/claude-3-7-sonnet`
|
|
40
|
-
| `llmgateway/claude-3-
|
|
41
|
-
| `llmgateway/claude-
|
|
42
|
-
| `llmgateway/claude-haiku-4-5`
|
|
43
|
-
| `llmgateway/claude-
|
|
44
|
-
| `llmgateway/claude-opus-4-
|
|
45
|
-
| `llmgateway/claude-opus-4-
|
|
46
|
-
| `llmgateway/claude-opus-4-
|
|
47
|
-
| `llmgateway/claude-opus-4-
|
|
48
|
-
| `llmgateway/claude-
|
|
49
|
-
| `llmgateway/claude-
|
|
50
|
-
| `llmgateway/claude-sonnet-4-
|
|
51
|
-
| `llmgateway/
|
|
52
|
-
| `llmgateway/
|
|
53
|
-
| `llmgateway/
|
|
54
|
-
| `llmgateway/
|
|
55
|
-
| `llmgateway/
|
|
56
|
-
| `llmgateway/deepseek-
|
|
57
|
-
| `llmgateway/
|
|
58
|
-
| `llmgateway/
|
|
59
|
-
| `llmgateway/
|
|
60
|
-
| `llmgateway/
|
|
61
|
-
| `llmgateway/gemini-2.
|
|
62
|
-
| `llmgateway/gemini-
|
|
63
|
-
| `llmgateway/gemini-
|
|
64
|
-
| `llmgateway/gemini-
|
|
65
|
-
| `llmgateway/gemini-
|
|
66
|
-
| `llmgateway/gemini-
|
|
67
|
-
| `llmgateway/
|
|
68
|
-
| `llmgateway/
|
|
69
|
-
| `llmgateway/
|
|
70
|
-
| `llmgateway/
|
|
71
|
-
| `llmgateway/
|
|
72
|
-
| `llmgateway/
|
|
73
|
-
| `llmgateway/
|
|
74
|
-
| `llmgateway/
|
|
75
|
-
| `llmgateway/
|
|
76
|
-
| `llmgateway/
|
|
77
|
-
| `llmgateway/glm-4
|
|
78
|
-
| `llmgateway/glm-4.
|
|
79
|
-
| `llmgateway/glm-4.
|
|
80
|
-
| `llmgateway/glm-4.
|
|
81
|
-
| `llmgateway/glm-4.
|
|
82
|
-
| `llmgateway/glm-4.
|
|
83
|
-
| `llmgateway/glm-4.
|
|
84
|
-
| `llmgateway/glm-
|
|
85
|
-
| `llmgateway/glm-
|
|
86
|
-
| `llmgateway/glm-
|
|
87
|
-
| `llmgateway/
|
|
88
|
-
| `llmgateway/
|
|
89
|
-
| `llmgateway/
|
|
90
|
-
| `llmgateway/
|
|
91
|
-
| `llmgateway/
|
|
92
|
-
| `llmgateway/
|
|
93
|
-
| `llmgateway/
|
|
94
|
-
| `llmgateway/gpt-
|
|
95
|
-
| `llmgateway/gpt-
|
|
96
|
-
| `llmgateway/gpt-
|
|
97
|
-
| `llmgateway/gpt-
|
|
98
|
-
| `llmgateway/gpt-
|
|
99
|
-
| `llmgateway/gpt-
|
|
100
|
-
| `llmgateway/gpt-
|
|
101
|
-
| `llmgateway/gpt-
|
|
102
|
-
| `llmgateway/gpt-
|
|
103
|
-
| `llmgateway/gpt-
|
|
104
|
-
| `llmgateway/gpt-5`
|
|
105
|
-
| `llmgateway/gpt-5
|
|
106
|
-
| `llmgateway/gpt-5-
|
|
107
|
-
| `llmgateway/gpt-5-
|
|
108
|
-
| `llmgateway/gpt-5-pro`
|
|
109
|
-
| `llmgateway/gpt-5.
|
|
110
|
-
| `llmgateway/gpt-5.
|
|
111
|
-
| `llmgateway/gpt-5.
|
|
112
|
-
| `llmgateway/gpt-5.
|
|
113
|
-
| `llmgateway/gpt-5.
|
|
114
|
-
| `llmgateway/gpt-5.
|
|
115
|
-
| `llmgateway/gpt-5.
|
|
116
|
-
| `llmgateway/gpt-5.
|
|
117
|
-
| `llmgateway/gpt-
|
|
118
|
-
| `llmgateway/gpt-
|
|
119
|
-
| `llmgateway/
|
|
120
|
-
| `llmgateway/
|
|
121
|
-
| `llmgateway/
|
|
122
|
-
| `llmgateway/
|
|
123
|
-
| `llmgateway/
|
|
124
|
-
| `llmgateway/
|
|
125
|
-
| `llmgateway/
|
|
126
|
-
| `llmgateway/grok-4-
|
|
127
|
-
| `llmgateway/grok-
|
|
128
|
-
| `llmgateway/
|
|
129
|
-
| `llmgateway/
|
|
130
|
-
| `llmgateway/
|
|
131
|
-
| `llmgateway/
|
|
132
|
-
| `llmgateway/
|
|
133
|
-
| `llmgateway/
|
|
134
|
-
| `llmgateway/
|
|
135
|
-
| `llmgateway/
|
|
136
|
-
| `llmgateway/
|
|
137
|
-
| `llmgateway/
|
|
138
|
-
| `llmgateway/
|
|
139
|
-
| `llmgateway/
|
|
140
|
-
| `llmgateway/
|
|
141
|
-
| `llmgateway/
|
|
142
|
-
| `llmgateway/
|
|
143
|
-
| `llmgateway/llama-
|
|
144
|
-
| `llmgateway/
|
|
145
|
-
| `llmgateway/
|
|
146
|
-
| `llmgateway/
|
|
147
|
-
| `llmgateway/
|
|
148
|
-
| `llmgateway/
|
|
149
|
-
| `llmgateway/
|
|
150
|
-
| `llmgateway/
|
|
151
|
-
| `llmgateway/
|
|
152
|
-
| `llmgateway/
|
|
153
|
-
| `llmgateway/
|
|
154
|
-
| `llmgateway/
|
|
155
|
-
| `llmgateway/
|
|
156
|
-
| `llmgateway/
|
|
157
|
-
| `llmgateway/
|
|
158
|
-
| `llmgateway/
|
|
159
|
-
| `llmgateway/
|
|
160
|
-
| `llmgateway/
|
|
161
|
-
| `llmgateway/
|
|
162
|
-
| `llmgateway/
|
|
163
|
-
| `llmgateway/
|
|
164
|
-
| `llmgateway/
|
|
165
|
-
| `llmgateway/
|
|
166
|
-
| `llmgateway/
|
|
167
|
-
| `llmgateway/
|
|
168
|
-
| `llmgateway/
|
|
169
|
-
| `llmgateway/
|
|
170
|
-
| `llmgateway/
|
|
171
|
-
| `llmgateway/
|
|
172
|
-
| `llmgateway/
|
|
173
|
-
| `llmgateway/
|
|
174
|
-
| `llmgateway/
|
|
175
|
-
| `llmgateway/
|
|
176
|
-
| `llmgateway/
|
|
177
|
-
| `llmgateway/
|
|
178
|
-
| `llmgateway/
|
|
179
|
-
| `llmgateway/
|
|
180
|
-
| `llmgateway/
|
|
181
|
-
| `llmgateway/
|
|
182
|
-
| `llmgateway/
|
|
183
|
-
| `llmgateway/
|
|
184
|
-
| `llmgateway/
|
|
185
|
-
| `llmgateway/
|
|
186
|
-
| `llmgateway/
|
|
187
|
-
| `llmgateway/
|
|
188
|
-
| `llmgateway/
|
|
189
|
-
| `llmgateway/
|
|
190
|
-
| `llmgateway/
|
|
191
|
-
| `llmgateway/
|
|
192
|
-
| `llmgateway/
|
|
193
|
-
| `llmgateway/qwen3-
|
|
194
|
-
| `llmgateway/qwen3-
|
|
195
|
-
| `llmgateway/qwen3-
|
|
196
|
-
| `llmgateway/qwen3-
|
|
197
|
-
| `llmgateway/qwen3-
|
|
198
|
-
| `llmgateway/qwen3-30b-a3b-
|
|
199
|
-
| `llmgateway/qwen3-
|
|
200
|
-
| `llmgateway/qwen3-
|
|
201
|
-
| `llmgateway/qwen3-
|
|
202
|
-
| `llmgateway/qwen3-
|
|
203
|
-
| `llmgateway/qwen3-
|
|
204
|
-
| `llmgateway/qwen3-
|
|
205
|
-
| `llmgateway/qwen3-
|
|
206
|
-
| `llmgateway/qwen3-
|
|
207
|
-
| `llmgateway/qwen3-max`
|
|
208
|
-
| `llmgateway/qwen3-
|
|
209
|
-
| `llmgateway/
|
|
210
|
-
| `llmgateway/
|
|
211
|
-
| `llmgateway/
|
|
212
|
-
| `llmgateway/
|
|
213
|
-
| `llmgateway/
|
|
214
|
-
| `llmgateway/
|
|
215
|
-
| `llmgateway/
|
|
216
|
-
| `llmgateway/
|
|
217
|
-
| `llmgateway/
|
|
218
|
-
| `llmgateway/qwen3.5-9b` | 262K | | | | | | $0.10 | $0.15 |
|
|
219
|
-
| `llmgateway/qwen3.6-35b-a3b` | 262K | | | | | | $0.25 | $1 |
|
|
220
|
-
| `llmgateway/qwen3.6-max-preview` | 262K | | | | | | $1 | $8 |
|
|
221
|
-
| `llmgateway/qwen3.6-plus` | 1.0M | | | | | | $0.50 | $3 |
|
|
222
|
-
| `llmgateway/qwen3.7-max` | 1.0M | | | | | | $3 | $8 |
|
|
223
|
-
| `llmgateway/qwen3.7-plus` | 1.0M | | | | | | $0.40 | $2 |
|
|
224
|
-
| `llmgateway/qwen35-397b-a17b` | 262K | | | | | | $0.60 | $4 |
|
|
225
|
-
| `llmgateway/qwq-plus` | 131K | | | | | | $0.80 | $2 |
|
|
226
|
-
| `llmgateway/seed-1-6-250615` | 256K | | | | | | $0.25 | $2 |
|
|
227
|
-
| `llmgateway/seed-1-6-250915` | 256K | | | | | | $0.25 | $2 |
|
|
228
|
-
| `llmgateway/seed-1-6-flash-250715` | 256K | | | | | | $0.07 | $0.30 |
|
|
229
|
-
| `llmgateway/seed-1-8-251228` | 256K | | | | | | $0.25 | $2 |
|
|
230
|
-
| `llmgateway/sonar` | 128K | | | | | | $1 | $1 |
|
|
231
|
-
| `llmgateway/sonar-pro` | 200K | | | | | | $3 | $15 |
|
|
232
|
-
| `llmgateway/sonar-reasoning-pro` | 128K | | | | | | $2 | $8 |
|
|
35
|
+
| Model | Context | Tools | Reasoning | Image | Audio | Video | Input $/1M | Output $/1M |
|
|
36
|
+
| -------------------------------------------------- | ------- | ----- | --------- | ----- | ----- | ----- | ---------- | ----------- |
|
|
37
|
+
| `llmgateway/auto` | 128K | | | | | | — | — |
|
|
38
|
+
| `llmgateway/claude-3-7-sonnet` | 200K | | | | | | $3 | $15 |
|
|
39
|
+
| `llmgateway/claude-3-7-sonnet-20250219` | 200K | | | | | | $3 | $15 |
|
|
40
|
+
| `llmgateway/claude-3-opus` | 200K | | | | | | $15 | $75 |
|
|
41
|
+
| `llmgateway/claude-haiku-4-5` | 200K | | | | | | $1 | $5 |
|
|
42
|
+
| `llmgateway/claude-haiku-4-5-20251001` | 200K | | | | | | $1 | $5 |
|
|
43
|
+
| `llmgateway/claude-opus-4-1-20250805` | 200K | | | | | | $15 | $75 |
|
|
44
|
+
| `llmgateway/claude-opus-4-5-20251101` | 200K | | | | | | $5 | $25 |
|
|
45
|
+
| `llmgateway/claude-opus-4-6` | 1.0M | | | | | | $5 | $25 |
|
|
46
|
+
| `llmgateway/claude-opus-4-7` | 1.0M | | | | | | $5 | $25 |
|
|
47
|
+
| `llmgateway/claude-opus-4-8` | 1.0M | | | | | | $5 | $25 |
|
|
48
|
+
| `llmgateway/claude-sonnet-4-5` | 200K | | | | | | $3 | $15 |
|
|
49
|
+
| `llmgateway/claude-sonnet-4-5-20250929` | 200K | | | | | | $3 | $15 |
|
|
50
|
+
| `llmgateway/claude-sonnet-4-6` | 1.0M | | | | | | $3 | $15 |
|
|
51
|
+
| `llmgateway/codestral-2508` | 256K | | | | | | $0.30 | $0.90 |
|
|
52
|
+
| `llmgateway/custom` | 128K | | | | | | — | — |
|
|
53
|
+
| `llmgateway/deepseek-v3.1` | 128K | | | | | | $0.56 | $2 |
|
|
54
|
+
| `llmgateway/deepseek-v3.2` | 164K | | | | | | $0.28 | $0.42 |
|
|
55
|
+
| `llmgateway/deepseek-v4-flash` | 1.1M | | | | | | $0.14 | $0.28 |
|
|
56
|
+
| `llmgateway/deepseek-v4-pro` | 1.1M | | | | | | $0.43 | $0.87 |
|
|
57
|
+
| `llmgateway/fugu-ultra` | 1.0M | | | | | | $5 | $30 |
|
|
58
|
+
| `llmgateway/gemini-2.5-flash` | 1.0M | | | | | | $0.30 | $3 |
|
|
59
|
+
| `llmgateway/gemini-2.5-flash-lite` | 1.0M | | | | | | $0.10 | $0.40 |
|
|
60
|
+
| `llmgateway/gemini-2.5-flash-lite-preview-09-2025` | 1.0M | | | | | | $0.10 | $0.40 |
|
|
61
|
+
| `llmgateway/gemini-2.5-pro` | 1.0M | | | | | | $1 | $10 |
|
|
62
|
+
| `llmgateway/gemini-3-flash-preview` | 1.0M | | | | | | $0.50 | $3 |
|
|
63
|
+
| `llmgateway/gemini-3.1-flash-lite` | 1.0M | | | | | | $0.25 | $2 |
|
|
64
|
+
| `llmgateway/gemini-3.1-pro-preview` | 1.0M | | | | | | $2 | $12 |
|
|
65
|
+
| `llmgateway/gemini-3.5-flash` | 1.0M | | | | | | $2 | $9 |
|
|
66
|
+
| `llmgateway/gemini-pro-latest` | 1.0M | | | | | | $2 | $12 |
|
|
67
|
+
| `llmgateway/gemma-4-26b-a4b-it` | 262K | | | | | | $0.07 | $0.34 |
|
|
68
|
+
| `llmgateway/gemma-4-31b-it` | 262K | | | | | | $0.13 | $0.38 |
|
|
69
|
+
| `llmgateway/glm-4-32b-0414-128k` | 128K | | | | | | $0.10 | $0.10 |
|
|
70
|
+
| `llmgateway/glm-4.5` | 131K | | | | | | $0.60 | $2 |
|
|
71
|
+
| `llmgateway/glm-4.5-air` | 131K | | | | | | $0.20 | $1 |
|
|
72
|
+
| `llmgateway/glm-4.5-airx` | 128K | | | | | | $1 | $5 |
|
|
73
|
+
| `llmgateway/glm-4.5-flash` | 128K | | | | | | — | — |
|
|
74
|
+
| `llmgateway/glm-4.5-x` | 128K | | | | | | $2 | $9 |
|
|
75
|
+
| `llmgateway/glm-4.5v` | 128K | | | | | | $0.60 | $2 |
|
|
76
|
+
| `llmgateway/glm-4.6` | 205K | | | | | | $0.60 | $2 |
|
|
77
|
+
| `llmgateway/glm-4.6v` | 131K | | | | | | $0.30 | $0.90 |
|
|
78
|
+
| `llmgateway/glm-4.6v-flash` | 128K | | | | | | — | — |
|
|
79
|
+
| `llmgateway/glm-4.6v-flashx` | 128K | | | | | | $0.04 | $0.40 |
|
|
80
|
+
| `llmgateway/glm-4.7` | 205K | | | | | | $0.60 | $2 |
|
|
81
|
+
| `llmgateway/glm-4.7-flash` | 200K | | | | | | $0.06 | $0.40 |
|
|
82
|
+
| `llmgateway/glm-4.7-flash-free` | 200K | | | | | | — | — |
|
|
83
|
+
| `llmgateway/glm-4.7-flashx` | 200K | | | | | | $0.07 | $0.40 |
|
|
84
|
+
| `llmgateway/glm-5` | 203K | | | | | | $1 | $3 |
|
|
85
|
+
| `llmgateway/glm-5.1` | 205K | | | | | | $1 | $4 |
|
|
86
|
+
| `llmgateway/glm-5.2` | 1.0M | | | | | | $1 | $4 |
|
|
87
|
+
| `llmgateway/gpt-3.5-turbo` | 16K | | | | | | $0.50 | $2 |
|
|
88
|
+
| `llmgateway/gpt-4` | 8K | | | | | | $30 | $60 |
|
|
89
|
+
| `llmgateway/gpt-4-turbo` | 128K | | | | | | $10 | $30 |
|
|
90
|
+
| `llmgateway/gpt-4.1` | 1.0M | | | | | | $2 | $8 |
|
|
91
|
+
| `llmgateway/gpt-4.1-mini` | 1.0M | | | | | | $0.40 | $2 |
|
|
92
|
+
| `llmgateway/gpt-4.1-nano` | 1.0M | | | | | | $0.10 | $0.40 |
|
|
93
|
+
| `llmgateway/gpt-4o` | 128K | | | | | | $3 | $10 |
|
|
94
|
+
| `llmgateway/gpt-4o-mini` | 128K | | | | | | $0.15 | $0.60 |
|
|
95
|
+
| `llmgateway/gpt-4o-mini-search-preview` | 128K | | | | | | $0.15 | $0.60 |
|
|
96
|
+
| `llmgateway/gpt-4o-search-preview` | 128K | | | | | | $3 | $10 |
|
|
97
|
+
| `llmgateway/gpt-5` | 400K | | | | | | $1 | $10 |
|
|
98
|
+
| `llmgateway/gpt-5-chat-latest` | 400K | | | | | | $1 | $10 |
|
|
99
|
+
| `llmgateway/gpt-5-mini` | 400K | | | | | | $0.25 | $2 |
|
|
100
|
+
| `llmgateway/gpt-5-nano` | 400K | | | | | | $0.05 | $0.40 |
|
|
101
|
+
| `llmgateway/gpt-5-pro` | 400K | | | | | | $15 | $120 |
|
|
102
|
+
| `llmgateway/gpt-5.1` | 400K | | | | | | $1 | $10 |
|
|
103
|
+
| `llmgateway/gpt-5.1-codex` | 400K | | | | | | $1 | $10 |
|
|
104
|
+
| `llmgateway/gpt-5.1-codex-mini` | 400K | | | | | | $0.25 | $2 |
|
|
105
|
+
| `llmgateway/gpt-5.2` | 400K | | | | | | $2 | $14 |
|
|
106
|
+
| `llmgateway/gpt-5.2-chat-latest` | 128K | | | | | | $2 | $14 |
|
|
107
|
+
| `llmgateway/gpt-5.2-codex` | 400K | | | | | | $2 | $14 |
|
|
108
|
+
| `llmgateway/gpt-5.2-pro` | 400K | | | | | | $21 | $168 |
|
|
109
|
+
| `llmgateway/gpt-5.3-chat-latest` | 128K | | | | | | $2 | $14 |
|
|
110
|
+
| `llmgateway/gpt-5.3-codex` | 400K | | | | | | $2 | $14 |
|
|
111
|
+
| `llmgateway/gpt-5.4` | 1.1M | | | | | | $3 | $15 |
|
|
112
|
+
| `llmgateway/gpt-5.4-mini` | 400K | | | | | | $0.75 | $5 |
|
|
113
|
+
| `llmgateway/gpt-5.4-nano` | 400K | | | | | | $0.20 | $1 |
|
|
114
|
+
| `llmgateway/gpt-5.4-pro` | 1.1M | | | | | | $30 | $180 |
|
|
115
|
+
| `llmgateway/gpt-5.5` | 1.1M | | | | | | $5 | $30 |
|
|
116
|
+
| `llmgateway/gpt-5.5-pro` | 1.1M | | | | | | $30 | $180 |
|
|
117
|
+
| `llmgateway/gpt-oss-120b` | 131K | | | | | | $0.15 | $0.75 |
|
|
118
|
+
| `llmgateway/gpt-oss-20b` | 131K | | | | | | $0.10 | $0.50 |
|
|
119
|
+
| `llmgateway/grok-4` | 256K | | | | | | $3 | $15 |
|
|
120
|
+
| `llmgateway/grok-4-1-fast-non-reasoning` | 2.0M | | | | | | $0.20 | $0.50 |
|
|
121
|
+
| `llmgateway/grok-4-1-fast-reasoning` | 2.0M | | | | | | $0.20 | $0.50 |
|
|
122
|
+
| `llmgateway/grok-4-20-beta-0309-non-reasoning` | 2.0M | | | | | | $2 | $6 |
|
|
123
|
+
| `llmgateway/grok-4-20-beta-0309-reasoning` | 2.0M | | | | | | $2 | $6 |
|
|
124
|
+
| `llmgateway/grok-4-20-non-reasoning` | 2.0M | | | | | | $2 | $6 |
|
|
125
|
+
| `llmgateway/grok-4-20-reasoning` | 2.0M | | | | | | $2 | $6 |
|
|
126
|
+
| `llmgateway/grok-4-3` | 1.0M | | | | | | $1 | $3 |
|
|
127
|
+
| `llmgateway/grok-build-0-1` | 256K | | | | | | $1 | $2 |
|
|
128
|
+
| `llmgateway/kimi-k2` | 256K | | | | | | $1 | $3 |
|
|
129
|
+
| `llmgateway/kimi-k2-thinking` | 262K | | | | | | $0.60 | $3 |
|
|
130
|
+
| `llmgateway/kimi-k2-thinking-turbo` | 262K | | | | | | $1 | $8 |
|
|
131
|
+
| `llmgateway/kimi-k2.5` | 262K | | | | | | $0.60 | $3 |
|
|
132
|
+
| `llmgateway/kimi-k2.6` | 262K | | | | | | $0.95 | $4 |
|
|
133
|
+
| `llmgateway/kimi-k2.7-code` | 262K | | | | | | $0.95 | $4 |
|
|
134
|
+
| `llmgateway/kimi-k2.7-code-highspeed` | 262K | | | | | | $2 | $8 |
|
|
135
|
+
| `llmgateway/llama-3-70b-instruct` | 8K | | | | | | $0.51 | $0.74 |
|
|
136
|
+
| `llmgateway/llama-3-8b-instruct` | 8K | | | | | | $0.04 | $0.04 |
|
|
137
|
+
| `llmgateway/llama-3.1-70b-instruct` | 128K | | | | | | $0.72 | $0.72 |
|
|
138
|
+
| `llmgateway/llama-3.1-nemotron-ultra-253b` | 128K | | | | | | $0.60 | $2 |
|
|
139
|
+
| `llmgateway/llama-3.2-11b-instruct` | 128K | | | | | | $0.07 | $0.33 |
|
|
140
|
+
| `llmgateway/llama-3.2-3b-instruct` | 33K | | | | | | $0.03 | $0.05 |
|
|
141
|
+
| `llmgateway/llama-3.3-70b-instruct` | 131K | | | | | | $0.13 | $0.40 |
|
|
142
|
+
| `llmgateway/llama-4-maverick-17b-instruct` | 1.0M | | | | | | $0.24 | $0.97 |
|
|
143
|
+
| `llmgateway/llama-4-scout-17b-instruct` | 131K | | | | | | $0.17 | $0.66 |
|
|
144
|
+
| `llmgateway/mimo-v2-omni` | 256K | | | | | | $0.40 | $2 |
|
|
145
|
+
| `llmgateway/mimo-v2-pro` | 1.0M | | | | | | $1 | $3 |
|
|
146
|
+
| `llmgateway/mimo-v2.5` | 1.0M | | | | | | $0.14 | $0.28 |
|
|
147
|
+
| `llmgateway/mimo-v2.5-pro` | 1.0M | | | | | | $0.43 | $0.87 |
|
|
148
|
+
| `llmgateway/minimax-m2` | 197K | | | | | | $0.20 | $1 |
|
|
149
|
+
| `llmgateway/minimax-m2.1` | 205K | | | | | | $0.27 | $1 |
|
|
150
|
+
| `llmgateway/minimax-m2.1-lightning` | 197K | | | | | | $0.12 | $0.48 |
|
|
151
|
+
| `llmgateway/minimax-m2.5` | 229K | | | | | | $0.30 | $1 |
|
|
152
|
+
| `llmgateway/minimax-m2.5-highspeed` | 205K | | | | | | $0.60 | $2 |
|
|
153
|
+
| `llmgateway/minimax-m2.7` | 205K | | | | | | $0.30 | $1 |
|
|
154
|
+
| `llmgateway/minimax-m2.7-highspeed` | 205K | | | | | | $0.60 | $2 |
|
|
155
|
+
| `llmgateway/minimax-m3` | 512K | | | | | | $0.60 | $2 |
|
|
156
|
+
| `llmgateway/minimax-text-01` | 1.0M | | | | | | $0.20 | $1 |
|
|
157
|
+
| `llmgateway/ministral-14b-2512` | 262K | | | | | | $0.20 | $0.20 |
|
|
158
|
+
| `llmgateway/ministral-3b-2512` | 131K | | | | | | $0.10 | $0.10 |
|
|
159
|
+
| `llmgateway/ministral-8b-2512` | 262K | | | | | | $0.15 | $0.15 |
|
|
160
|
+
| `llmgateway/mistral-large-2512` | 262K | | | | | | $0.50 | $2 |
|
|
161
|
+
| `llmgateway/mistral-large-latest` | 128K | | | | | | $4 | $12 |
|
|
162
|
+
| `llmgateway/mistral-small-2506` | 128K | | | | | | $0.10 | $0.30 |
|
|
163
|
+
| `llmgateway/nemotron-3-ultra-550b` | 262K | | | | | | $0.50 | $3 |
|
|
164
|
+
| `llmgateway/o1` | 200K | | | | | | $15 | $60 |
|
|
165
|
+
| `llmgateway/o3` | 200K | | | | | | $2 | $8 |
|
|
166
|
+
| `llmgateway/o3-mini` | 200K | | | | | | $1 | $4 |
|
|
167
|
+
| `llmgateway/o4-mini` | 200K | | | | | | $1 | $4 |
|
|
168
|
+
| `llmgateway/pixtral-large-latest` | 128K | | | | | | $4 | $12 |
|
|
169
|
+
| `llmgateway/qwen-coder-plus` | 131K | | | | | | $0.50 | $1 |
|
|
170
|
+
| `llmgateway/qwen-flash` | 1.0M | | | | | | $0.05 | $0.40 |
|
|
171
|
+
| `llmgateway/qwen-max` | 33K | | | | | | $2 | $6 |
|
|
172
|
+
| `llmgateway/qwen-max-latest` | 33K | | | | | | $2 | $6 |
|
|
173
|
+
| `llmgateway/qwen-omni-turbo` | 33K | | | | | | $0.20 | $0.80 |
|
|
174
|
+
| `llmgateway/qwen-plus` | 131K | | | | | | $0.40 | $1 |
|
|
175
|
+
| `llmgateway/qwen-plus-latest` | 1.0M | | | | | | $0.40 | $1 |
|
|
176
|
+
| `llmgateway/qwen-turbo` | 1.0M | | | | | | $0.05 | $0.20 |
|
|
177
|
+
| `llmgateway/qwen-vl-max` | 131K | | | | | | $0.80 | $3 |
|
|
178
|
+
| `llmgateway/qwen-vl-plus` | 131K | | | | | | $0.21 | $0.64 |
|
|
179
|
+
| `llmgateway/qwen2-5-vl-32b-instruct` | 131K | | | | | | $1 | $4 |
|
|
180
|
+
| `llmgateway/qwen2-5-vl-72b-instruct` | 33K | | | | | | $0.13 | $0.40 |
|
|
181
|
+
| `llmgateway/qwen3-235b-a22b-fp8` | 41K | | | | | | $0.20 | $0.80 |
|
|
182
|
+
| `llmgateway/qwen3-235b-a22b-instruct-2507` | 262K | | | | | | $0.20 | $0.60 |
|
|
183
|
+
| `llmgateway/qwen3-235b-a22b-thinking-2507` | 262K | | | | | | $0.20 | $0.60 |
|
|
184
|
+
| `llmgateway/qwen3-30b-a3b-instruct-2507` | 262K | | | | | | $0.10 | $0.30 |
|
|
185
|
+
| `llmgateway/qwen3-32b` | 33K | | | | | | $0.10 | $0.30 |
|
|
186
|
+
| `llmgateway/qwen3-4b-fp8` | 128K | | | | | | $0.03 | $0.03 |
|
|
187
|
+
| `llmgateway/qwen3-coder-30b-a3b-instruct` | 262K | | | | | | $0.10 | $0.30 |
|
|
188
|
+
| `llmgateway/qwen3-coder-480b-a35b-instruct` | 262K | | | | | | $0.40 | $2 |
|
|
189
|
+
| `llmgateway/qwen3-coder-flash` | 1.0M | | | | | | $0.30 | $2 |
|
|
190
|
+
| `llmgateway/qwen3-coder-next` | 262K | | | | | | $0.11 | $0.68 |
|
|
191
|
+
| `llmgateway/qwen3-coder-plus` | 1.0M | | | | | | $6 | $60 |
|
|
192
|
+
| `llmgateway/qwen3-max` | 262K | | | | | | $3 | $15 |
|
|
193
|
+
| `llmgateway/qwen3-max-2026-01-23` | 262K | | | | | | $1 | $6 |
|
|
194
|
+
| `llmgateway/qwen3-next-80b-a3b-instruct` | 131K | | | | | | $0.50 | $2 |
|
|
195
|
+
| `llmgateway/qwen3-next-80b-a3b-thinking` | 131K | | | | | | $0.50 | $6 |
|
|
196
|
+
| `llmgateway/qwen3-vl-235b-a22b-instruct` | 131K | | | | | | $0.50 | $2 |
|
|
197
|
+
| `llmgateway/qwen3-vl-235b-a22b-thinking` | 131K | | | | | | $0.50 | $2 |
|
|
198
|
+
| `llmgateway/qwen3-vl-30b-a3b-instruct` | 131K | | | | | | $0.20 | $0.70 |
|
|
199
|
+
| `llmgateway/qwen3-vl-30b-a3b-thinking` | 131K | | | | | | $0.20 | $1 |
|
|
200
|
+
| `llmgateway/qwen3-vl-8b-instruct` | 131K | | | | | | $0.08 | $0.50 |
|
|
201
|
+
| `llmgateway/qwen3-vl-flash` | 262K | | | | | | $0.05 | $0.40 |
|
|
202
|
+
| `llmgateway/qwen3-vl-plus` | 262K | | | | | | $0.20 | $2 |
|
|
203
|
+
| `llmgateway/qwen3.5-9b` | 262K | | | | | | $0.10 | $0.15 |
|
|
204
|
+
| `llmgateway/qwen3.6-35b-a3b` | 262K | | | | | | $0.25 | $1 |
|
|
205
|
+
| `llmgateway/qwen3.6-max-preview` | 262K | | | | | | $1 | $8 |
|
|
206
|
+
| `llmgateway/qwen3.6-plus` | 262K | | | | | | $0.50 | $3 |
|
|
207
|
+
| `llmgateway/qwen3.7-max` | 1.0M | | | | | | $3 | $8 |
|
|
208
|
+
| `llmgateway/qwen3.7-plus` | 1.0M | | | | | | $0.40 | $2 |
|
|
209
|
+
| `llmgateway/qwen35-397b-a17b` | 262K | | | | | | $0.60 | $4 |
|
|
210
|
+
| `llmgateway/qwq-plus` | 131K | | | | | | $0.80 | $2 |
|
|
211
|
+
| `llmgateway/seed-1-6-250615` | 256K | | | | | | $0.25 | $2 |
|
|
212
|
+
| `llmgateway/seed-1-6-250915` | 256K | | | | | | $0.25 | $2 |
|
|
213
|
+
| `llmgateway/seed-1-6-flash-250715` | 256K | | | | | | $0.07 | $0.30 |
|
|
214
|
+
| `llmgateway/seed-1-8-251228` | 256K | | | | | | $0.25 | $2 |
|
|
215
|
+
| `llmgateway/sonar` | 130K | | | | | | $1 | $1 |
|
|
216
|
+
| `llmgateway/sonar-pro` | 200K | | | | | | $3 | $15 |
|
|
217
|
+
| `llmgateway/sonar-reasoning-pro` | 128K | | | | | | $2 | $8 |
|
|
233
218
|
|
|
234
219
|
## Advanced configuration
|
|
235
220
|
|
|
@@ -47,6 +47,10 @@ export const supportAgent = new Agent({
|
|
|
47
47
|
|
|
48
48
|
**resolveResourceId** (`(ctx: ResolveResourceIdContext) => string | Promise<string>`): Decide which \`resourceId\` owns resource-level memory for a channel thread, separately from who sent the message. Runs only when a new thread is created; reused threads keep their stored owner and never call the hook. Return \`ctx.defaultResourceId\` (\`${platform}:${message.author.userId}\`) to keep the built-in behavior.
|
|
49
49
|
|
|
50
|
+
**waitUntil** (`(promise: Promise<unknown>) => void`): Platform \`waitUntil\` function. Required on Vercel so background agent runs survive after the webhook returns 200. On Vercel pass \`waitUntil\` from \`@vercel/functions\`. Cloudflare Workers and Netlify Functions are detected automatically from the request context. AWS Lambda does not need \`waitUntil\` because it waits for the event loop to drain naturally.
|
|
51
|
+
|
|
52
|
+
**resolveWaitUntil** (`(c: Context) => ((promise: Promise<unknown>) => void) | undefined`): Resolver for runtimes where \`waitUntil\` lives on the Hono request context but is not covered by the built-in helper. Resolution order: bare \`waitUntil\` → \`resolveWaitUntil(c)\` → default (Cloudflare Workers, Netlify).
|
|
53
|
+
|
|
50
54
|
## Per-adapter options
|
|
51
55
|
|
|
52
56
|
Wrap an adapter in a `ChannelAdapterConfig` object to set per-adapter options:
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# workflowSnapshotToStream()
|
|
2
|
+
|
|
3
|
+
Converts a `WorkflowState` object (as returned by `workflow.getWorkflowRunById()`) into a `ReadableStream` of AI SDK UIMessage data parts. The stream contains the same `WorkflowDataPart` and `WorkflowStepDataPart` chunks that the live [`workflowRoute()`](https://mastra.ai/reference/ai-sdk/workflow-route) produces.
|
|
4
|
+
|
|
5
|
+
Use this when you want to display a completed or suspended workflow run using the same UI components that render live workflow streams.
|
|
6
|
+
|
|
7
|
+
## Usage example
|
|
8
|
+
|
|
9
|
+
Next.js App Router endpoint that replays a past workflow run as a stream:
|
|
10
|
+
|
|
11
|
+
```typescript
|
|
12
|
+
import { workflowSnapshotToStream } from '@mastra/ai-sdk'
|
|
13
|
+
import { createUIMessageStreamResponse } from 'ai'
|
|
14
|
+
import { mastra } from '@/src/mastra'
|
|
15
|
+
|
|
16
|
+
export async function GET(req: Request) {
|
|
17
|
+
const { searchParams } = new URL(req.url)
|
|
18
|
+
const runId = searchParams.get('runId')!
|
|
19
|
+
|
|
20
|
+
const workflowRun = await mastra.getWorkflow('myWorkflow').getWorkflowRunById(runId)
|
|
21
|
+
|
|
22
|
+
if (!workflowRun) {
|
|
23
|
+
return new Response('Not found', { status: 404 })
|
|
24
|
+
}
|
|
25
|
+
|
|
26
|
+
const stream = workflowSnapshotToStream(workflowRun)
|
|
27
|
+
return createUIMessageStreamResponse({ stream })
|
|
28
|
+
}
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
## Parameters
|
|
32
|
+
|
|
33
|
+
**workflowRun** (`WorkflowState`): The workflow run state object, as returned by \`getWorkflowRunById()\` or the workflow runs API. Contains \`runId\`, \`workflowName\`, \`status\`, and \`steps\`.
|
|
34
|
+
|
|
35
|
+
## Returns
|
|
36
|
+
|
|
37
|
+
`ReadableStream` — A stream of AI SDK UIMessage data parts containing:
|
|
38
|
+
|
|
39
|
+
- A `start` marker
|
|
40
|
+
- A `WorkflowDataPart` with the overall workflow status and all step summaries
|
|
41
|
+
- A `WorkflowStepDataPart` for each step with its full output
|
|
42
|
+
- A `finish` marker
|
|
43
|
+
|
|
44
|
+
Pass this stream to [`createUIMessageStreamResponse()`](https://ai-sdk.dev/docs/reference/ai-sdk-ui/create-ui-message-stream-response) or `createUIMessageStream()` from the AI SDK.
|
|
45
|
+
|
|
46
|
+
## Related
|
|
47
|
+
|
|
48
|
+
- [`workflowRoute()`](https://mastra.ai/reference/ai-sdk/workflow-route): Stream live workflow runs
|
|
49
|
+
- [`handleWorkflowStream()`](https://mastra.ai/reference/ai-sdk/handle-workflow-stream): Framework-agnostic handler for live workflow streaming
|
|
50
|
+
- [`toAISdkStream()`](https://mastra.ai/reference/ai-sdk/to-ai-sdk-stream): Convert live Mastra streams to AI SDK format
|
|
51
|
+
- [`toAISdkMessages()`](https://mastra.ai/reference/ai-sdk/to-ai-sdk-messages): Convert stored agent messages to AI SDK format
|
|
@@ -666,6 +666,14 @@ Target a specific Mastra server URL.
|
|
|
666
666
|
mastra api --url https://example.com agent list
|
|
667
667
|
```
|
|
668
668
|
|
|
669
|
+
#### `--server-api-prefix <prefix>`
|
|
670
|
+
|
|
671
|
+
Set the API route prefix of the target server. Defaults to `/api`. Use this when the server is mounted under a custom prefix (for example a `@mastra/fastify` `MastraServer` with `prefix: "/api/mastra-studio"`), the same way `mastra studio` accepts `--server-api-prefix`. You can also set the `MASTRA_API_PREFIX` environment variable instead of passing the flag.
|
|
672
|
+
|
|
673
|
+
```bash
|
|
674
|
+
mastra api --url https://example.com --server-api-prefix /api/mastra-studio agent list
|
|
675
|
+
```
|
|
676
|
+
|
|
669
677
|
#### `--header <"Key: Value">`
|
|
670
678
|
|
|
671
679
|
Send a custom HTTP header. Repeat the flag to send multiple headers.
|
|
@@ -29,13 +29,34 @@ console.log(`Average scores:`, result.scores)
|
|
|
29
29
|
console.log(`Processed ${result.summary.totalItems} items`)
|
|
30
30
|
```
|
|
31
31
|
|
|
32
|
+
### With gates and thresholds
|
|
33
|
+
|
|
34
|
+
```typescript
|
|
35
|
+
import { runEvals } from '@mastra/core/evals'
|
|
36
|
+
import { checks } from '@mastra/evals/checks'
|
|
37
|
+
import { faithfulnessScorer } from './scorers'
|
|
38
|
+
|
|
39
|
+
const result = await runEvals({
|
|
40
|
+
target: myAgent,
|
|
41
|
+
data: [{ input: 'What is the weather in Brooklyn?' }],
|
|
42
|
+
gates: [checks.calledTool('get_weather'), checks.noToolErrors()],
|
|
43
|
+
scorers: [{ scorer: faithfulnessScorer, threshold: 0.7 }, checks.includes('Brooklyn')],
|
|
44
|
+
})
|
|
45
|
+
|
|
46
|
+
result.verdict // 'passed' | 'scored' | 'failed'
|
|
47
|
+
result.gateResults // [{ id, passed, score }]
|
|
48
|
+
result.thresholdResults // [{ id, passed, averageScore, threshold }]
|
|
49
|
+
```
|
|
50
|
+
|
|
32
51
|
## Parameters
|
|
33
52
|
|
|
34
53
|
**target** (`Agent | Workflow`): The agent or workflow to evaluate.
|
|
35
54
|
|
|
36
55
|
**data** (`RunEvalsDataItem[]`): Array of test cases with input data and optional ground truth.
|
|
37
56
|
|
|
38
|
-
**scorers** (`
|
|
57
|
+
**scorers** (`ScorerEntry[] | AgentScorerConfig | WorkflowScorerConfig`): Scorers to use. Each entry is either a bare \`MastraScorer\` or \`{ scorer, threshold }\` for threshold tracking. An \`AgentScorerConfig\` object separates agent-level and trajectory scorers. A \`WorkflowScorerConfig\` object specifies scorers for the workflow, individual steps, and trajectory.
|
|
58
|
+
|
|
59
|
+
**gates** (`MastraScorer[]`): Scorers that must score 1.0 for the run to pass. If any gate averages below 1.0 across data items, the verdict is \`failed\`. Gates run before regular scorers on each data item.
|
|
39
60
|
|
|
40
61
|
**targetOptions** (`AgentExecutionOptions | WorkflowRunOptions`): Options forwarded to the target during execution. For agents: options passed to agent.generate() (e.g. maxSteps, modelSettings, instructions). For workflows: options passed to run.start() (e.g. perStep, outputOptions, initialState).
|
|
41
62
|
|
|
@@ -83,8 +104,55 @@ For workflows, use `WorkflowScorerConfig` to specify scorers at different levels
|
|
|
83
104
|
|
|
84
105
|
**summary.totalItems** (`number`): Total number of test cases processed.
|
|
85
106
|
|
|
107
|
+
**verdict** (`'passed' | 'scored' | 'failed'`): Present when \`gates\` or threshold-bearing scorers are provided. \`passed\` = all gates and thresholds met. \`scored\` = gates passed but a threshold was missed. \`failed\` = at least one gate did not score 1.0.
|
|
108
|
+
|
|
109
|
+
**gateResults** (`GateResult[]`): Per-gate results averaged across all data items. Each entry has \`id\`, \`passed\` (boolean), and \`score\` (0–1).
|
|
110
|
+
|
|
111
|
+
**thresholdResults** (`ThresholdResult[]`): Per-threshold-scorer results averaged across all data items. Each entry has \`id\`, \`passed\`, \`averageScore\`, and \`threshold\`.
|
|
112
|
+
|
|
113
|
+
## ScorerEntry
|
|
114
|
+
|
|
115
|
+
A scorer entry in the `scorers` array can be either a bare scorer or a scorer with a threshold:
|
|
116
|
+
|
|
117
|
+
**scorer** (`MastraScorer`): The scorer instance.
|
|
118
|
+
|
|
119
|
+
**threshold** (`number | { min?: number; max?: number }`): A number implies minimum threshold (score at or above passes). Use \`{ min, max }\` for range-based checks — e.g. \`{ max: 0.3 }\` for scorers like hallucination where a high score is bad. Both \`min\` and \`max\` must be between 0 and 1.
|
|
120
|
+
|
|
86
121
|
## Examples
|
|
87
122
|
|
|
123
|
+
### Gates and verdict
|
|
124
|
+
|
|
125
|
+
Use `gates` for hard pass/fail requirements and `{ scorer, threshold }` for tracked quality metrics:
|
|
126
|
+
|
|
127
|
+
```typescript
|
|
128
|
+
import { runEvals } from '@mastra/core/evals'
|
|
129
|
+
import { checks } from '@mastra/evals/checks'
|
|
130
|
+
|
|
131
|
+
const result = await runEvals({
|
|
132
|
+
target: weatherAgent,
|
|
133
|
+
data: [{ input: 'What is the weather in Brooklyn?' }],
|
|
134
|
+
gates: [checks.calledTool('get_weather'), checks.noToolErrors()],
|
|
135
|
+
scorers: [
|
|
136
|
+
{ scorer: faithfulnessScorer, threshold: 0.7 }, // min threshold (number shorthand)
|
|
137
|
+
{ scorer: hallucinationScorer, threshold: { max: 0.3 } }, // max threshold (high = bad)
|
|
138
|
+
{ scorer: toneScorer, threshold: { min: 0.5, max: 0.9 } }, // range threshold
|
|
139
|
+
checks.includes('Brooklyn'), // bare scorer, no threshold
|
|
140
|
+
],
|
|
141
|
+
})
|
|
142
|
+
|
|
143
|
+
if (result.verdict === 'failed') {
|
|
144
|
+
console.log(
|
|
145
|
+
'Gate failures:',
|
|
146
|
+
result.gateResults?.filter(g => !g.passed),
|
|
147
|
+
)
|
|
148
|
+
} else if (result.verdict === 'scored') {
|
|
149
|
+
console.log(
|
|
150
|
+
'Threshold misses:',
|
|
151
|
+
result.thresholdResults?.filter(t => !t.passed),
|
|
152
|
+
)
|
|
153
|
+
}
|
|
154
|
+
```
|
|
155
|
+
|
|
88
156
|
### Agent Evaluation
|
|
89
157
|
|
|
90
158
|
```typescript
|
|
@@ -246,6 +314,8 @@ const result = await runEvals({
|
|
|
246
314
|
|
|
247
315
|
## Related
|
|
248
316
|
|
|
317
|
+
- [Gates and Verdicts](https://mastra.ai/docs/evals/gates-and-verdicts): Conceptual guide to severity semantics
|
|
318
|
+
- [Quick Checks](https://mastra.ai/reference/evals/checks): Zero-LLM composable micro-scorers
|
|
249
319
|
- [createScorer()](https://mastra.ai/reference/evals/create-scorer): Create custom scorers for experiments
|
|
250
320
|
- [MastraScorer](https://mastra.ai/reference/evals/mastra-scorer): Learn about scorer structure and methods
|
|
251
321
|
- [Trajectory Accuracy](https://mastra.ai/reference/evals/trajectory-accuracy): Built-in trajectory evaluation scorers
|
package/.docs/reference/index.md
CHANGED
|
@@ -40,6 +40,7 @@ The Reference section provides documentation of Mastra's API, including paramete
|
|
|
40
40
|
- [toAISdkV5Messages()](https://mastra.ai/reference/ai-sdk/to-ai-sdk-v5-messages)
|
|
41
41
|
- [withMastra()](https://mastra.ai/reference/ai-sdk/with-mastra)
|
|
42
42
|
- [workflowRoute()](https://mastra.ai/reference/ai-sdk/workflow-route)
|
|
43
|
+
- [workflowSnapshotToStream()](https://mastra.ai/reference/ai-sdk/workflow-snapshot-to-stream)
|
|
43
44
|
- [Auth0](https://mastra.ai/reference/auth/auth0)
|
|
44
45
|
- [Better Auth](https://mastra.ai/reference/auth/better-auth)
|
|
45
46
|
- [Clerk](https://mastra.ai/reference/auth/clerk)
|
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
1
|
# @mastra/mcp-docs-server
|
|
2
2
|
|
|
3
|
+
## 1.2.2-alpha.5
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- Updated dependencies [[`bf3fe49`](https://github.com/mastra-ai/mastra/commit/bf3fe49f9467dbbdb8f9eaf74e0f7971ffb19559), [`24ceaea`](https://github.com/mastra-ai/mastra/commit/24ceaea0bdd8609cabbab764380608ca6621a194), [`6ccf67b`](https://github.com/mastra-ai/mastra/commit/6ccf67bf075753754927a57bc2e1734ba2c820c5), [`825d8de`](https://github.com/mastra-ai/mastra/commit/825d8def9fa64c2bcc3d8dd6b49e09342c3ac5c7), [`ffa09e7`](https://github.com/mastra-ai/mastra/commit/ffa09e772a5c92270eabe2090fc42d45bd8ec4b7), [`461a7c5`](https://github.com/mastra-ai/mastra/commit/461a7c501449295287f4f0ee4b0b42344f39fcf8), [`4211472`](https://github.com/mastra-ai/mastra/commit/4211472a5a2bd319c60cd2e42d9109c3eef7ac1c), [`9e45902`](https://github.com/mastra-ai/mastra/commit/9e4590208e745055cecca202e2db0e5c65e17d3c), [`5c0df77`](https://github.com/mastra-ai/mastra/commit/5c0df776c40efa420f8c07a2f3ee66010296618e)]:
|
|
8
|
+
- @mastra/core@1.47.0-alpha.3
|
|
9
|
+
|
|
10
|
+
## 1.2.2-alpha.3
|
|
11
|
+
|
|
12
|
+
### Patch Changes
|
|
13
|
+
|
|
14
|
+
- Updated dependencies [[`86623c1`](https://github.com/mastra-ai/mastra/commit/86623c1adf7d22de32cc916dda17f4155184db36), [`7c9dd77`](https://github.com/mastra-ai/mastra/commit/7c9dd77bd18cb8dc72797e25f1a0fbdc71a11347), [`9990965`](https://github.com/mastra-ai/mastra/commit/999096571635a83b42ef40841fd7028cfa630779), [`c0ffa3c`](https://github.com/mastra-ai/mastra/commit/c0ffa3c897ccd326de880df734740a7f0681a18f), [`0504bf5`](https://github.com/mastra-ai/mastra/commit/0504bf5e8cffc571a4b343326178de371e6f859b), [`5afe423`](https://github.com/mastra-ai/mastra/commit/5afe423e4badf040f1b0d4525183a856fcb8146e), [`86623c1`](https://github.com/mastra-ai/mastra/commit/86623c1adf7d22de32cc916dda17f4155184db36), [`8c9f1c0`](https://github.com/mastra-ai/mastra/commit/8c9f1c0361d89066f9bcd14a2f69e761b01766c8)]:
|
|
15
|
+
- @mastra/core@1.47.0-alpha.2
|
|
16
|
+
|
|
3
17
|
## 1.2.2-alpha.2
|
|
4
18
|
|
|
5
19
|
### Patch Changes
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@mastra/mcp-docs-server",
|
|
3
|
-
"version": "1.2.2-alpha.
|
|
3
|
+
"version": "1.2.2-alpha.5",
|
|
4
4
|
"description": "MCP server for accessing Mastra.ai documentation, changelogs, and news.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
@@ -28,8 +28,8 @@
|
|
|
28
28
|
"jsdom": "^26.1.0",
|
|
29
29
|
"local-pkg": "^1.1.2",
|
|
30
30
|
"zod": "^4.4.3",
|
|
31
|
-
"@mastra/
|
|
32
|
-
"@mastra/
|
|
31
|
+
"@mastra/core": "1.47.0-alpha.3",
|
|
32
|
+
"@mastra/mcp": "^1.12.0"
|
|
33
33
|
},
|
|
34
34
|
"devDependencies": {
|
|
35
35
|
"@hono/node-server": "^1.19.11",
|
|
@@ -47,7 +47,7 @@
|
|
|
47
47
|
"vitest": "4.1.8",
|
|
48
48
|
"@internal/lint": "0.0.108",
|
|
49
49
|
"@internal/types-builder": "0.0.83",
|
|
50
|
-
"@mastra/core": "1.
|
|
50
|
+
"@mastra/core": "1.47.0-alpha.3"
|
|
51
51
|
},
|
|
52
52
|
"homepage": "https://mastra.ai",
|
|
53
53
|
"repository": {
|