@mastra/mcp-docs-server 1.2.19-alpha.0 → 1.2.19-alpha.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.docs/docs/evals/built-in-scorers.md +1 -0
- package/.docs/docs/evals/multi-turn.md +84 -1
- package/.docs/docs/evals/overview.md +63 -1
- package/.docs/docs/sandbox/filesystem.md +8 -8
- package/.docs/docs/sandbox/overview.md +33 -66
- package/.docs/models/gateways/merge-gateway.md +2 -1
- package/.docs/models/gateways/netlify.md +1 -2
- package/.docs/models/gateways/openrouter.md +6 -2
- package/.docs/models/gateways/vercel.md +5 -1
- package/.docs/models/index.md +1 -1
- package/.docs/models/providers/crossmodel.md +56 -55
- package/.docs/models/providers/deepseek.md +8 -7
- package/.docs/models/providers/digitalocean.md +8 -8
- package/.docs/models/providers/edenai.md +8 -7
- package/.docs/models/providers/google.md +3 -3
- package/.docs/models/providers/hyper.md +3 -3
- package/.docs/models/providers/kilo.md +14 -9
- package/.docs/models/providers/llmgateway-providers.md +3 -2
- package/.docs/models/providers/nano-gpt.md +7 -3
- package/.docs/models/providers/ofox.md +113 -110
- package/.docs/models/providers/opencode-go.md +25 -23
- package/.docs/models/providers/opencode.md +1 -1
- package/.docs/models/providers/scaleway.md +2 -1
- package/.docs/reference/core/mastra-class.md +1 -1
- package/.docs/reference/evals/checks.md +6 -0
- package/.docs/reference/evals/multi-turn-judge.md +101 -0
- package/.docs/reference/index.md +1 -0
- package/.docs/reference/workspace/workspace-class.md +15 -3
- package/CHANGELOG.md +21 -0
- package/package.json +5 -5
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# Ofox
|
|
4
4
|
|
|
5
|
-
Access
|
|
5
|
+
Access 110 Ofox models through Mastra's model router. Authentication is handled automatically using the `OFOX_API_KEY` environment variable.
|
|
6
6
|
|
|
7
7
|
Learn more in the [Ofox documentation](https://ofox.ai/docs).
|
|
8
8
|
|
|
@@ -34,115 +34,118 @@ for await (const chunk of stream) {
|
|
|
34
34
|
|
|
35
35
|
## Models
|
|
36
36
|
|
|
37
|
-
| Model
|
|
38
|
-
|
|
|
39
|
-
| `ofox/anthropic/claude-fable-5`
|
|
40
|
-
| `ofox/anthropic/claude-haiku-4.5`
|
|
41
|
-
| `ofox/anthropic/claude-opus-4.5`
|
|
42
|
-
| `ofox/anthropic/claude-opus-4.6`
|
|
43
|
-
| `ofox/anthropic/claude-opus-4.7`
|
|
44
|
-
| `ofox/anthropic/claude-opus-4.8`
|
|
45
|
-
| `ofox/anthropic/claude-opus-5`
|
|
46
|
-
| `ofox/anthropic/claude-sonnet-4.5`
|
|
47
|
-
| `ofox/anthropic/claude-sonnet-4.6`
|
|
48
|
-
| `ofox/anthropic/claude-sonnet-5`
|
|
49
|
-
| `ofox/bailian/qwen-flash`
|
|
50
|
-
| `ofox/bailian/qwen-max`
|
|
51
|
-
| `ofox/bailian/qwen-plus`
|
|
52
|
-
| `ofox/bailian/qwen-turbo`
|
|
53
|
-
| `ofox/bailian/qwen-vl-max`
|
|
54
|
-
| `ofox/bailian/qwen3-coder-flash`
|
|
55
|
-
| `ofox/bailian/qwen3-coder-next`
|
|
56
|
-
| `ofox/bailian/qwen3-coder-plus`
|
|
57
|
-
| `ofox/bailian/qwen3-max`
|
|
58
|
-
| `ofox/bailian/qwen3.5-122b-a10b`
|
|
59
|
-
| `ofox/bailian/qwen3.5-27b`
|
|
60
|
-
| `ofox/bailian/qwen3.5-35b-a3b`
|
|
61
|
-
| `ofox/bailian/qwen3.5-397b-a17b`
|
|
62
|
-
| `ofox/bailian/qwen3.5-flash`
|
|
63
|
-
| `ofox/bailian/qwen3.5-plus`
|
|
64
|
-
| `ofox/bailian/qwen3.6-27b`
|
|
65
|
-
| `ofox/bailian/qwen3.6-flash`
|
|
66
|
-
| `ofox/bailian/qwen3.6-max-preview`
|
|
67
|
-
| `ofox/bailian/qwen3.6-plus`
|
|
68
|
-
| `ofox/bailian/qwen3.7-max`
|
|
69
|
-
| `ofox/bailian/qwen3.7-plus`
|
|
70
|
-
| `ofox/bailian/qwen3.8-
|
|
71
|
-
| `ofox/
|
|
72
|
-
| `ofox/deepseek/deepseek-
|
|
73
|
-
| `ofox/deepseek/deepseek-v4-
|
|
74
|
-
| `ofox/deepseek/deepseek-v4-
|
|
75
|
-
| `ofox/
|
|
76
|
-
| `ofox/
|
|
77
|
-
| `ofox/
|
|
78
|
-
| `ofox/google/gemini-
|
|
79
|
-
| `ofox/google/gemini-
|
|
80
|
-
| `ofox/google/gemini-
|
|
81
|
-
| `ofox/google/gemini-3
|
|
82
|
-
| `ofox/google/gemini-3.
|
|
83
|
-
| `ofox/google/gemini-3.
|
|
84
|
-
| `ofox/google/gemini-3.
|
|
85
|
-
| `ofox/
|
|
86
|
-
| `ofox/
|
|
87
|
-
| `ofox/
|
|
88
|
-
| `ofox/minimax/
|
|
89
|
-
| `ofox/minimax/minimax-m2
|
|
90
|
-
| `ofox/minimax/minimax-m2.
|
|
91
|
-
| `ofox/minimax/minimax-m2.
|
|
92
|
-
| `ofox/minimax/minimax-m2.
|
|
93
|
-
| `ofox/minimax/minimax-
|
|
94
|
-
| `ofox/
|
|
95
|
-
| `ofox/
|
|
96
|
-
| `ofox/
|
|
97
|
-
| `ofox/moonshotai/kimi-k2.
|
|
98
|
-
| `ofox/moonshotai/kimi-
|
|
99
|
-
| `ofox/
|
|
100
|
-
| `ofox/
|
|
101
|
-
| `ofox/
|
|
102
|
-
| `ofox/openai/gpt-
|
|
103
|
-
| `ofox/openai/gpt-
|
|
104
|
-
| `ofox/openai/gpt-
|
|
105
|
-
| `ofox/openai/gpt-
|
|
106
|
-
| `ofox/openai/gpt-5
|
|
107
|
-
| `ofox/openai/gpt-5
|
|
108
|
-
| `ofox/openai/gpt-5
|
|
109
|
-
| `ofox/openai/gpt-5.
|
|
110
|
-
| `ofox/openai/gpt-5.
|
|
111
|
-
| `ofox/openai/gpt-5.
|
|
112
|
-
| `ofox/openai/gpt-5.
|
|
113
|
-
| `ofox/openai/gpt-5.
|
|
114
|
-
| `ofox/openai/gpt-5.
|
|
115
|
-
| `ofox/openai/gpt-5.4
|
|
116
|
-
| `ofox/openai/gpt-5.
|
|
117
|
-
| `ofox/openai/gpt-5.
|
|
118
|
-
| `ofox/openai/gpt-5.
|
|
119
|
-
| `ofox/openai/gpt-5.
|
|
120
|
-
| `ofox/
|
|
121
|
-
| `ofox/
|
|
122
|
-
| `ofox/
|
|
123
|
-
| `ofox/volcengine/doubao-seed-1-
|
|
124
|
-
| `ofox/volcengine/doubao-seed-
|
|
125
|
-
| `ofox/volcengine/doubao-seed-
|
|
126
|
-
| `ofox/volcengine/doubao-seed-
|
|
127
|
-
| `ofox/volcengine/doubao-seed-2.0-
|
|
128
|
-
| `ofox/volcengine/doubao-seed-2.
|
|
129
|
-
| `ofox/volcengine/doubao-seed-2.
|
|
130
|
-
| `ofox/volcengine/doubao-seed-
|
|
131
|
-
| `ofox/volcengine/doubao-seed-
|
|
132
|
-
| `ofox/
|
|
133
|
-
| `ofox/
|
|
134
|
-
| `ofox/
|
|
135
|
-
| `ofox/x-ai/grok-4.
|
|
136
|
-
| `ofox/x-ai/grok-4.
|
|
137
|
-
| `ofox/
|
|
138
|
-
| `ofox/
|
|
139
|
-
| `ofox/
|
|
140
|
-
| `ofox/z-ai/glm-
|
|
141
|
-
| `ofox/z-ai/glm-
|
|
142
|
-
| `ofox/z-ai/glm-
|
|
143
|
-
| `ofox/z-ai/glm-5
|
|
144
|
-
| `ofox/z-ai/glm-5
|
|
145
|
-
| `ofox/z-ai/glm-
|
|
37
|
+
| Model | Context | Tools | Reasoning | Image | Audio | Video | Input $/1M | Output $/1M |
|
|
38
|
+
| -------------------------------------------- | ------- | ----- | --------- | ----- | ----- | ----- | ---------- | ----------- |
|
|
39
|
+
| `ofox/anthropic/claude-fable-5` | 1.0M | | | | | | $10 | $50 |
|
|
40
|
+
| `ofox/anthropic/claude-haiku-4.5` | 200K | | | | | | $1 | $5 |
|
|
41
|
+
| `ofox/anthropic/claude-opus-4.5` | 200K | | | | | | $5 | $25 |
|
|
42
|
+
| `ofox/anthropic/claude-opus-4.6` | 1.0M | | | | | | $5 | $25 |
|
|
43
|
+
| `ofox/anthropic/claude-opus-4.7` | 1.0M | | | | | | $5 | $25 |
|
|
44
|
+
| `ofox/anthropic/claude-opus-4.8` | 1.0M | | | | | | $5 | $25 |
|
|
45
|
+
| `ofox/anthropic/claude-opus-5` | 1.0M | | | | | | $5 | $25 |
|
|
46
|
+
| `ofox/anthropic/claude-sonnet-4.5` | 200K | | | | | | $3 | $15 |
|
|
47
|
+
| `ofox/anthropic/claude-sonnet-4.6` | 1.0M | | | | | | $3 | $15 |
|
|
48
|
+
| `ofox/anthropic/claude-sonnet-5` | 1.0M | | | | | | $2 | $10 |
|
|
49
|
+
| `ofox/bailian/qwen-flash` | 1.0M | | | | | | $0.02 | $0.22 |
|
|
50
|
+
| `ofox/bailian/qwen-max` | 33K | | | | | | $0.35 | $1 |
|
|
51
|
+
| `ofox/bailian/qwen-plus` | 1.0M | | | | | | $0.12 | $0.29 |
|
|
52
|
+
| `ofox/bailian/qwen-turbo` | 128K | | | | | | $0.05 | $0.09 |
|
|
53
|
+
| `ofox/bailian/qwen-vl-max` | 131K | | | | | | $0.23 | $0.58 |
|
|
54
|
+
| `ofox/bailian/qwen3-coder-flash` | 1.0M | | | | | | $0.50 | $3 |
|
|
55
|
+
| `ofox/bailian/qwen3-coder-next` | 262K | | | | | | $0.20 | $2 |
|
|
56
|
+
| `ofox/bailian/qwen3-coder-plus` | 1.0M | | | | | | $2 | $9 |
|
|
57
|
+
| `ofox/bailian/qwen3-max` | 262K | | | | | | $0.36 | $1 |
|
|
58
|
+
| `ofox/bailian/qwen3.5-122b-a10b` | 256K | | | | | | $0.29 | $2 |
|
|
59
|
+
| `ofox/bailian/qwen3.5-27b` | 262K | | | | | | $0.29 | $2 |
|
|
60
|
+
| `ofox/bailian/qwen3.5-35b-a3b` | 262K | | | | | | $0.29 | $2 |
|
|
61
|
+
| `ofox/bailian/qwen3.5-397b-a17b` | 256K | | | | | | $0.55 | $4 |
|
|
62
|
+
| `ofox/bailian/qwen3.5-flash` | 1.0M | | | | | | $0.10 | $0.40 |
|
|
63
|
+
| `ofox/bailian/qwen3.5-plus` | 1.0M | | | | | | $0.40 | $2 |
|
|
64
|
+
| `ofox/bailian/qwen3.6-27b` | 256K | | | | | | $0.60 | $4 |
|
|
65
|
+
| `ofox/bailian/qwen3.6-flash` | 1.0M | | | | | | $0.25 | $2 |
|
|
66
|
+
| `ofox/bailian/qwen3.6-max-preview` | 262K | | | | | | $2 | $13 |
|
|
67
|
+
| `ofox/bailian/qwen3.6-plus` | 1.0M | | | | | | $0.50 | $3 |
|
|
68
|
+
| `ofox/bailian/qwen3.7-max` | 1.0M | | | | | | $3 | $8 |
|
|
69
|
+
| `ofox/bailian/qwen3.7-plus` | 1.0M | | | | | | $0.40 | $2 |
|
|
70
|
+
| `ofox/bailian/qwen3.8-27b` | 1.1M | | | | | | $0.45 | $3 |
|
|
71
|
+
| `ofox/bailian/qwen3.8-max` | 1.0M | | | | | | $2 | $6 |
|
|
72
|
+
| `ofox/deepseek/deepseek-v3.2` | 128K | | | | | | $0.29 | $0.43 |
|
|
73
|
+
| `ofox/deepseek/deepseek-v4-flash` | 1.0M | | | | | | $0.44 | $1 |
|
|
74
|
+
| `ofox/deepseek/deepseek-v4-flash-vision-exp` | 1.0M | | | | | | $0.44 | $1 |
|
|
75
|
+
| `ofox/deepseek/deepseek-v4-pro` | 1.0M | | | | | | $1 | $4 |
|
|
76
|
+
| `ofox/deepseek/deepseek-v4-pro-0423` | 1.0M | | | | | | $1 | $4 |
|
|
77
|
+
| `ofox/deepseek/deepseek-v4-pro-0813` | 1.0M | | | | | | $1 | $4 |
|
|
78
|
+
| `ofox/google/gemini-2.5-flash` | 1.0M | | | | | | $0.30 | $3 |
|
|
79
|
+
| `ofox/google/gemini-2.5-flash-lite` | 1.0M | | | | | | $0.10 | $0.40 |
|
|
80
|
+
| `ofox/google/gemini-2.5-pro` | 1.0M | | | | | | $1 | $10 |
|
|
81
|
+
| `ofox/google/gemini-3-flash-preview` | 1.0M | | | | | | $0.50 | $3 |
|
|
82
|
+
| `ofox/google/gemini-3.1-flash-lite` | 1.0M | | | | | | $0.25 | $2 |
|
|
83
|
+
| `ofox/google/gemini-3.1-pro-preview` | 1.0M | | | | | | $2 | $12 |
|
|
84
|
+
| `ofox/google/gemini-3.5-flash` | 1.0M | | | | | | $2 | $9 |
|
|
85
|
+
| `ofox/google/gemini-3.5-flash-lite` | 1.0M | | | | | | $0.30 | $3 |
|
|
86
|
+
| `ofox/google/gemini-3.6-flash` | 1.0M | | | | | | $0.75 | $4 |
|
|
87
|
+
| `ofox/google/gemini-3.7-flash` | 1.0M | | | | | | $0.75 | $4 |
|
|
88
|
+
| `ofox/minimax/m2-her` | 200K | | | | | | $0.30 | $1 |
|
|
89
|
+
| `ofox/minimax/minimax-m2` | 197K | | | | | | $0.30 | $1 |
|
|
90
|
+
| `ofox/minimax/minimax-m2.1` | 205K | | | | | | $0.30 | $1 |
|
|
91
|
+
| `ofox/minimax/minimax-m2.1-lightning` | 205K | | | | | | $0.30 | $2 |
|
|
92
|
+
| `ofox/minimax/minimax-m2.5` | 205K | | | | | | $0.30 | $1 |
|
|
93
|
+
| `ofox/minimax/minimax-m2.5-lightning` | 205K | | | | | | $0.30 | $2 |
|
|
94
|
+
| `ofox/minimax/minimax-m2.7` | 205K | | | | | | $0.30 | $1 |
|
|
95
|
+
| `ofox/minimax/minimax-m2.7-highspeed` | 205K | | | | | | $0.60 | $2 |
|
|
96
|
+
| `ofox/minimax/minimax-m3` | 512K | | | | | | $0.60 | $2 |
|
|
97
|
+
| `ofox/moonshotai/kimi-k2.5` | 262K | | | | | | $0.60 | $3 |
|
|
98
|
+
| `ofox/moonshotai/kimi-k2.6` | 262K | | | | | | $0.95 | $4 |
|
|
99
|
+
| `ofox/moonshotai/kimi-k2.7-code` | 262K | | | | | | $0.95 | $4 |
|
|
100
|
+
| `ofox/moonshotai/kimi-k2.7-code-highspeed` | 262K | | | | | | $2 | $8 |
|
|
101
|
+
| `ofox/moonshotai/kimi-k3` | 1.0M | | | | | | $3 | $15 |
|
|
102
|
+
| `ofox/openai/gpt-4.1` | 1.0M | | | | | | $2 | $8 |
|
|
103
|
+
| `ofox/openai/gpt-4.1-mini` | 1.0M | | | | | | $0.40 | $2 |
|
|
104
|
+
| `ofox/openai/gpt-4o` | 128K | | | | | | $3 | $10 |
|
|
105
|
+
| `ofox/openai/gpt-4o-mini` | 128K | | | | | | $0.15 | $0.60 |
|
|
106
|
+
| `ofox/openai/gpt-5` | 400K | | | | | | $1 | $10 |
|
|
107
|
+
| `ofox/openai/gpt-5-mini` | 256K | | | | | | $0.25 | $2 |
|
|
108
|
+
| `ofox/openai/gpt-5-nano` | 400K | | | | | | $0.05 | $0.40 |
|
|
109
|
+
| `ofox/openai/gpt-5.1` | 400K | | | | | | $1 | $10 |
|
|
110
|
+
| `ofox/openai/gpt-5.1-codex-max` | 256K | | | | | | $1 | $10 |
|
|
111
|
+
| `ofox/openai/gpt-5.1-codex-mini` | 256K | | | | | | $0.25 | $2 |
|
|
112
|
+
| `ofox/openai/gpt-5.2` | 400K | | | | | | $2 | $14 |
|
|
113
|
+
| `ofox/openai/gpt-5.2-codex` | 400K | | | | | | $2 | $14 |
|
|
114
|
+
| `ofox/openai/gpt-5.3-codex` | 400K | | | | | | $2 | $14 |
|
|
115
|
+
| `ofox/openai/gpt-5.4` | 1.1M | | | | | | $3 | $15 |
|
|
116
|
+
| `ofox/openai/gpt-5.4-mini` | 400K | | | | | | $0.75 | $5 |
|
|
117
|
+
| `ofox/openai/gpt-5.4-nano` | 400K | | | | | | $0.20 | $1 |
|
|
118
|
+
| `ofox/openai/gpt-5.4-pro` | 1.1M | | | | | | $30 | $180 |
|
|
119
|
+
| `ofox/openai/gpt-5.5` | 1.1M | | | | | | $5 | $30 |
|
|
120
|
+
| `ofox/openai/gpt-5.6-luna` | 1.1M | | | | | | $0.20 | $1 |
|
|
121
|
+
| `ofox/openai/gpt-5.6-sol` | 1.1M | | | | | | $3 | $15 |
|
|
122
|
+
| `ofox/openai/gpt-5.6-terra` | 1.1M | | | | | | $2 | $12 |
|
|
123
|
+
| `ofox/volcengine/doubao-seed-1-6` | 256K | | | | | | $0.12 | $0.29 |
|
|
124
|
+
| `ofox/volcengine/doubao-seed-1-6-flash` | 256K | | | | | | $0.03 | $0.22 |
|
|
125
|
+
| `ofox/volcengine/doubao-seed-1-6-vision` | 256K | | | | | | $0.12 | $1 |
|
|
126
|
+
| `ofox/volcengine/doubao-seed-1-8` | 256K | | | | | | $0.12 | $0.29 |
|
|
127
|
+
| `ofox/volcengine/doubao-seed-2.0-code` | 256K | | | | | | $0.67 | $3 |
|
|
128
|
+
| `ofox/volcengine/doubao-seed-2.0-lite` | 256K | | | | | | $0.13 | $0.76 |
|
|
129
|
+
| `ofox/volcengine/doubao-seed-2.0-mini` | 256K | | | | | | $0.06 | $0.56 |
|
|
130
|
+
| `ofox/volcengine/doubao-seed-2.0-pro` | 256K | | | | | | $0.67 | $3 |
|
|
131
|
+
| `ofox/volcengine/doubao-seed-2.1-pro` | 256K | | | | | | $0.71 | $4 |
|
|
132
|
+
| `ofox/volcengine/doubao-seed-2.1-turbo` | 256K | | | | | | $0.35 | $2 |
|
|
133
|
+
| `ofox/volcengine/doubao-seed-character` | 256K | | | | | | $0.18 | $0.88 |
|
|
134
|
+
| `ofox/volcengine/doubao-seed-evolving` | 256K | | | | | | $0.88 | $4 |
|
|
135
|
+
| `ofox/x-ai/grok-4.1-fast` | 2.0M | | | | | | $0.20 | $0.50 |
|
|
136
|
+
| `ofox/x-ai/grok-4.20` | 2.0M | | | | | | $4 | $12 |
|
|
137
|
+
| `ofox/x-ai/grok-4.3` | 1.0M | | | | | | $1 | $3 |
|
|
138
|
+
| `ofox/x-ai/grok-4.5` | 500K | | | | | | $2 | $6 |
|
|
139
|
+
| `ofox/x-ai/grok-4.6` | 500K | | | | | | $2 | $6 |
|
|
140
|
+
| `ofox/z-ai/glm-4.6` | 205K | | | | | | $0.40 | $2 |
|
|
141
|
+
| `ofox/z-ai/glm-4.7` | 205K | | | | | | $0.40 | $2 |
|
|
142
|
+
| `ofox/z-ai/glm-4.7-flashx` | 200K | | | | | | $0.07 | $0.43 |
|
|
143
|
+
| `ofox/z-ai/glm-5` | 205K | | | | | | $1 | $3 |
|
|
144
|
+
| `ofox/z-ai/glm-5-turbo` | 200K | | | | | | $1 | $4 |
|
|
145
|
+
| `ofox/z-ai/glm-5.1` | 200K | | | | | | $1 | $4 |
|
|
146
|
+
| `ofox/z-ai/glm-5.2` | 1.0M | | | | | | $0.98 | $3 |
|
|
147
|
+
| `ofox/z-ai/glm-5.3` | 1.0M | | | | | | $1 | $4 |
|
|
148
|
+
| `ofox/z-ai/glm-5v-turbo` | 200K | | | | | | $1 | $4 |
|
|
146
149
|
|
|
147
150
|
## Advanced configuration
|
|
148
151
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# OpenCode Go
|
|
4
4
|
|
|
5
|
-
Access
|
|
5
|
+
Access 28 OpenCode Go models through Mastra's model router. Authentication is handled automatically using the `OPENCODE_API_KEY` environment variable.
|
|
6
6
|
|
|
7
7
|
Learn more in the [OpenCode Go documentation](https://opencode.ai/docs/zen).
|
|
8
8
|
|
|
@@ -34,28 +34,30 @@ for await (const chunk of stream) {
|
|
|
34
34
|
|
|
35
35
|
## Models
|
|
36
36
|
|
|
37
|
-
| Model
|
|
38
|
-
|
|
|
39
|
-
| `opencode-go/deepseek-v4-flash`
|
|
40
|
-
| `opencode-go/deepseek-v4-
|
|
41
|
-
| `opencode-go/
|
|
42
|
-
| `opencode-go/glm-5.
|
|
43
|
-
| `opencode-go/glm-5.
|
|
44
|
-
| `opencode-go/
|
|
45
|
-
| `opencode-go/
|
|
46
|
-
| `opencode-go/
|
|
47
|
-
| `opencode-go/
|
|
48
|
-
| `opencode-go/kimi-k2.
|
|
49
|
-
| `opencode-go/kimi-
|
|
50
|
-
| `opencode-go/
|
|
51
|
-
| `opencode-go/mimo-v2.5
|
|
52
|
-
| `opencode-go/
|
|
53
|
-
| `opencode-go/minimax-
|
|
54
|
-
| `opencode-go/
|
|
55
|
-
| `opencode-go/
|
|
56
|
-
| `opencode-go/
|
|
57
|
-
| `opencode-go/qwen3.
|
|
58
|
-
| `opencode-go/qwen3.
|
|
37
|
+
| Model | Context | Tools | Reasoning | Image | Audio | Video | Input $/1M | Output $/1M |
|
|
38
|
+
| ------------------------------------------ | ------- | ----- | --------- | ----- | ----- | ----- | ---------- | ----------- |
|
|
39
|
+
| `opencode-go/deepseek-v4-flash` | 1.0M | | | | | | $0.22 | $0.66 |
|
|
40
|
+
| `opencode-go/deepseek-v4-flash-vision-exp` | 1.0M | | | | | | $0.22 | $0.66 |
|
|
41
|
+
| `opencode-go/deepseek-v4-pro` | 1.0M | | | | | | $0.66 | $2 |
|
|
42
|
+
| `opencode-go/glm-5.1` | 203K | | | | | | $1 | $4 |
|
|
43
|
+
| `opencode-go/glm-5.2` | 1.0M | | | | | | $1 | $4 |
|
|
44
|
+
| `opencode-go/glm-5.3` | 1.0M | | | | | | $1 | $4 |
|
|
45
|
+
| `opencode-go/gpt-5.6-luna` | 1.1M | | | | | | $0.20 | $1 |
|
|
46
|
+
| `opencode-go/grok-4.5` | 500K | | | | | | $2 | $6 |
|
|
47
|
+
| `opencode-go/hy3` | 256K | | | | | | $0.02 | $0.07 |
|
|
48
|
+
| `opencode-go/kimi-k2.6` | 262K | | | | | | $0.95 | $4 |
|
|
49
|
+
| `opencode-go/kimi-k2.7-code` | 262K | | | | | | $0.95 | $4 |
|
|
50
|
+
| `opencode-go/kimi-k3` | 1.0M | | | | | | $3 | $15 |
|
|
51
|
+
| `opencode-go/mimo-v2.5` | 1.0M | | | | | | $0.14 | $0.28 |
|
|
52
|
+
| `opencode-go/mimo-v2.5-pro` | 1.0M | | | | | | $0.43 | $0.87 |
|
|
53
|
+
| `opencode-go/minimax-m2.7` | 205K | | | | | | $0.30 | $1 |
|
|
54
|
+
| `opencode-go/minimax-m3` | 1.0M | | | | | | $0.30 | $1 |
|
|
55
|
+
| `opencode-go/muse-spark-1.2-contributor` | 1.0M | | | | | | $0.10 | $0.20 |
|
|
56
|
+
| `opencode-go/ox-alpha-free` | 1.0M | | | | | | — | — |
|
|
57
|
+
| `opencode-go/qwen3.6-plus` | 1.0M | | | | | | $0.50 | $3 |
|
|
58
|
+
| `opencode-go/qwen3.7-max` | 1.0M | | | | | | $3 | $8 |
|
|
59
|
+
| `opencode-go/qwen3.7-plus` | 1.0M | | | | | | $0.40 | $2 |
|
|
60
|
+
| `opencode-go/qwen3.8-max` | 1.0M | | | | | | $2 | $6 |
|
|
59
61
|
|
|
60
62
|
## Advanced configuration
|
|
61
63
|
|
|
@@ -77,7 +77,7 @@ for await (const chunk of stream) {
|
|
|
77
77
|
| `opencode/gpt-5.5` | 1.1M | | | | | | $5 | $30 |
|
|
78
78
|
| `opencode/gpt-5.5-pro` | 1.1M | | | | | | $30 | $180 |
|
|
79
79
|
| `opencode/gpt-5.6-luna` | 1.1M | | | | | | $0.20 | $1 |
|
|
80
|
-
| `opencode/gpt-5.6-sol` | 1.1M | | | | | | $
|
|
80
|
+
| `opencode/gpt-5.6-sol` | 1.1M | | | | | | $2 | $10 |
|
|
81
81
|
| `opencode/gpt-5.6-terra` | 1.1M | | | | | | $3 | $15 |
|
|
82
82
|
| `opencode/grok-4.5` | 500K | | | | | | $2 | $6 |
|
|
83
83
|
| `opencode/grok-4.6` | 500K | | | | | | $2 | $6 |
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# Scaleway
|
|
4
4
|
|
|
5
|
-
Access
|
|
5
|
+
Access 15 Scaleway models through Mastra's model router. Authentication is handled automatically using the `SCALEWAY_API_KEY` environment variable.
|
|
6
6
|
|
|
7
7
|
Learn more in the [Scaleway documentation](https://www.scaleway.com/en/docs/generative-apis/).
|
|
8
8
|
|
|
@@ -37,6 +37,7 @@ for await (const chunk of stream) {
|
|
|
37
37
|
| Model | Context | Tools | Reasoning | Image | Audio | Video | Input $/1M | Output $/1M |
|
|
38
38
|
| ---------------------------------------------- | ------- | ----- | --------- | ----- | ----- | ----- | ---------- | ----------- |
|
|
39
39
|
| `scaleway/bge-multilingual-gemma2` | 8K | | | | | | $0.10 | — |
|
|
40
|
+
| `scaleway/deepseek-v4-flash-0731` | 256K | | | | | | $0.47 | $0.94 |
|
|
40
41
|
| `scaleway/gemma-4-26b-a4b-it` | 256K | | | | | | $0.25 | $0.50 |
|
|
41
42
|
| `scaleway/glm-5.2` | 256K | | | | | | $2 | $6 |
|
|
42
43
|
| `scaleway/gpt-oss-120b` | 128K | | | | | | $0.15 | $0.60 |
|
|
@@ -79,7 +79,7 @@ Visit the [Configuration reference](https://mastra.ai/reference/configuration) f
|
|
|
79
79
|
|
|
80
80
|
**bundler** (`BundlerConfig`): Configuration for the asset bundler with options for externals, sourcemap, transpilePackages, and dynamicPackages. (Default: `{ externals: [], sourcemap: false, transpilePackages: [], dynamicPackages: [] }`)
|
|
81
81
|
|
|
82
|
-
**scorers** (`Record<string, Scorer>`): Scorers for evaluating agent responses and workflow outputs (Default: `{}`)
|
|
82
|
+
**scorers** (`Record<string, Scorer>`): Scorers for evaluating agent responses and workflow outputs. Registration also makes a scorer resolvable by ID, which is required to persist its scores. See Score persistence (Default: `{}`)
|
|
83
83
|
|
|
84
84
|
**processors** (`Record<string, Processor>`): Input/output processors for transforming agent inputs and outputs (Default: `{}`)
|
|
85
85
|
|
|
@@ -6,6 +6,8 @@ Quick Checks are zero-LLM, composable micro-scorers for common assertions. They
|
|
|
6
6
|
|
|
7
7
|
Internally they're standard `createScorer()` instances, so they have the same observability, storage, and pipeline integration as any other scorer.
|
|
8
8
|
|
|
9
|
+
> **Register checks to persist their scores:** A check's score is only written to the scores store when the check is registered on the Mastra instance, alongside `storage`. Each check has a fixed id (`checks.includes()` is `check-includes`, `checks.calledTool()` is `check-called-tool`, and so on), so one registered instance covers every use of that check regardless of its arguments. See [Score persistence](https://mastra.ai/docs/evals/overview).
|
|
10
|
+
|
|
9
11
|
## Usage example
|
|
10
12
|
|
|
11
13
|
```typescript
|
|
@@ -203,6 +205,10 @@ const result = await runEvals({
|
|
|
203
205
|
})
|
|
204
206
|
```
|
|
205
207
|
|
|
208
|
+
## Multi-turn behavior
|
|
209
|
+
|
|
210
|
+
Checks read the accumulated `run.output`, so in a [multi-turn eval](https://mastra.ai/docs/evals/multi-turn) they see every turn. `checks.calledTool('get_weather', { times: 2 })` counts calls across the whole conversation, and `checks.includes()` searches all assistant text. Use per-turn `turns[].scorers` when a specific turn has to satisfy the check.
|
|
211
|
+
|
|
206
212
|
## Related
|
|
207
213
|
|
|
208
214
|
- [Quick Checks overview](https://mastra.ai/docs/evals/quick-checks)
|
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
|
|
2
|
+
|
|
3
|
+
# Multi-turn Judge scorer
|
|
4
|
+
|
|
5
|
+
**Added in:** `@mastra/evals@1.9.0`
|
|
6
|
+
|
|
7
|
+
The `createMultiTurnJudgeScorer()` function creates an LLM-as-judge scorer that grades a whole conversation against a single plain-English criterion. It returns a **binary** score: `1` when the criterion is satisfied, otherwise `0`, and the `reason` echoes the criterion with the judge's explanation.
|
|
8
|
+
|
|
9
|
+
Unlike the other prebuilt LLM judges, which read a single assistant message, this scorer reads every assistant turn accumulated in `run.output`, so it works with the [multi-turn `inputs`](https://mastra.ai/docs/evals/multi-turn) form of [`runEvals()`](https://mastra.ai/reference/evals/run-evals).
|
|
10
|
+
|
|
11
|
+
## Parameters
|
|
12
|
+
|
|
13
|
+
**model** (`MastraModelConfig`): The language model used to grade the conversation. A smaller, cheaper model is usually sufficient for grading.
|
|
14
|
+
|
|
15
|
+
**criterion** (`string`): What the conversation must satisfy, in plain English, e.g. "The agent gave forecasts for London and Paris, and weather-appropriate packing advice".
|
|
16
|
+
|
|
17
|
+
**options** (`MultiTurnJudgeScorerOptions`): Configuration options for the scorer
|
|
18
|
+
|
|
19
|
+
## `.run()` returns
|
|
20
|
+
|
|
21
|
+
**score** (`number`): 1 when the judge considers the criterion satisfied, otherwise 0 (multiplied by scale).
|
|
22
|
+
|
|
23
|
+
**reason** (`string`): The verdict, the criterion it graded, and the judge's explanation of why the criterion is or is not satisfied.
|
|
24
|
+
|
|
25
|
+
## Usage with multi-turn evals
|
|
26
|
+
|
|
27
|
+
Pass the scorer to `runEvals` alongside an `inputs` array. Every assistant turn is included in the prompt sent to the judge:
|
|
28
|
+
|
|
29
|
+
```typescript
|
|
30
|
+
import { runEvals } from '@mastra/core/evals'
|
|
31
|
+
import { createMultiTurnJudgeScorer } from '@mastra/evals/scorers/prebuilt'
|
|
32
|
+
import { weatherAgent } from '../agents'
|
|
33
|
+
|
|
34
|
+
const result = await runEvals({
|
|
35
|
+
data: [
|
|
36
|
+
{
|
|
37
|
+
inputs: [
|
|
38
|
+
"I'm planning a trip to London, Paris, and Tokyo next week.",
|
|
39
|
+
"How's the weather looking in London?",
|
|
40
|
+
'And Paris?',
|
|
41
|
+
'Tokyo too?',
|
|
42
|
+
'Should I pack an umbrella for the London leg?',
|
|
43
|
+
],
|
|
44
|
+
},
|
|
45
|
+
],
|
|
46
|
+
target: weatherAgent,
|
|
47
|
+
scorers: [
|
|
48
|
+
{
|
|
49
|
+
scorer: createMultiTurnJudgeScorer({
|
|
50
|
+
model: 'anthropic/claude-haiku-4-5',
|
|
51
|
+
criterion:
|
|
52
|
+
'The agent provided weather forecasts for London, Paris, and Tokyo, and gave weather-appropriate packing or clothing advice.',
|
|
53
|
+
}),
|
|
54
|
+
threshold: 1,
|
|
55
|
+
},
|
|
56
|
+
],
|
|
57
|
+
})
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Use `threshold: 1` to turn the verdict into a pass or fail: the score is binary, so any lower threshold always passes.
|
|
61
|
+
|
|
62
|
+
## Persisting scores
|
|
63
|
+
|
|
64
|
+
Scores are only written to the scores store when a scorer with the same ID is registered on the Mastra instance, because persistence resolves scorer metadata through `Mastra.getScorerById()`. Only the ID is looked up, so the registered instance's `criterion` can be a placeholder:
|
|
65
|
+
|
|
66
|
+
```typescript
|
|
67
|
+
import { Mastra } from '@mastra/core'
|
|
68
|
+
import { LibSQLStore } from '@mastra/libsql'
|
|
69
|
+
import { createMultiTurnJudgeScorer } from '@mastra/evals/scorers/prebuilt'
|
|
70
|
+
|
|
71
|
+
export const mastra = new Mastra({
|
|
72
|
+
agents: { weatherAgent },
|
|
73
|
+
storage: new LibSQLStore({ url: 'file:./mastra.db' }),
|
|
74
|
+
scorers: {
|
|
75
|
+
'multi-turn-judge-scorer': createMultiTurnJudgeScorer({
|
|
76
|
+
model: 'anthropic/claude-haiku-4-5',
|
|
77
|
+
criterion: 'placeholder',
|
|
78
|
+
}),
|
|
79
|
+
},
|
|
80
|
+
})
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
See [Score persistence](https://mastra.ai/docs/evals/overview) for the full requirement and the warning you get when a scorer isn't registered.
|
|
84
|
+
|
|
85
|
+
## Scoring details
|
|
86
|
+
|
|
87
|
+
The scorer runs in two phases:
|
|
88
|
+
|
|
89
|
+
1. **Grade**: Every assistant message in `run.output` is collected in order and rendered as a numbered transcript, then the judge decides whether the conversation as a whole satisfies the criterion. Assistant messages with no text (a turn that only carried tool calls, for example) are skipped.
|
|
90
|
+
2. **Score**: A `satisfied` verdict scores `1` and anything else scores `0`, multiplied by `scale`.
|
|
91
|
+
|
|
92
|
+
The judge only sees what the assistant said. The user's turns and any tool results aren't included, so write criteria in terms of the agent's responses. This keeps the graded text limited to the agent's own output, but it also means a reply that only makes sense next to the question that prompted it ("Yes, bring one.") can't be judged on its own. For criteria that depend on the user's turns, grade each turn with `turns[].scorers` or write a [custom scorer](https://mastra.ai/docs/evals/multi-turn) that renders both roles.
|
|
93
|
+
|
|
94
|
+
The transcript is passed to the judge as untrusted data, fenced with explicit delimiters and an instruction to ignore anything inside it that reads as an instruction, so an agent response can't talk its way into a passing verdict.
|
|
95
|
+
|
|
96
|
+
## Related
|
|
97
|
+
|
|
98
|
+
- [Multi-turn evals](https://mastra.ai/docs/evals/multi-turn)
|
|
99
|
+
- [`runEvals()`](https://mastra.ai/reference/evals/run-evals)
|
|
100
|
+
- [Rubric scorer](https://mastra.ai/reference/evals/rubric)
|
|
101
|
+
- [createScorer](https://mastra.ai/reference/evals/create-scorer)
|
package/.docs/reference/index.md
CHANGED
|
@@ -152,6 +152,7 @@ The Reference section provides documentation of Mastra's API, including paramete
|
|
|
152
152
|
- [Faithfulness](https://mastra.ai/reference/evals/faithfulness)
|
|
153
153
|
- [Hallucination](https://mastra.ai/reference/evals/hallucination)
|
|
154
154
|
- [Keyword Coverage Scorer](https://mastra.ai/reference/evals/keyword-coverage)
|
|
155
|
+
- [Multi-turn Judge Scorer](https://mastra.ai/reference/evals/multi-turn-judge)
|
|
155
156
|
- [Noise Sensitivity Scorer](https://mastra.ai/reference/evals/noise-sensitivity)
|
|
156
157
|
- [Prompt Alignment Scorer](https://mastra.ai/reference/evals/prompt-alignment)
|
|
157
158
|
- [Rubric Scorer](https://mastra.ai/reference/evals/rubric)
|
|
@@ -31,9 +31,9 @@ const workspace = new Workspace({
|
|
|
31
31
|
|
|
32
32
|
**name** (`string`): Human-readable name (Default: `workspace-{id}`)
|
|
33
33
|
|
|
34
|
-
**filesystem** (`WorkspaceFilesystem | WorkspaceFilesystemResolver`): Filesystem provider instance, or a resolver function that receives requestContext and returns a filesystem per request. See
|
|
34
|
+
**filesystem** (`WorkspaceFilesystem | WorkspaceFilesystemResolver`): Filesystem provider instance, or a resolver function that receives requestContext and returns a filesystem per request. See filesystems per user or thread.
|
|
35
35
|
|
|
36
|
-
**sandbox** (`WorkspaceSandbox | WorkspaceSandboxResolver`): Sandbox provider instance, or a resolver function that receives requestContext and returns a sandbox per request. See
|
|
36
|
+
**sandbox** (`WorkspaceSandbox | WorkspaceSandboxResolver`): Sandbox provider instance, or a resolver function that receives requestContext and returns a sandbox per request. See sandboxes per user or thread.
|
|
37
37
|
|
|
38
38
|
**instructions.dynamicSandbox** (`'placeholder' | 'resolve' | (({ requestContext }) => string)`): Controls how a resolver-backed sandbox contributes to workspace instructions. 'placeholder' (default) emits stable text without calling the resolver. 'resolve' calls the resolver and uses the sandbox's own instructions. A function returns custom text without resolving. Has no effect on a static sandbox. (Default: `'placeholder'`)
|
|
39
39
|
|
|
@@ -233,6 +233,18 @@ Initialization performs:
|
|
|
233
233
|
- Starts the sandbox provider (creates working directory, sets up isolation if configured)
|
|
234
234
|
- Indexes files from `autoIndexPaths` for search
|
|
235
235
|
|
|
236
|
+
#### `stop()`
|
|
237
|
+
|
|
238
|
+
Stop the workspace's live resources without destroying them.
|
|
239
|
+
|
|
240
|
+
```typescript
|
|
241
|
+
await workspace.stop()
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
`stop()` shuts down language servers and closes the browser, then stops the sandbox provider. Remote sandbox providers suspend or pause the sandbox so it can resume later. `LocalSandbox` kills its background processes and unmounts filesystems. Stopping isn't a teardown. The filesystem, search index, and skills keep working, and the next sandbox operation starts the sandbox again.
|
|
245
|
+
|
|
246
|
+
`mastra.shutdown()` calls `stop()` for registered workspaces, so a process restart suspends remote sandboxes instead of deleting them. To fully tear a workspace down, call `destroy()` or `mastra.removeWorkspace(id, { destroy: true })` explicitly.
|
|
247
|
+
|
|
236
248
|
#### `destroy()`
|
|
237
249
|
|
|
238
250
|
Destroy the workspace and clean up resources.
|
|
@@ -243,7 +255,7 @@ await workspace.destroy()
|
|
|
243
255
|
|
|
244
256
|
`destroy()` closes workspace-owned resources in order: language servers, browsers, sandbox providers, and filesystem providers. It also clears cached sandbox references.
|
|
245
257
|
|
|
246
|
-
Call `destroy()` when your application is done with a workspace
|
|
258
|
+
Call `destroy()` when your application is done with a workspace and its sandbox. To remove a workspace from the Mastra registry, use [`mastra.removeWorkspace()`](https://mastra.ai/reference/core/removeWorkspace).
|
|
247
259
|
|
|
248
260
|
`LocalFilesystem.destroy()` doesn't delete files on disk. Resolver-backed filesystem and sandbox providers are owned by your application and must be cleaned up by your application.
|
|
249
261
|
|
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,26 @@
|
|
|
1
1
|
# @mastra/mcp-docs-server
|
|
2
2
|
|
|
3
|
+
## 1.2.19-alpha.4
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- Updated dependencies [[`e737014`](https://github.com/mastra-ai/mastra/commit/e737014e0fc7035759762bb5b48baef1d6c0f6a7), [`d6ce34a`](https://github.com/mastra-ai/mastra/commit/d6ce34aeceb06ddf3d595a1eed5cc74f481a46a1), [`e6f8450`](https://github.com/mastra-ai/mastra/commit/e6f845074d478527026b18d85031b23353e1d0a4)]:
|
|
8
|
+
- @mastra/core@1.62.0-alpha.2
|
|
9
|
+
|
|
10
|
+
## 1.2.19-alpha.2
|
|
11
|
+
|
|
12
|
+
### Patch Changes
|
|
13
|
+
|
|
14
|
+
- Updated dependencies [[`f95f468`](https://github.com/mastra-ai/mastra/commit/f95f468cf1e7c2b924a13826494f98b8f2ccd581)]:
|
|
15
|
+
- @mastra/core@1.61.1-alpha.1
|
|
16
|
+
|
|
17
|
+
## 1.2.19-alpha.1
|
|
18
|
+
|
|
19
|
+
### Patch Changes
|
|
20
|
+
|
|
21
|
+
- Updated dependencies [[`1e47b75`](https://github.com/mastra-ai/mastra/commit/1e47b7520cab4cfaa8daed52f17e2e6d14ff7539)]:
|
|
22
|
+
- @mastra/core@1.61.1-alpha.0
|
|
23
|
+
|
|
3
24
|
## 1.2.18
|
|
4
25
|
|
|
5
26
|
### Patch Changes
|