@mastra/evals 1.3.0-alpha.0 → 1.3.1-alpha.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/CHANGELOG.md +41 -0
  2. package/dist/docs/SKILL.md +1 -1
  3. package/dist/docs/assets/SOURCE_MAP.json +1 -1
  4. package/package.json +11 -11
package/CHANGELOG.md CHANGED
@@ -1,5 +1,46 @@
1
1
  # @mastra/evals
2
2
 
3
+ ## 1.3.1-alpha.0
4
+
5
+ ### Patch Changes
6
+
7
+ - Security remediation for the 2026-06-17 "easy-day-js" supply-chain incident. Patch bump to publish clean versions and move the `latest` dist-tag forward, superseding the compromised versions that declared the malicious `easy-day-js` dependency. ([#18056](https://github.com/mastra-ai/mastra/pull/18056))
8
+
9
+ - Updated dependencies [[`77a2351`](https://github.com/mastra-ai/mastra/commit/77a2351ee79296e360bce822cb3391f7cfd6489d)]:
10
+ - @mastra/core@1.43.1-alpha.0
11
+
12
+ ## 1.3.0
13
+
14
+ ### Minor Changes
15
+
16
+ - Added `createRubricScorer`, an LLM-as-judge scorer that grades agent output against a rubric of criteria and returns a binary verdict (1 only when every required criterion is satisfied) with per-criterion feedback. Drop it into `isTaskComplete` to make an agent self-correct until the rubric is met. ([#17724](https://github.com/mastra-ai/mastra/pull/17724))
17
+
18
+ ```typescript
19
+ import { createRubricScorer } from '@mastra/evals/scorers/prebuilt';
20
+
21
+ const rubricScorer = createRubricScorer({
22
+ model: '__GATEWAY_OPENAI_MODEL_MINI__',
23
+ criteria: [
24
+ { description: 'The response includes an analysis section' },
25
+ { description: 'The response includes concrete recommendations' },
26
+ ],
27
+ });
28
+
29
+ await supervisor.stream('Research AI in education', {
30
+ maxSteps: 10,
31
+ isTaskComplete: { scorers: [rubricScorer], strategy: 'all' },
32
+ });
33
+ ```
34
+
35
+ The rubric accepts a criteria array or a newline-delimited string, supports optional (non-gating) criteria, and can be supplied per run via request context under a `rubric` key.
36
+
37
+ ### Patch Changes
38
+
39
+ - Fixed AnswerRelevancyScorer (and other scorers) scoring live-agent turns 0. The scorer's user-input extractor ignored the `parts` array of format-2 messages, so when an agent persisted a message without the optional `content` string the input came through empty and every statement was judged irrelevant. The extractor now reads text from `parts` (last text part wins), matching the assistant-output extractor. ([#17769](https://github.com/mastra-ai/mastra/pull/17769))
40
+
41
+ - Updated dependencies [[`d468acb`](https://github.com/mastra-ai/mastra/commit/d468acb07aec1bb19a2cb0ada8042b05b46746b2), [`575f815`](https://github.com/mastra-ai/mastra/commit/575f815c5c3567b71c0b83cbb7fa98c8253a9d9c), [`34839c1`](https://github.com/mastra-ai/mastra/commit/34839c1910b6964bf59ed0cee58844efebbb684e), [`053735a`](https://github.com/mastra-ai/mastra/commit/053735a75c2c18e23ce34d9468007efa4a45f4c4), [`306909a`](https://github.com/mastra-ai/mastra/commit/306909a693de77d709b38706e2673c9547d24a28), [`5191af8`](https://github.com/mastra-ai/mastra/commit/5191af80c799eea25357c545fc05d91b3883531d), [`43bd3d4`](https://github.com/mastra-ai/mastra/commit/43bd3d421987463fdf35386a45199c49499ed069), [`e6fa79e`](https://github.com/mastra-ai/mastra/commit/e6fa79ec72a2ddffdd25e85270398951e9d552a4), [`904bcdf`](https://github.com/mastra-ai/mastra/commit/904bcdf7b8004aa7be823f9f70ca63580e47e470), [`7f5ee1d`](https://github.com/mastra-ai/mastra/commit/7f5ee1dca46daee8d2817f2ebe49e6335da81956), [`1e9aab5`](https://github.com/mastra-ai/mastra/commit/1e9aab50ff11e6e88fde4d7cbf512c44a9fe8d61), [`2bccba4`](https://github.com/mastra-ai/mastra/commit/2bccba4c03cadc815c2d54cbf4dd43a922140a8d), [`bf8eb6d`](https://github.com/mastra-ai/mastra/commit/bf8eb6d0ec213a403eb9265a594ad283c44ab3dc), [`e9be4e7`](https://github.com/mastra-ai/mastra/commit/e9be4e747ec3d8b65548bff92f9377db06105376), [`493a328`](https://github.com/mastra-ai/mastra/commit/493a328f4346a1deeb9f1e2e44c8f2a3a4d7591b), [`d53cfc2`](https://github.com/mastra-ai/mastra/commit/d53cfc2c7f8d78343a4aa84ec4e129ba25f3325e), [`65799d4`](https://github.com/mastra-ai/mastra/commit/65799d4d549e5ebb9c848fbe3f51ac090f64becf), [`c268c89`](https://github.com/mastra-ai/mastra/commit/c268c89f4c63a93ee474d3cffdf3ea60bf00d4f2), [`34839c1`](https://github.com/mastra-ai/mastra/commit/34839c1910b6964bf59ed0cee58844efebbb684e), [`014e00f`](https://github.com/mastra-ai/mastra/commit/014e00f2b3a597a016b72f9901c6ab27d491f822), [`029a414`](https://github.com/mastra-ai/mastra/commit/029a4141719793bd3e898a39eb5a0466a55f5f3a), [`d468acb`](https://github.com/mastra-ai/mastra/commit/d468acb07aec1bb19a2cb0ada8042b05b46746b2), [`b147b29`](https://github.com/mastra-ai/mastra/commit/b147b2907f0cd1aa812efe6d6e3f58d22e66fc88), [`d371ac1`](https://github.com/mastra-ai/mastra/commit/d371ac1d9820afaaf7cfdbc380a475946a994d8f), [`2bccba4`](https://github.com/mastra-ai/mastra/commit/2bccba4c03cadc815c2d54cbf4dd43a922140a8d), [`0c72f03`](https://github.com/mastra-ai/mastra/commit/0c72f032abb13254df5a7856d64be2f207b8006d), [`cf182b7`](https://github.com/mastra-ai/mastra/commit/cf182b7fb495767946d9840ef29f19cfa906f31f), [`3b45ea9`](https://github.com/mastra-ai/mastra/commit/3b45ea95015557a6cb9d70dc5252af54ab1b78ac), [`a049c2a`](https://github.com/mastra-ai/mastra/commit/a049c2a9dfb41d0ee2e7a28874a88cd64fd5669f), [`f084be1`](https://github.com/mastra-ai/mastra/commit/f084be1fcbe33ad7480913e44d6130c421c0976f), [`b147b29`](https://github.com/mastra-ai/mastra/commit/b147b2907f0cd1aa812efe6d6e3f58d22e66fc88), [`2a96528`](https://github.com/mastra-ai/mastra/commit/2a9652848dfa3c5a2426f952e9d93554c26fd90f), [`f2ab060`](https://github.com/mastra-ai/mastra/commit/f2ab060162bea81505fda553e2cee29c1979fd04), [`5d302c8`](https://github.com/mastra-ai/mastra/commit/5d302c8eda1a6ac74eab5e442c4f64db6cc97a06), [`34839c1`](https://github.com/mastra-ai/mastra/commit/34839c1910b6964bf59ed0cee58844efebbb684e), [`a952852`](https://github.com/mastra-ai/mastra/commit/a952852c971a21fb646cd907c75fcf4443cdc963), [`2656d9c`](https://github.com/mastra-ai/mastra/commit/2656d9c2976d4f3354253bfbbbf9b88a1b2bbf34), [`63e3fe1`](https://github.com/mastra-ai/mastra/commit/63e3fe13cc1ea96f91d7c68aea92f400faf9e4da), [`1d4ce8d`](https://github.com/mastra-ai/mastra/commit/1d4ce8daaa54511f325c1b609d31b8e54009d677), [`8c68372`](https://github.com/mastra-ai/mastra/commit/8c68372e85fe0b066ec12c58bd29ffb93e54c552)]:
42
+ - @mastra/core@1.42.0
43
+
3
44
  ## 1.3.0-alpha.0
4
45
 
5
46
  ### Minor Changes
package/dist/docs/SKILL.md CHANGED
@@ -3,7 +3,7 @@ name: mastra-evals
3
3
  description: Documentation for @mastra/evals. Use when working with @mastra/evals APIs, configuration, or implementation.
4
4
  metadata:
5
5
  package: "@mastra/evals"
6
- version: "1.3.0-alpha.0"
6
+ version: "1.3.1-alpha.0"
7
7
  ---
8
8
 
9
9
  ## When to use
package/dist/docs/assets/SOURCE_MAP.json CHANGED
@@ -1,5 +1,5 @@
1
1
  {
2
- "version": "1.3.0-alpha.0",
2
+ "version": "1.3.1-alpha.0",
3
3
  "package": "@mastra/evals",
4
4
  "exports": {},
5
5
  "modules": {}
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@mastra/evals",
3
- "version": "1.3.0-alpha.0",
3
+ "version": "1.3.1-alpha.0",
4
4
  "description": "",
5
5
  "type": "module",
6
6
  "files": [
@@ -67,21 +67,21 @@
67
67
  "@ai-sdk/openai": "^1.3.24",
68
68
  "@types/sentiment": "^5.0.4",
69
69
  "@types/string-similarity": "^4.0.2",
70
- "@vitest/coverage-v8": "4.1.5",
71
- "@vitest/ui": "4.1.5",
72
- "dotenv": "^17.3.1",
70
+ "@vitest/coverage-v8": "4.1.8",
71
+ "@vitest/ui": "4.1.8",
72
+ "dotenv": "^17.4.2",
73
73
  "eslint": "^10.4.1",
74
74
  "tsup": "^8.5.1",
75
75
  "tsx": "^4.22.4",
76
76
  "typescript": "^6.0.3",
77
- "vitest": "4.1.5",
77
+ "vitest": "4.1.8",
78
78
  "zod": "^4.4.3",
79
- "@internal/ai-sdk-v5": "0.0.50",
80
- "@internal/llm-recorder": "0.0.39",
81
- "@internal/types-builder": "0.0.78",
82
- "@internal/lint": "0.0.103",
83
- "@internal/test-utils": "0.0.39",
84
- "@mastra/core": "1.42.0-alpha.4"
79
+ "@internal/ai-sdk-v5": "0.0.52",
80
+ "@internal/test-utils": "0.0.41",
81
+ "@internal/lint": "0.0.105",
82
+ "@internal/types-builder": "0.0.80",
83
+ "@mastra/core": "1.43.1-alpha.0",
84
+ "@internal/llm-recorder": "0.0.41"
85
85
  },
86
86
  "engines": {
87
87
  "node": ">=22.13.0"