@mastra/mysql 0.8.0 → 0.8.1-alpha.1

package/CHANGELOG.md

@@ -1,5 +1,60 @@
 # @mastra/mysql
 
+## 0.8.1-alpha.1
+
+### Patch Changes
+
+- Added HTTP endpoints for caller-driven experiments: `POST /datasets/:datasetId/experiments` now accepts `start: false` to create an experiment without spawning the runner (with an optional target and run-level `scorerIds`), and new routes `POST /datasets/:datasetId/experiments/:experimentId/items/:itemId/run`, `POST /datasets/:datasetId/experiments/:experimentId/results`, and `POST /datasets/:datasetId/experiments/:experimentId/finalize` let external orchestrators run one item server-side, submit externally computed per-item results (idempotent upsert on retries), and finalize the run with server-computed counts. ([#21888](https://github.com/mastra-ai/mastra/pull/21888))
+
+- Added `createDatasetExperiment()`, `runExperimentItem()`, `submitExperimentResult()`, and `finalizeExperiment()` methods so a caller-owned orchestrator (for example Temporal) can drive an experiment loop while Mastra either executes each item server-side or ingests externally computed results. ([#21888](https://github.com/mastra-ai/mastra/pull/21888))
+
+- Added `upsertExperimentResult()` to the experiments storage domain plus an `attempt` column on experiment results and a nullable target with an optional `scorerIds` column on experiments, enabling retry-safe result writes for caller-driven experiments (retried submissions with the same `(experimentId, itemId, attempt)` key converge on a single row). `saveScore()` now accepts an optional caller-supplied `id` and upserts on it, so retried experiment submissions replace their previous score rows (latest wins) instead of accumulating duplicates. ([#21888](https://github.com/mastra-ai/mastra/pull/21888))
+
+- Added caller-driven experiments so an external orchestrator (for example Temporal workers) can own the experiment loop while Mastra stays the system of record. ([#21888](https://github.com/mastra-ai/mastra/pull/21888))
+
+  Create an experiment with `dataset.createExperiment()` (idempotent when you pass your own id). With a target, Mastra runs each item for you: call `dataset.runExperimentItem()` per item and Mastra executes the registered agent or workflow, resolves scorers (experiment `scorers`, falling back to item `scorerIds`, then dataset `scorerIds`), and upserts the result. Without a target, run everything yourself and report per-item results with `dataset.submitExperimentResult()` (upsert semantics on `(experimentId, itemId, attempt)` so retried workers converge on a single row). Either way, close the run with `dataset.finalizeExperiment()` and Mastra computes per-item succeeded/failed/skipped counts from the persisted rows. Results go into the same storage as native runs, so Studio views, comparisons, and review summaries work unchanged.
+
+  ```typescript
+  // Caller drives the loop, Mastra runs each item
+  const { experimentId } = await dataset.createExperiment({
+    id: workflowRunId,
+    targetType: 'agent',
+    targetId: 'support-agent',
+    scorers: ['accuracy'],
+  });
+
+  await dataset.runExperimentItem({ experimentId, itemId });
+
+  // Or: caller runs everything, Mastra ingests results
+  const ingest = await dataset.createExperiment({ id: workflowRunId });
+  await dataset.submitExperimentResult({
+    experimentId: ingest.experimentId,
+    itemId,
+    output,
+    scores: [{ scorerId: 'accuracy', score: 0.92 }],
+  });
+
+  const experiment = await dataset.finalizeExperiment({ experimentId });
+  ```
+
+- Fixed versioned dataset item lookups to return the item visible in the requested dataset snapshot. ([#21979](https://github.com/mastra-ai/mastra/pull/21979))
+
+- Updated dependencies [[`9267e9b`](https://github.com/mastra-ai/mastra/commit/9267e9b3d9c2fcf16936050495a787054c2431ab), [`acc3471`](https://github.com/mastra-ai/mastra/commit/acc3471de5f3fde8027ee4e355af292b2bc1bc30), [`b6a771e`](https://github.com/mastra-ai/mastra/commit/b6a771ef23d203ddb348efca8065eff65def8191), [`9267e9b`](https://github.com/mastra-ai/mastra/commit/9267e9b3d9c2fcf16936050495a787054c2431ab), [`26d4016`](https://github.com/mastra-ai/mastra/commit/26d40160ff7f7d8bf95fee2039a52cbc83863533), [`9267e9b`](https://github.com/mastra-ai/mastra/commit/9267e9b3d9c2fcf16936050495a787054c2431ab), [`9267e9b`](https://github.com/mastra-ai/mastra/commit/9267e9b3d9c2fcf16936050495a787054c2431ab), [`57c5103`](https://github.com/mastra-ai/mastra/commit/57c51035a2a36e3df3c4f32f46bb789a66ed5946)]:
+  - @mastra/core@1.61.0-alpha.3
+
+## 0.8.1-alpha.0
+
+### Patch Changes
+
+- Fixed concurrent resume() calls on the same suspended workflow run executing downstream steps more than once. A resume now atomically claims the run before executing anything, so only one caller continues a given suspension. Losing callers throw WORKFLOW_RESUME_ALREADY_CLAIMED without running any steps. Fixes #20443 ([#21725](https://github.com/mastra-ai/mastra/pull/21725))
+
+- Workflow state updates now support an optional expectedStatus guard, so a status change is only applied when the stored run is in an expected state. This is what makes concurrent workflow resumes safe. ([#21725](https://github.com/mastra-ai/mastra/pull/21725))
+
+- Resume conflicts now return 409 Conflict. When a suspended workflow run has already been resumed by another caller, the resume endpoints respond with 409 instead of a generic error. ([#21725](https://github.com/mastra-ai/mastra/pull/21725))
+
+- Updated dependencies [[`88d14ca`](https://github.com/mastra-ai/mastra/commit/88d14cac008582a618fecc3d5c7fd3bdf4f6ddc3), [`84a5b69`](https://github.com/mastra-ai/mastra/commit/84a5b699f84d6bae0a34efe5a970d891090b9f41), [`84a5b69`](https://github.com/mastra-ai/mastra/commit/84a5b699f84d6bae0a34efe5a970d891090b9f41), [`84a5b69`](https://github.com/mastra-ai/mastra/commit/84a5b699f84d6bae0a34efe5a970d891090b9f41), [`038b7b4`](https://github.com/mastra-ai/mastra/commit/038b7b405cb4ac25ab3f3031334111b1f87ac112), [`4132d61`](https://github.com/mastra-ai/mastra/commit/4132d61f8367077120ee9e6420d3224dffd93c93)]:
+  - @mastra/core@1.60.1-alpha.0
+
 ## 0.8.0
 
 ### Minor Changes

package/dist/index.cjs

@@ -2577,7 +2577,11 @@ var DatasetsMySQL = class DatasetsMySQL extends _mastra_core_storage.DatasetsSto
 		try {
 			const tableItemsName = formatTableName(_mastra_core_storage.TABLE_DATASET_ITEMS);
 			let rows;
-			if (args.datasetVersion !== void 0) [rows] = await this.pool.execute(`SELECT * FROM ${tableItemsName} WHERE \`id\` = ? AND \`datasetVersion\` = ? AND \`isDeleted\` = 0`, [args.id, args.datasetVersion]);
+			if (args.datasetVersion !== void 0) [rows] = await this.pool.execute(`SELECT * FROM ${tableItemsName} WHERE \`id\` = ? AND \`datasetVersion\` <= ? AND (\`validTo\` IS NULL OR \`validTo\` > ?) AND \`isDeleted\` = 0 ORDER BY \`datasetVersion\` DESC LIMIT 1`, [
+				args.id,
+				args.datasetVersion,
+				args.datasetVersion
+			]);
 			else [rows] = await this.pool.execute(`SELECT * FROM ${tableItemsName} WHERE \`id\` = ? AND \`validTo\` IS NULL AND \`isDeleted\` = 0`, [args.id]);
 			return rows.length > 0 ? this.mapItem(rows[0]) : null;
 		} catch (error) {
@@ -3014,7 +3018,8 @@ var ExperimentsMySQL = class ExperimentsMySQL extends _mastra_core_storage.Exper
 				"experimentSetId",
 				"comparisonId",
 				"variantId",
-				"trialIndex"
+				"trialIndex",
+				"scorerIds"
 			]
 		});
 		await this.operations.alterTable({
@@ -3023,7 +3028,8 @@ var ExperimentsMySQL = class ExperimentsMySQL extends _mastra_core_storage.Exper
 			ifNotExists: [
 				"comment",
 				"organizationId",
-				"projectId"
+				"projectId",
+				"attempt"
 			]
 		});
 		await this.createDefaultIndexes();
@@ -3041,8 +3047,9 @@ var ExperimentsMySQL = class ExperimentsMySQL extends _mastra_core_storage.Exper
 			agentVersion: row.agentVersion ?? null,
 			organizationId: row.organizationId ?? null,
 			projectId: row.projectId ?? null,
-			targetType: row.targetType,
-			targetId: row.targetId,
+			targetType: row.targetType ?? null,
+			targetId: row.targetId ?? null,
+			scorerIds: parseJSON$2(row.scorerIds) ?? null,
 			name: row.name ?? void 0,
 			description: row.description ?? void 0,
 			metadata: parseJSON$2(row.metadata),
@@ -3078,6 +3085,7 @@ var ExperimentsMySQL = class ExperimentsMySQL extends _mastra_core_storage.Exper
 			startedAt: parseDateTime(row.startedAt) ?? /* @__PURE__ */ new Date(),
 			completedAt: parseDateTime(row.completedAt) ?? /* @__PURE__ */ new Date(),
 			retryCount: row.retryCount,
+			attempt: row.attempt != null ? Number(row.attempt) : 0,
 			traceId: row.traceId ?? null,
 			status: row.status ?? null,
 			tags: row.tags ? parseJSON$2(row.tags) ?? null : null,
@@ -3098,8 +3106,9 @@ var ExperimentsMySQL = class ExperimentsMySQL extends _mastra_core_storage.Exper
 					agentVersion: input.agentVersion ?? null,
 					organizationId: input.organizationId ?? null,
 					projectId: input.projectId ?? null,
-					targetType: input.targetType,
-					targetId: input.targetId,
+					targetType: input.targetType ?? null,
+					targetId: input.targetId ?? null,
+					scorerIds: input.scorerIds ? JSON.stringify(input.scorerIds) : null,
 					name: input.name ?? null,
 					description: input.description ?? null,
 					metadata: input.metadata ? JSON.stringify(input.metadata) : null,
@@ -3127,8 +3136,9 @@ var ExperimentsMySQL = class ExperimentsMySQL extends _mastra_core_storage.Exper
 				agentVersion: input.agentVersion ?? null,
 				organizationId: input.organizationId ?? null,
 				projectId: input.projectId ?? null,
-				targetType: input.targetType,
-				targetId: input.targetId,
+				targetType: input.targetType ?? null,
+				targetId: input.targetId ?? null,
+				scorerIds: input.scorerIds ?? null,
 				name: input.name,
 				description: input.description,
 				metadata: input.metadata,
@@ -3372,12 +3382,13 @@ var ExperimentsMySQL = class ExperimentsMySQL extends _mastra_core_storage.Exper
 					organizationId: input.organizationId ?? null,
 					projectId: input.projectId ?? null,
 					input: JSON.stringify(input.input),
-					output: input.output ? JSON.stringify(input.output) : null,
-					groundTruth: input.groundTruth ? JSON.stringify(input.groundTruth) : null,
+					output: input.output != null ? JSON.stringify(input.output) : null,
+					groundTruth: input.groundTruth != null ? JSON.stringify(input.groundTruth) : null,
 					error: input.error ? JSON.stringify(input.error) : null,
 					startedAt: input.startedAt,
 					completedAt: input.completedAt,
 					retryCount: input.retryCount,
+					attempt: input.attempt ?? 0,
 					traceId: input.traceId ?? null,
 					status: input.status ?? null,
 					tags: input.tags ? JSON.stringify(input.tags) : null,
@@ -3398,6 +3409,7 @@ var ExperimentsMySQL = class ExperimentsMySQL extends _mastra_core_storage.Exper
 				startedAt: input.startedAt,
 				completedAt: input.completedAt,
 				retryCount: input.retryCount,
+				attempt: input.attempt ?? 0,
 				traceId: input.traceId ?? null,
 				status: input.status ?? null,
 				tags: input.tags ?? null,
@@ -3430,6 +3442,66 @@ var ExperimentsMySQL = class ExperimentsMySQL extends _mastra_core_storage.Exper
 			}, error);
 		}
 	}
+	async upsertExperimentResult(input) {
+		if (input.toolMockReport) throw new _mastra_core_error.MastraError({
+			id: "MYSQL_EXPERIMENT_TOOL_MOCK_REPORT_UNSUPPORTED",
+			domain: _mastra_core_error.ErrorDomain.STORAGE,
+			category: _mastra_core_error.ErrorCategory.USER,
+			text: "Tool mock reports are not supported on the MySQL storage adapter. Use a supported adapter (LibSQL, PostgreSQL, MongoDB, or Spanner) to persist experiment tool mock reports."
+		});
+		try {
+			const attempt = input.attempt ?? 0;
+			const existingId = (await this.operations.query(`SELECT ${quoteIdentifier("id", "column name")} FROM ${formatTableName(_mastra_core_storage.TABLE_EXPERIMENT_RESULTS)} WHERE ${quoteIdentifier("experimentId", "column name")} = ? AND ${quoteIdentifier("itemId", "column name")} = ? AND COALESCE(${quoteIdentifier("attempt", "column name")}, 0) = ?`, [
+				input.experimentId,
+				input.itemId,
+				attempt
+			]))[0]?.id;
+			if (!existingId) return await this.addExperimentResult({
+				...input,
+				attempt
+			});
+			await this.operations.update({
+				tableName: _mastra_core_storage.TABLE_EXPERIMENT_RESULTS,
+				keys: { id: existingId },
+				data: {
+					itemDatasetVersion: input.itemDatasetVersion ?? null,
+					organizationId: input.organizationId ?? null,
+					projectId: input.projectId ?? null,
+					input: JSON.stringify(input.input),
+					output: input.output != null ? JSON.stringify(input.output) : null,
+					groundTruth: input.groundTruth != null ? JSON.stringify(input.groundTruth) : null,
+					error: input.error ? JSON.stringify(input.error) : null,
+					startedAt: input.startedAt,
+					completedAt: input.completedAt,
+					retryCount: input.retryCount,
+					attempt,
+					traceId: input.traceId ?? null,
+					status: input.status ?? null,
+					tags: input.tags ? JSON.stringify(input.tags) : null
+				}
+			});
+			const updated = await this.getExperimentResultById({ id: existingId });
+			if (!updated) throw new _mastra_core_error.MastraError({
+				id: "MYSQL_UPSERT_EXPERIMENT_RESULT_NOT_FOUND",
+				domain: _mastra_core_error.ErrorDomain.STORAGE,
+				category: _mastra_core_error.ErrorCategory.USER,
+				text: `Experiment result ${existingId} not found after upsert`,
+				details: {
+					experimentId: input.experimentId,
+					itemId: input.itemId,
+					attempt
+				}
+			});
+			return updated;
+		} catch (error) {
+			if (error instanceof _mastra_core_error.MastraError) throw error;
+			throw new _mastra_core_error.MastraError({
+				id: "MYSQL_UPSERT_EXPERIMENT_RESULT_FAILED",
+				domain: _mastra_core_error.ErrorDomain.STORAGE,
+				category: _mastra_core_error.ErrorCategory.THIRD_PARTY
+			}, error);
+		}
+	}
 	async updateExperimentResult(input) {
 		try {
 			const existing = await this.operations.load({
@@ -9315,7 +9387,7 @@ var ScoresMySQL = class ScoresMySQL extends _mastra_core_storage.ScoresStorage {
 	}
 	async saveScore(score) {
 		try {
-			const id = crypto.randomUUID();
+			const id = score.id ?? crypto.randomUUID();
 			const now = /* @__PURE__ */ new Date();
 			await this.operations.insert({
 				tableName: _mastra_core_storage.TABLE_SCORERS,
@@ -10529,9 +10601,15 @@ var WorkflowsMySQL = class WorkflowsMySQL extends _mastra_core_storage.Workflows
 				await connection.rollback();
 				return;
 			}
+			const existing = parseSnapshot(rows[0].snapshot);
+			const { expectedStatus, ...state } = opts;
+			if (!(0, _mastra_core_storage.matchesExpectedWorkflowStatus)(existing.status, expectedStatus)) {
+				await connection.rollback();
+				return;
+			}
 			const updatedSnapshot = {
-				...parseSnapshot(rows[0].snapshot),
-				...opts
+				...existing,
+				...state
 			};
 			await connection.execute(`UPDATE ${formatTableName(_mastra_core_storage.TABLE_WORKFLOW_SNAPSHOT)} SET ${quoteIdentifier("snapshot", "column name")} = ?, ${quoteIdentifier("updatedAt", "column name")} = ? WHERE ${quoteIdentifier("workflow_name", "column name")} = ? AND ${quoteIdentifier("run_id", "column name")} = ?`, [
 				JSON.stringify(updatedSnapshot),