@mastra/mongodb 1.18.0 → 1.18.1-alpha.1

package/CHANGELOG.md

@@ -1,5 +1,58 @@
 # @mastra/mongodb
 
+## 1.18.1-alpha.1
+
+### Patch Changes
+
+- Added HTTP endpoints for caller-driven experiments: `POST /datasets/:datasetId/experiments` now accepts `start: false` to create an experiment without spawning the runner (with an optional target and run-level `scorerIds`), and new routes `POST /datasets/:datasetId/experiments/:experimentId/items/:itemId/run`, `POST /datasets/:datasetId/experiments/:experimentId/results`, and `POST /datasets/:datasetId/experiments/:experimentId/finalize` let external orchestrators run one item server-side, submit externally computed per-item results (idempotent upsert on retries), and finalize the run with server-computed counts. ([#21888](https://github.com/mastra-ai/mastra/pull/21888))
+
+- Added `createDatasetExperiment()`, `runExperimentItem()`, `submitExperimentResult()`, and `finalizeExperiment()` methods so a caller-owned orchestrator (for example Temporal) can drive an experiment loop while Mastra either executes each item server-side or ingests externally computed results. ([#21888](https://github.com/mastra-ai/mastra/pull/21888))
+
+- Added `upsertExperimentResult()` to the experiments storage domain plus an `attempt` column on experiment results and a nullable target with an optional `scorerIds` column on experiments, enabling retry-safe result writes for caller-driven experiments (retried submissions with the same `(experimentId, itemId, attempt)` key converge on a single row). `saveScore()` now accepts an optional caller-supplied `id` and upserts on it, so retried experiment submissions replace their previous score rows (latest wins) instead of accumulating duplicates. ([#21888](https://github.com/mastra-ai/mastra/pull/21888))
+
+- Added caller-driven experiments so an external orchestrator (for example Temporal workers) can own the experiment loop while Mastra stays the system of record. ([#21888](https://github.com/mastra-ai/mastra/pull/21888))
+
+  Create an experiment with `dataset.createExperiment()` (idempotent when you pass your own id). With a target, Mastra runs each item for you: call `dataset.runExperimentItem()` per item and Mastra executes the registered agent or workflow, resolves scorers (experiment `scorers`, falling back to item `scorerIds`, then dataset `scorerIds`), and upserts the result. Without a target, run everything yourself and report per-item results with `dataset.submitExperimentResult()` (upsert semantics on `(experimentId, itemId, attempt)` so retried workers converge on a single row). Either way, close the run with `dataset.finalizeExperiment()` and Mastra computes per-item succeeded/failed/skipped counts from the persisted rows. Results go into the same storage as native runs, so Studio views, comparisons, and review summaries work unchanged.
+
+  ```typescript
+  // Caller drives the loop, Mastra runs each item
+  const { experimentId } = await dataset.createExperiment({
+    id: workflowRunId,
+    targetType: 'agent',
+    targetId: 'support-agent',
+    scorers: ['accuracy'],
+  });
+
+  await dataset.runExperimentItem({ experimentId, itemId });
+
+  // Or: caller runs everything, Mastra ingests results
+  const ingest = await dataset.createExperiment({ id: workflowRunId });
+  await dataset.submitExperimentResult({
+    experimentId: ingest.experimentId,
+    itemId,
+    output,
+    scores: [{ scorerId: 'accuracy', score: 0.92 }],
+  });
+
+  const experiment = await dataset.finalizeExperiment({ experimentId });
+  ```
+
+- Updated dependencies [[`9267e9b`](https://github.com/mastra-ai/mastra/commit/9267e9b3d9c2fcf16936050495a787054c2431ab), [`acc3471`](https://github.com/mastra-ai/mastra/commit/acc3471de5f3fde8027ee4e355af292b2bc1bc30), [`b6a771e`](https://github.com/mastra-ai/mastra/commit/b6a771ef23d203ddb348efca8065eff65def8191), [`9267e9b`](https://github.com/mastra-ai/mastra/commit/9267e9b3d9c2fcf16936050495a787054c2431ab), [`26d4016`](https://github.com/mastra-ai/mastra/commit/26d40160ff7f7d8bf95fee2039a52cbc83863533), [`9267e9b`](https://github.com/mastra-ai/mastra/commit/9267e9b3d9c2fcf16936050495a787054c2431ab), [`9267e9b`](https://github.com/mastra-ai/mastra/commit/9267e9b3d9c2fcf16936050495a787054c2431ab), [`57c5103`](https://github.com/mastra-ai/mastra/commit/57c51035a2a36e3df3c4f32f46bb789a66ed5946)]:
+  - @mastra/core@1.61.0-alpha.3
+
+## 1.18.1-alpha.0
+
+### Patch Changes
+
+- Fixed concurrent resume() calls on the same suspended workflow run executing downstream steps more than once. A resume now atomically claims the run before executing anything, so only one caller continues a given suspension. Losing callers throw WORKFLOW_RESUME_ALREADY_CLAIMED without running any steps. Fixes #20443 ([#21725](https://github.com/mastra-ai/mastra/pull/21725))
+
+- Workflow state updates now support an optional expectedStatus guard, so a status change is only applied when the stored run is in an expected state. This is what makes concurrent workflow resumes safe. ([#21725](https://github.com/mastra-ai/mastra/pull/21725))
+
+- Resume conflicts now return 409 Conflict. When a suspended workflow run has already been resumed by another caller, the resume endpoints respond with 409 instead of a generic error. ([#21725](https://github.com/mastra-ai/mastra/pull/21725))
+
+- Updated dependencies [[`88d14ca`](https://github.com/mastra-ai/mastra/commit/88d14cac008582a618fecc3d5c7fd3bdf4f6ddc3), [`84a5b69`](https://github.com/mastra-ai/mastra/commit/84a5b699f84d6bae0a34efe5a970d891090b9f41), [`84a5b69`](https://github.com/mastra-ai/mastra/commit/84a5b699f84d6bae0a34efe5a970d891090b9f41), [`84a5b69`](https://github.com/mastra-ai/mastra/commit/84a5b699f84d6bae0a34efe5a970d891090b9f41), [`038b7b4`](https://github.com/mastra-ai/mastra/commit/038b7b405cb4ac25ab3f3031334111b1f87ac112), [`4132d61`](https://github.com/mastra-ai/mastra/commit/4132d61f8367077120ee9e6420d3224dffd93c93)]:
+  - @mastra/core@1.60.1-alpha.0
+
 ## 1.18.0
 
 ### Minor Changes

package/dist/docs/SKILL.md

@@ -3,7 +3,7 @@ name: mastra-mongodb
 description: Documentation for @mastra/mongodb. Use when working with @mastra/mongodb APIs, configuration, or implementation.
 metadata:
   package: "@mastra/mongodb"
-  version: "1.18.0"
+  version: "1.18.1-alpha.1"
 ---
 
 ## When to use

package/dist/docs/assets/SOURCE_MAP.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.18.0",
+  "version": "1.18.1-alpha.1",
   "package": "@mastra/mongodb",
   "exports": {},
   "modules": {}

package/dist/docs/references/reference-rag-retrieval.md

@@ -77,9 +77,19 @@ For detailed information about available operators and syntax, see the [Metadata
 
 Basic filtering examples:
 
+**MongoDB**:
+
 ```ts
+import { MongoDBVector } from '@mastra/mongodb'
+
+const mongoVector = new MongoDBVector({
+  id: 'mongodb-vector',
+  uri: process.env.MONGODB_URI,
+  dbName: process.env.MONGODB_DB_NAME,
+})
+
 // Simple equality filter
-const results = await pgVector.query({
+const equalityResults = await mongoVector.query({
   indexName: 'embeddings',
   queryVector: embedding,
   topK: 10,
@@ -89,7 +99,7 @@ const results = await pgVector.query({
 })
 
 // Numeric comparison
-const results = await pgVector.query({
+const priceResults = await mongoVector.query({
   indexName: 'embeddings',
   queryVector: embedding,
   topK: 10,
@@ -99,7 +109,7 @@ const results = await pgVector.query({
 })
 
 // Multiple conditions
-const results = await pgVector.query({
+const compoundResults = await mongoVector.query({
   indexName: 'embeddings',
   queryVector: embedding,
   topK: 10,
@@ -111,7 +121,7 @@ const results = await pgVector.query({
 })
 
 // Array operations
-const results = await pgVector.query({
+const tagResults = await mongoVector.query({
   indexName: 'embeddings',
   queryVector: embedding,
   topK: 10,
@@ -121,7 +131,64 @@ const results = await pgVector.query({
 })
 
 // Logical operators
-const results = await pgVector.query({
+const categoryResults = await mongoVector.query({
+  indexName: 'embeddings',
+  queryVector: embedding,
+  topK: 10,
+  filter: {
+    $or: [{ category: 'electronics' }, { category: 'accessories' }],
+    $and: [{ price: { $gt: 50 } }, { price: { $lt: 200 } }],
+  },
+})
+```
+
+**pgVector**:
+
+```ts
+// Simple equality filter
+const equalityResults = await pgVector.query({
+  indexName: 'embeddings',
+  queryVector: embedding,
+  topK: 10,
+  filter: {
+    source: 'article1.txt',
+  },
+})
+
+// Numeric comparison
+const priceResults = await pgVector.query({
+  indexName: 'embeddings',
+  queryVector: embedding,
+  topK: 10,
+  filter: {
+    price: { $gt: 100 },
+  },
+})
+
+// Multiple conditions
+const compoundResults = await pgVector.query({
+  indexName: 'embeddings',
+  queryVector: embedding,
+  topK: 10,
+  filter: {
+    category: 'electronics',
+    price: { $lt: 1000 },
+    inStock: true,
+  },
+})
+
+// Array operations
+const tagResults = await pgVector.query({
+  indexName: 'embeddings',
+  queryVector: embedding,
+  topK: 10,
+  filter: {
+    tags: { $in: ['sale', 'new'] },
+  },
+})
+
+// Logical operators
+const categoryResults = await pgVector.query({
   indexName: 'embeddings',
   queryVector: embedding,
   topK: 10,
@@ -141,6 +208,47 @@ Common use cases for metadata filtering:
 - Combine multiple conditions for precise querying
 - Filter by document attributes (e.g., language, author)
 
+### Where the filter is applied
+
+Vector stores differ in _when_ they apply a metadata filter, which affects how filtered queries scale.
+
+MongoDB can evaluate the filter inside the vector index itself. This keeps the query on a single round trip to `$vectorSearch`, so it avoids the pre-filter pass that collects matching document IDs and the 16 MB BSON limit that pass is subject to. Declaring the fields you filter on in `filterFields` when you create the index is what enables it:
+
+```ts
+// Declare the metadata fields you want to filter on
+await mongoVector.createIndex({
+  indexName: 'embeddings',
+  dimension: 1536,
+  filterFields: ['source', 'price', 'category', 'inStock', 'tags'],
+})
+
+// createIndex() returns before the index finishes building
+await mongoVector.waitForIndexReady({ indexName: 'embeddings' })
+
+// The filter is applied during the index search
+const results = await mongoVector.query({
+  indexName: 'embeddings',
+  queryVector: embedding,
+  topK: 10,
+  filter: { source: 'article1.txt' },
+})
+```
+
+Mastra passes the filter to the index only when every field it references is declared in `filterFields` and every operator is one the index accepts: `$and`, `$or`, `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$in`, and `$nin`. A filter that uses an undeclared field or any other operator takes a fallback path: Mastra matches the collection first and passes the matching document IDs into the vector search. That fallback holds only while the ID set fits within MongoDB's 16 MB BSON document limit. On large collections the query fails once that limit is exceeded, so declare your filter fields when you expect selective filters over large data sets.
+
+pgVector applies the filter as an ordinary query condition:
+
+```ts
+const results = await pgVector.query({
+  indexName: 'embeddings',
+  queryVector: embedding,
+  topK: 10,
+  filter: { source: 'article1.txt' },
+})
+```
+
+Postgres vector indexes (HNSW and IVFFlat) can't restrict that search to rows matching a condition. When a filter is present, pgVector instead compares the query vector against every matching row and returns the closest `topK`. Results are exact, but the work grows with the number of rows the filter matches. Indexing the metadata column speeds up row retrieval. The distance comparisons still happen per row.
+
 ### Vector Query Tool
 
 Sometimes you want to give your agent the ability to query a vector database directly. The Vector Query Tool allows your agent to be in charge of retrieval decisions, combining semantic search with optional filtering and reranking based on the agent's understanding of the user's needs.

package/dist/index.cjs

@@ -10,7 +10,7 @@ let _mastra_core_agent = require("@mastra/core/agent");
 let _mastra_core_evals = require("@mastra/core/evals");
 let _mastra_core_storage_domains_skills = require("@mastra/core/storage/domains/skills");
 //#region package.json
-var version = "1.18.0";
+var version = "1.18.1-alpha.1";
 //#endregion
 //#region src/vector/filter.ts
 /**
@@ -3678,8 +3678,9 @@ function transformExperimentRow(row) {
 		datasetVersion: row.datasetVersion != null ? Number(row.datasetVersion) : null,
 		organizationId: row.organizationId ?? null,
 		projectId: row.projectId ?? null,
-		targetType: row.targetType,
-		targetId: row.targetId,
+		targetType: row.targetType ?? null,
+		targetId: row.targetId ?? null,
+		scorerIds: row.scorerIds ?? null,
 		status: row.status,
 		totalItems: Number(row.totalItems ?? 0),
 		succeededCount: Number(row.succeededCount ?? 0),
@@ -3707,6 +3708,7 @@ function transformExperimentResultRow(row) {
 		startedAt: toDate(row.startedAt),
 		completedAt: toDate(row.completedAt),
 		retryCount: Number(row.retryCount ?? 0),
+		attempt: row.attempt != null ? Number(row.attempt) : 0,
 		traceId: row.traceId ?? null,
 		status: row.status ?? null,
 		tags: Array.isArray(row.tags) ? row.tags : parseJsonField(row.tags) ?? null,
@@ -3849,7 +3851,8 @@ var MongoDBExperimentsStorage = class MongoDBExperimentsStorage extends _mastra_
 				collection: _mastra_core_storage.TABLE_EXPERIMENT_RESULTS,
 				keys: {
 					experimentId: 1,
-					itemId: 1
+					itemId: 1,
+					attempt: 1
 				},
 				options: { unique: true }
 			},
@@ -3876,6 +3879,9 @@ var MongoDBExperimentsStorage = class MongoDBExperimentsStorage extends _mastra_
 	}
 	async createDefaultIndexes() {
 		if (this.#skipDefaultIndexes) return;
+		try {
+			await (await this.getCollection(_mastra_core_storage.TABLE_EXPERIMENT_RESULTS)).dropIndex("experimentId_1_itemId_1");
+		} catch {}
 		for (const indexDef of this.getDefaultIndexDefinitions()) try {
 			await (await this.getCollection(indexDef.collection)).createIndex(indexDef.keys, indexDef.options);
 		} catch (error) {
@@ -3912,8 +3918,9 @@ var MongoDBExperimentsStorage = class MongoDBExperimentsStorage extends _mastra_
 			datasetVersion: input.datasetVersion ?? null,
 			organizationId: input.organizationId ?? null,
 			projectId: input.projectId ?? null,
-			targetType: input.targetType,
-			targetId: input.targetId,
+			targetType: input.targetType ?? null,
+			targetId: input.targetId ?? null,
+			scorerIds: input.scorerIds ?? null,
 			status: "pending",
 			totalItems: input.totalItems,
 			succeededCount: 0,
@@ -4084,6 +4091,7 @@ var MongoDBExperimentsStorage = class MongoDBExperimentsStorage extends _mastra_
 			startedAt: input.startedAt,
 			completedAt: input.completedAt,
 			retryCount: input.retryCount,
+			attempt: input.attempt ?? 0,
 			traceId: input.traceId ?? null,
 			status: input.status ?? null,
 			tags: input.tags ?? null,
@@ -4106,6 +4114,7 @@ var MongoDBExperimentsStorage = class MongoDBExperimentsStorage extends _mastra_
 				startedAt: input.startedAt,
 				completedAt: input.completedAt,
 				retryCount: input.retryCount,
+				attempt: input.attempt ?? 0,
 				traceId: input.traceId ?? null,
 				status: input.status ?? null,
 				tags: input.tags ?? null,
@@ -4121,6 +4130,48 @@ var MongoDBExperimentsStorage = class MongoDBExperimentsStorage extends _mastra_
 			}, error);
 		}
 	}
+	async upsertExperimentResult(input) {
+		const attempt = input.attempt ?? 0;
+		try {
+			return transformExperimentResultRow(await (await this.getCollection(_mastra_core_storage.TABLE_EXPERIMENT_RESULTS)).findOneAndUpdate({
+				experimentId: input.experimentId,
+				itemId: input.itemId,
+				$or: [{ attempt }, ...attempt === 0 ? [{ attempt: null }, { attempt: { $exists: false } }] : []]
+			}, {
+				$set: {
+					itemDatasetVersion: input.itemDatasetVersion ?? null,
+					organizationId: input.organizationId ?? null,
+					projectId: input.projectId ?? null,
+					input: input.input,
+					output: input.output ?? null,
+					groundTruth: input.groundTruth ?? null,
+					error: input.error ?? null,
+					startedAt: input.startedAt,
+					completedAt: input.completedAt,
+					retryCount: input.retryCount,
+					attempt,
+					traceId: input.traceId ?? null,
+					status: input.status ?? null,
+					tags: input.tags ?? null,
+					toolMockReport: input.toolMockReport ?? null
+				},
+				$setOnInsert: {
+					id: (0, crypto$1.randomUUID)(),
+					createdAt: /* @__PURE__ */ new Date()
+				}
+			}, {
+				upsert: true,
+				returnDocument: "after"
+			}));
+		} catch (error) {
+			throw new _mastra_core_error.MastraError({
+				id: (0, _mastra_core_storage.createStorageErrorId)("MONGODB", "UPSERT_EXPERIMENT_RESULT", "FAILED"),
+				domain: _mastra_core_error.ErrorDomain.STORAGE,
+				category: _mastra_core_error.ErrorCategory.THIRD_PARTY,
+				details: { experimentId: input.experimentId }
+			}, error);
+		}
+	}
 	async updateExperimentResult(input) {
 		const updateFields = {};
 		if (input.status !== void 0) updateFields.status = input.status;
@@ -10060,7 +10111,8 @@ var ScoresStorageMongoDB = class ScoresStorageMongoDB extends _mastra_core_stora
 		}
 		try {
 			const now = /* @__PURE__ */ new Date();
-			const scoreId = (0, crypto$1.randomUUID)();
+			const suppliedId = validatedScore.id;
+			const scoreId = suppliedId ?? (0, crypto$1.randomUUID)();
 			const scorer = typeof validatedScore.scorer === "string" ? (0, _mastra_core_storage.safelyParseJSON)(validatedScore.scorer) : validatedScore.scorer;
 			const preprocessStepResult = typeof validatedScore.preprocessStepResult === "string" ? (0, _mastra_core_storage.safelyParseJSON)(validatedScore.preprocessStepResult) : validatedScore.preprocessStepResult;
 			const analyzeStepResult = typeof validatedScore.analyzeStepResult === "string" ? (0, _mastra_core_storage.safelyParseJSON)(validatedScore.analyzeStepResult) : validatedScore.analyzeStepResult;
@@ -10083,7 +10135,9 @@ var ScoresStorageMongoDB = class ScoresStorageMongoDB extends _mastra_core_stora
 				createdAt,
 				updatedAt
 			};
-			await (await this.getCollection(_mastra_core_storage.TABLE_SCORERS)).insertOne(dataToSave);
+			const collection = await this.getCollection(_mastra_core_storage.TABLE_SCORERS);
+			if (suppliedId) await collection.replaceOne({ id: scoreId }, dataToSave, { upsert: true });
+			else await collection.insertOne(dataToSave);
 			return { score: dataToSave };
 		} catch (error) {
 			throw new _mastra_core_error.MastraError({
@@ -11066,11 +11120,15 @@ var WorkflowsStorageMongoDB = class WorkflowsStorageMongoDB extends _mastra_core
 	}
 	async updateWorkflowState({ workflowName, runId, opts }) {
 		try {
-			const updatedDoc = await (await this.getCollection(_mastra_core_storage.TABLE_WORKFLOW_SNAPSHOT)).findOneAndUpdate({
+			const collection = await this.getCollection(_mastra_core_storage.TABLE_WORKFLOW_SNAPSHOT);
+			const { expectedStatus, ...state } = opts;
+			const filter = {
 				workflow_name: workflowName,
 				run_id: runId
-			}, [{ $set: {
-				snapshot: { $mergeObjects: ["$snapshot", opts] },
+			};
+			if (expectedStatus !== void 0) filter["snapshot.status"] = { $in: Array.isArray(expectedStatus) ? expectedStatus : [expectedStatus] };
+			const updatedDoc = await collection.findOneAndUpdate(filter, [{ $set: {
+				snapshot: { $mergeObjects: ["$snapshot", state] },
 				updatedAt: /* @__PURE__ */ new Date()
 			} }], { returnDocument: "after" });
 			if (!updatedDoc) return;