@mastra/mongodb 1.12.1 → 1.13.0-alpha.0

package/CHANGELOG.md

@@ -1,5 +1,31 @@
 # @mastra/mongodb
 
+## 1.13.0-alpha.0
+
+### Minor Changes
+
+- Added atomic caller-defined dataset IDs for idempotent dataset creation across built-in storage adapters. Supplying `id` to `mastra.datasets.create()` now creates the dataset once and resolves compatible retries to the persisted record; incompatible immutable identity fields throw `DATASET_ID_CONFLICT`. ([#19370](https://github.com/mastra-ai/mastra/pull/19370))
+
+- Added caller-defined dataset item identities for safe retries across all dataset storage adapters. ([#19384](https://github.com/mastra-ai/mastra/pull/19384))
+
+  Dataset items can now include an `externalId` when calling `addItem` or `addItems`:
+
+  ```ts
+  await dataset.addItem({
+    externalId: 'source-item-123',
+    input: { prompt: 'Hello' },
+  });
+  ```
+
+  Retrying with the same identity and payload returns the existing item. Reusing an identity with different content returns a typed conflict, including during concurrent writes. Updates and deletes preserve the identity, Spanner retries transactions without changing the outcome, and MySQL batch writes now preserve every supported dataset item field.
+
+### Patch Changes
+
+- Raised the `@mastra/core` peer dependency floor to `>=1.51.0-0` so the dataset item identity helpers used by the storage adapters are available at runtime. ([#19384](https://github.com/mastra-ai/mastra/pull/19384))
+
+- Updated dependencies [[`a99eae8`](https://github.com/mastra-ai/mastra/commit/a99eae8908e500c1b2d12f9d277be616b98617a5), [`fd13f8e`](https://github.com/mastra-ai/mastra/commit/fd13f8e21990f9904c3eedba3a626bb4a929cdb8), [`f703f87`](https://github.com/mastra-ai/mastra/commit/f703f878de072d51fda557f9c50867d8252bef05), [`0ad646f`](https://github.com/mastra-ai/mastra/commit/0ad646f71a530f2454664299e5e01bfd13fa12e5)]:
+  - @mastra/core@1.51.0-alpha.13
+
 ## 1.12.1
 
 ### Patch Changes

package/dist/docs/SKILL.md

@@ -3,7 +3,7 @@ name: mastra-mongodb
 description: Documentation for @mastra/mongodb. Use when working with @mastra/mongodb APIs, configuration, or implementation.
 metadata:
   package: "@mastra/mongodb"
-  version: "1.12.1"
+  version: "1.13.0-alpha.0"
 ---
 
 ## When to use
@@ -16,12 +16,15 @@ Read the individual reference documents for detailed explanations and code examp
 
 ### Docs
 
+- [Semantic recall](references/docs-memory-semantic-recall.md) - Learn how to use semantic recall in Mastra to retrieve relevant messages from past conversations using vector search and embeddings.
 - [Working memory](references/docs-memory-working-memory.md) - Learn how to configure working memory in Mastra to store persistent user data, preferences.
 - [Retrieval, semantic search, reranking](references/docs-rag-retrieval.md) - Guide on retrieval processes in Mastra's RAG systems, including semantic search, filtering, and re-ranking.
 - [Storing embeddings in a vector database](references/docs-rag-vector-databases.md) - Guide on vector storage options in Mastra, including embedded and dedicated vector databases for similarity search.
+- [Storage overview](references/docs-storage-overview.md) - Configure storage for Mastra to persist runtime state across agents, workflows, observability, evals, schedules, and memory.
 
 ### Reference
 
+- [Reference: Composite storage](references/reference-storage-composite.md) - Documentation for combining multiple storage backends in Mastra.
 - [Reference: MongoDB storage](references/reference-storage-mongodb.md) - Documentation for the MongoDB storage implementation in Mastra.
 - [Reference: MongoDB vector store](references/reference-vectors-mongodb.md) - Documentation for the MongoDBVector class in Mastra, which provides vector search using MongoDB Atlas and Atlas Vector Search.
 

package/dist/docs/assets/SOURCE_MAP.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.12.1",
+  "version": "1.13.0-alpha.0",
   "package": "@mastra/mongodb",
   "exports": {},
   "modules": {}

package/dist/docs/references/docs-memory-semantic-recall.md

@@ -0,0 +1,401 @@
+> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
+
+# Semantic recall
+
+If you ask your friend what they did last weekend, they will search in their memory for events associated with "last weekend" and then tell you what they did. That's sort of like how semantic recall works in Mastra.
+
+> **📹 Watch:** Watch [Mastra semantic recall](https://www.youtube.com/watch?v=UVZtK8cK8xQ\&pp=ygUVbWFzdHJhIHdvcmtpbmcgbWVtb3J5) to see how agents retrieve relevant messages from past conversations.
+
+## How semantic recall works
+
+Semantic recall is RAG-based search that helps agents maintain context across longer interactions when messages are no longer within [recent message history](https://mastra.ai/docs/memory/message-history).
+
+It uses vector embeddings of messages for similarity search, integrates with various vector stores, and has configurable context windows around retrieved messages.
+
+![Diagram showing Mastra Memory semantic recall](/assets/images/semantic-recall-fd7b9336a6d0d18019216cb6d3dbe710.png)
+
+When it's enabled, new messages are used to query a vector DB for semantically similar messages.
+
+After getting a response from the LLM, all new messages (user, assistant, and tool calls/results) are inserted into the vector DB to be recalled in later interactions.
+
+## Quickstart
+
+Semantic recall is disabled by default. To enable it, set `semanticRecall: true` in `options` and provide a `vector` store and `embedder`:
+
+**LibSQL**:
+
+```typescript
+import { Agent } from '@mastra/core/agent'
+import { Memory } from '@mastra/memory'
+import { LibSQLStore, LibSQLVector } from '@mastra/libsql'
+import { ModelRouterEmbeddingModel } from '@mastra/core/llm'
+
+const agent = new Agent({
+  id: 'support-agent',
+  name: 'SupportAgent',
+  instructions: 'You are a helpful support agent.',
+  model: 'openai/gpt-5.5',
+  memory: new Memory({
+    storage: new LibSQLStore({
+      id: 'agent-storage',
+      url: 'file:./local.db',
+    }),
+    vector: new LibSQLVector({
+      id: 'agent-vector',
+      url: 'file:./local.db',
+    }),
+    embedder: new ModelRouterEmbeddingModel('openai/text-embedding-3-small'),
+    options: {
+      semanticRecall: true,
+    },
+  }),
+})
+```
+
+**MongoDB**:
+
+```typescript
+import { Agent } from '@mastra/core/agent'
+import { Memory } from '@mastra/memory'
+import { MongoDBStore, MongoDBVector } from '@mastra/mongodb'
+import { ModelRouterEmbeddingModel } from '@mastra/core/llm'
+
+const agent = new Agent({
+  id: 'support-agent',
+  name: 'SupportAgent',
+  instructions: 'You are a helpful support agent.',
+  model: 'openai/gpt-5.5',
+  memory: new Memory({
+    storage: new MongoDBStore({
+      id: 'agent-storage',
+      uri: process.env.MONGODB_URI,
+      dbName: process.env.MONGODB_DB_NAME,
+    }),
+    vector: new MongoDBVector({
+      id: 'agent-vector',
+      uri: process.env.MONGODB_URI,
+      dbName: process.env.MONGODB_DB_NAME,
+    }),
+    embedder: new ModelRouterEmbeddingModel('openai/text-embedding-3-small'),
+    options: {
+      semanticRecall: true,
+    },
+  }),
+})
+```
+
+## Using the `recall()` method
+
+While `listMessages` retrieves messages by thread ID with basic pagination, [`recall()`](https://mastra.ai/reference/memory/recall) adds support for **semantic search**. When you need to find messages by meaning rather than recency, use `recall()` with a `vectorSearchString`:
+
+```typescript
+const memory = await agent.getMemory()
+
+// Basic recall - similar to listMessages
+const { messages } = await memory!.recall({
+  threadId: 'thread-123',
+  perPage: 50,
+})
+
+// Semantic recall - find messages by meaning
+const { messages: relevantMessages } = await memory!.recall({
+  threadId: 'thread-123',
+  vectorSearchString: 'What did we discuss about the project deadline?',
+  threadConfig: {
+    semanticRecall: true,
+  },
+})
+```
+
+## Storage configuration
+
+Semantic recall relies on a [storage and vector db](https://mastra.ai/reference/memory/memory-class) to store messages and their embeddings.
+
+```ts
+import { Memory } from '@mastra/memory'
+import { Agent } from '@mastra/core/agent'
+import { LibSQLStore, LibSQLVector } from '@mastra/libsql'
+
+const agent = new Agent({
+  memory: new Memory({
+    // this is the default storage db if omitted
+    storage: new LibSQLStore({
+      id: 'agent-storage',
+      url: 'file:./local.db',
+    }),
+    // this is the default vector db if omitted
+    vector: new LibSQLVector({
+      id: 'agent-vector',
+      url: 'file:./local.db',
+    }),
+    options: {
+      semanticRecall: true,
+    },
+  }),
+})
+```
+
+Each vector store page below includes installation instructions, configuration parameters, and usage examples:
+
+- [Astra](https://mastra.ai/reference/vectors/astra)
+- [Chroma](https://mastra.ai/reference/vectors/chroma)
+- [Cloudflare Vectorize](https://mastra.ai/reference/vectors/vectorize)
+- [Convex](https://mastra.ai/reference/vectors/convex)
+- [Couchbase](https://mastra.ai/reference/vectors/couchbase)
+- [DuckDB](https://mastra.ai/reference/vectors/duckdb)
+- [Elasticsearch](https://mastra.ai/reference/vectors/elasticsearch)
+- [LanceDB](https://mastra.ai/reference/vectors/lance)
+- [libSQL](https://mastra.ai/reference/vectors/libsql)
+- [MongoDB](https://mastra.ai/reference/vectors/mongodb)
+- [OpenSearch](https://mastra.ai/reference/vectors/opensearch)
+- [Pinecone](https://mastra.ai/reference/vectors/pinecone)
+- [PostgreSQL](https://mastra.ai/reference/vectors/pg)
+- [Qdrant](https://mastra.ai/reference/vectors/qdrant)
+- [S3 Vectors](https://mastra.ai/reference/vectors/s3vectors)
+- [Turbopuffer](https://mastra.ai/reference/vectors/turbopuffer)
+- [Upstash](https://mastra.ai/reference/vectors/upstash)
+
+## Recall configuration
+
+The following options control semantic recall behavior:
+
+1. **topK**: The number of similar messages to retrieve
+2. **messageRange**: The surrounding messages to include with each match
+3. **scope**: Whether to search the current thread or all threads for a resource
+4. **filter**: Metadata criteria that restrict search results
+
+```typescript
+const agent = new Agent({
+  id: 'agent',
+  memory: new Memory({
+    options: {
+      semanticRecall: {
+        topK: 3, // Retrieve 3 similar messages
+        messageRange: 2, // Include 2 messages before and after each match
+        scope: 'resource', // Search all threads for this resource
+        filter: { projectId: { $eq: 'project-a' } },
+      },
+    },
+  }),
+})
+```
+
+> **Note:** `scope: 'resource'` is supported by the LibSQL, PostgreSQL, MongoDB, and Upstash storage adapters.
+
+### Metadata filtering
+
+The `filter` option restricts semantic recall results to messages with matching thread metadata.
+
+```typescript
+const agent = new Agent({
+  id: 'agent',
+  memory: new Memory({
+    options: {
+      semanticRecall: {
+        scope: 'resource',
+        filter: {
+          projectId: { $eq: 'project-a' },
+          category: { $in: ['work', 'personal'] },
+        },
+      },
+    },
+  }),
+})
+```
+
+Filters match metadata stored on message embeddings when messages are saved. If thread metadata changes later, existing embeddings keep their previous metadata until those messages are saved or indexed again.
+
+Supported filter operators:
+
+- `$and`: Logical AND
+- `$eq`: Equal to
+- `$gt`: Greater than
+- `$gte`: Greater than or equal
+- `$in`: In array
+- `$lt`: Less than
+- `$lte`: Less than or equal
+- `$ne`: Not equal to
+- `$nin`: Not in array
+- `$or`: Logical OR
+
+The following example demonstrates metadata filters for common use cases:
+
+```typescript
+// Filter by project
+const options = {
+  semanticRecall: { filter: { projectId: { $eq: 'my-project' } } },
+}
+
+// Filter by multiple categories
+const options = {
+  semanticRecall: { filter: { category: { $in: ['work', 'research'] } } },
+}
+
+// Filter by project and priority
+const options = {
+  semanticRecall: {
+    filter: {
+      $and: [{ projectId: { $eq: 'project-a' } }, { priority: { $gte: 3 } }],
+    },
+  },
+}
+```
+
+## Embedder configuration
+
+Semantic recall relies on an [embedding model](https://mastra.ai/reference/memory/memory-class) to convert messages into embeddings. Mastra supports embedding models through the model router using `provider/model` strings, or you can use any [embedding model](https://sdk.vercel.ai/docs/ai-sdk-core/embeddings) compatible with the AI SDK.
+
+### Using the Model Router (Recommended)
+
+The simplest way is to use a `provider/model` string with autocomplete support:
+
+```ts
+import { Memory } from '@mastra/memory'
+import { Agent } from '@mastra/core/agent'
+import { ModelRouterEmbeddingModel } from '@mastra/core/llm'
+
+const agent = new Agent({
+  id: 'agent',
+  memory: new Memory({
+    embedder: new ModelRouterEmbeddingModel('openai/text-embedding-3-small'),
+    options: {
+      semanticRecall: true,
+    },
+  }),
+})
+```
+
+Supported embedding models:
+
+- **OpenAI**: `text-embedding-3-small`, `text-embedding-3-large`, `text-embedding-ada-002`
+- **Google**: `gemini-embedding-001`
+- **OpenRouter**: Access embedding models from various providers
+
+```ts
+import { Agent } from '@mastra/core/agent'
+import { Memory } from '@mastra/memory'
+import { ModelRouterEmbeddingModel } from '@mastra/core/llm'
+
+const agent = new Agent({
+  id: 'agent',
+  memory: new Memory({
+    embedder: new ModelRouterEmbeddingModel({
+      providerId: 'openrouter',
+      modelId: 'openai/text-embedding-3-small',
+    }),
+  }),
+})
+```
+
+The model router automatically handles API key detection from environment variables (`OPENAI_API_KEY`, `GOOGLE_API_KEY`, `OPENROUTER_API_KEY`). Google models also fall back to `GOOGLE_GENERATIVE_AI_API_KEY`.
+
+### Using AI SDK Packages
+
+You can also use AI SDK embedding models directly:
+
+```ts
+import { Memory } from '@mastra/memory'
+import { Agent } from '@mastra/core/agent'
+import { ModelRouterEmbeddingModel } from '@mastra/core/llm'
+
+const agent = new Agent({
+  id: 'agent',
+  memory: new Memory({
+    embedder: new ModelRouterEmbeddingModel('openai/text-embedding-3-small'),
+  }),
+})
+```
+
+### Using FastEmbed (local)
+
+To use FastEmbed (a local embedding model), install `@mastra/fastembed`:
+
+**npm**:
+
+```bash
+npm install @mastra/fastembed@latest
+```
+
+**pnpm**:
+
+```bash
+pnpm add @mastra/fastembed@latest
+```
+
+**Yarn**:
+
+```bash
+yarn add @mastra/fastembed@latest
+```
+
+**Bun**:
+
+```bash
+bun add @mastra/fastembed@latest
+```
+
+Then configure it in your memory:
+
+```ts
+import { Memory } from '@mastra/memory'
+import { Agent } from '@mastra/core/agent'
+import { fastembed } from '@mastra/fastembed'
+
+const agent = new Agent({
+  id: 'agent',
+  memory: new Memory({
+    embedder: fastembed,
+  }),
+})
+```
+
+## PostgreSQL index optimization
+
+When using PostgreSQL as your vector store, you can optimize semantic recall performance by configuring the vector index. This is particularly important for large-scale deployments with thousands of messages.
+
+PostgreSQL supports both IVFFlat and HNSW indexes. By default, Mastra creates an IVFFlat index, but HNSW indexes typically provide better performance, especially with OpenAI embeddings which use inner product distance.
+
+```typescript
+import { Memory } from '@mastra/memory'
+import { PgStore, PgVector } from '@mastra/pg'
+
+const agent = new Agent({
+  memory: new Memory({
+    storage: new PgStore({
+      id: 'agent-storage',
+      connectionString: process.env.DATABASE_URL,
+    }),
+    vector: new PgVector({
+      id: 'agent-vector',
+      connectionString: process.env.DATABASE_URL,
+    }),
+    options: {
+      semanticRecall: {
+        topK: 5,
+        messageRange: 2,
+        indexConfig: {
+          type: 'hnsw', // Use HNSW for better performance
+          metric: 'dotproduct', // Best for OpenAI embeddings
+          m: 16, // Number of bi-directional links (default: 16)
+          efConstruction: 64, // Size of candidate list during construction (default: 64)
+        },
+      },
+    },
+  }),
+})
+```
+
+For detailed information about index configuration options and performance tuning, see the [PgVector configuration guide](https://mastra.ai/reference/vectors/pg).
+
+## Disable semantic recall
+
+Semantic recall is disabled by default (`semanticRecall: false`). Each call adds latency because new messages are converted into embeddings and used to query a vector database before the LLM receives them.
+
+Keep semantic recall disabled when:
+
+- Message history provides sufficient context for the current conversation.
+- You're building performance-sensitive applications, like realtime two-way audio, where embedding and vector query latency is noticeable.
+
+## Viewing recalled messages
+
+When tracing is enabled, any messages retrieved via semantic recall will appear in the agent's trace output, alongside recent message history (if configured).

package/dist/docs/references/docs-memory-working-memory.md

@@ -10,6 +10,8 @@ This is useful for maintaining ongoing state that's always relevant and should a
 
 If you use [Observational Memory](https://mastra.ai/docs/memory/observational-memory), `observationalMemory.observation.manageWorkingMemory` lets OM update working memory for the agent.
 
+> **📹 Watch:** Watch [Mastra working memory](https://www.youtube.com/watch?v=UMy_JHLf1n8\&pp=ygUVbWFzdHJhIHdvcmtpbmcgbWVtb3J5) to see how agents keep persistent user context available across interactions.
+
 Working memory can persist at two different scopes:
 
 - **Resource-scoped** (default): Memory persists across all conversation threads for the same user
@@ -43,9 +45,7 @@ const agent = new Agent({
 
 ## How it works
 
-Working memory is a block of Markdown text that the agent is able to update over time to store continuously relevant information:
-
-[YouTube video player](https://www.youtube-nocookie.com/embed/UMy_JHLf1n8)
+Working memory is a block of Markdown text that the agent is able to update over time to store continuously relevant information.
 
 ## Memory persistence scopes
 

package/dist/docs/references/docs-rag-vector-databases.md

@@ -14,7 +14,7 @@ import { MongoDBVector } from '@mastra/mongodb'
 const store = new MongoDBVector({
   id: 'mongodb-vector',
   uri: process.env.MONGODB_URI,
-  dbName: process.env.MONGODB_DATABASE,
+  dbName: process.env.MONGODB_DB_NAME,
 })
 await store.createIndex({
   indexName: 'myCollection',
@@ -27,10 +27,14 @@ await store.upsert({
 })
 ```
 
-### Using MongoDB Atlas Vector search
+### Using MongoDB Atlas Vector Search
 
 For detailed setup instructions and best practices, see the [official MongoDB Atlas Vector Search documentation](https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-overview/?utm_campaign=devrel\&utm_source=third-party-content\&utm_medium=cta\&utm_content=mastra-docs).
 
+### Using VoyageAI with MongoDB
+
+MongoDB works seamlessly with VoyageAI's embedding models, which are optimized for retrieval tasks. For complete examples and specialized models, see the [VoyageAI embeddings documentation](https://mastra.ai/models/embeddings) and [MongoDB vector reference](https://mastra.ai/reference/vectors/mongodb).
+
 **PgVector**:
 
 ```ts
@@ -359,6 +363,7 @@ The dimension size must match the output dimension of your chosen embedding mode
 
 - `OpenAI text-embedding-3-small`: 1536 dimensions (or custom, e.g., 256)
 - `Cohere embed-multilingual-v3`: 1024 dimensions
+- `VoyageAI voyage-3.5`: 1024 dimensions (or custom: 256, 512, 1024, 2048)
 - `Google gemini-embedding-001`: 768 dimensions (or custom)
 
 > **Warning:** Index dimensions can't be changed after creation. To use a different model, delete and recreate the index with the new dimension size.