@mastra/mongodb 1.12.1 → 1.13.0

package/dist/docs/references/docs-storage-overview.md

@@ -0,0 +1,214 @@
+> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
+
+# Storage overview
+
+Storage is the persistence layer for the Mastra runtime. It keeps memory, workflow state, observability data, eval results, schedules, and long-running agent state available after a process restarts.
+
+Storage powers:
+
+- [Memory](https://mastra.ai/docs/memory/overview): Message history, threads, resources, and working memory.
+- [Workflows](https://mastra.ai/docs/workflows/overview): Durable snapshots for suspended and resumed workflow runs.
+- [Observability](https://mastra.ai/docs/observability/overview): Traces, spans, metrics, logs, and feedback.
+- [Evals](https://mastra.ai/docs/evals/overview): Scores, datasets, experiments, and evaluation results.
+- [Long-running agents](https://mastra.ai/docs/long-running-agents/durable-agents): Background tasks, schedules, goals, and thread state.
+
+## When to configure storage
+
+Configure a persistent storage adapter when state must survive restarts, be shared across processes, or be visible in Studio across sessions. The default in-memory store is useful for tests and short local experiments, but it loses data when the process exits.
+
+Use storage when your application needs any of these behaviors:
+
+- Agents remember past messages or user facts.
+- Workflows suspend and resume after a restart.
+- Traces, metrics, logs, scores, or feedback stay available for analysis.
+- Schedules and background tasks continue across deployments.
+- Multiple runtime processes read and write the same state.
+
+## How storage works
+
+Mastra storage is organized into **domains**. A domain owns one type of runtime data, and a storage adapter implements one or more domains.
+
+| Domain            | What it stores                                                              |
+| ----------------- | --------------------------------------------------------------------------- |
+| `memory`          | Threads, messages, resources, working memory, and other agent memory state. |
+| `workflows`       | Workflow snapshots used to suspend and resume runs.                         |
+| `observability`   | Traces, spans, metrics, logs, and feedback.                                 |
+| `scores`          | Eval score records.                                                         |
+| `datasets`        | Dataset records and dataset items used by evals and experiments.            |
+| `experiments`     | Experiment runs and per-item experiment results.                            |
+| `backgroundTasks` | Background task records and execution state.                                |
+| `schedules`       | Schedule definitions and trigger history.                                   |
+| `threadState`     | Durable task, goal, and thread state.                                       |
+
+Adapter support varies by domain. For the full domain list and built-in schemas, see the [storage overview reference](https://mastra.ai/reference/storage/overview).
+
+## Choose a backend by data shape
+
+Different domains write and query different kinds of data. Pick a backend based on the domain's access pattern:
+
+- `memory`: Reads and writes rows during every remembered agent call. Use a transactional database such as libSQL, PostgreSQL, or MongoDB.
+- `observability`: Writes high-volume telemetry and often queries aggregations. Use a dedicated observability store or an online analytical processing (OLAP) backend such as ClickHouse or DuckDB.
+- `workflows`: Stores durable snapshots that must be available when a run resumes. Use a reliable persistent database.
+- `scores`, `datasets`, and `experiments`: Store lower-frequency evaluation data that's often read later for analysis.
+- `schedules`: Stores schedule definitions and fire history. Use an adapter that implements the schedules domain.
+
+When domains have different operational needs, use [composite storage](#composite-storage) to route each domain to the right backend.
+
+## Get started locally
+
+For local development, use libSQL with a file-backed database. It doesn't require a separate database server and persists state between restarts.
+
+```typescript
+import { Mastra } from '@mastra/core'
+import { LibSQLStore } from '@mastra/libsql'
+
+export const mastra = new Mastra({
+  storage: new LibSQLStore({
+    id: 'mastra-storage',
+    url: 'file:./mastra.db',
+  }),
+})
+```
+
+> **Sharing the database with Studio:** When running `mastra dev` alongside your application, use an absolute path so both processes access the same database:
+>
+> ```typescript
+> url: 'file:/absolute/path/to/your/project/mastra.db'
+> ```
+>
+> Relative paths like `file:./mastra.db` resolve based on each process's working directory, which may differ.
+
+Mastra initializes the required storage structures on first use.
+
+## Configure for production
+
+For production, use a persistent managed database. PostgreSQL is a good default for most teams because it works well for transactional runtime state and is widely available as a managed service.
+
+Production guidance:
+
+- Use a managed database with backups, monitoring, and connection pooling.
+- Keep local file databases such as `file:./mastra.db` out of multi-process production deployments.
+- Route high-volume domains, especially `observability`, to a dedicated backend with [composite storage](#composite-storage).
+- Configure [retention](https://mastra.ai/reference/storage/retention) policies on the storage adapter or composite store, then call `storage.prune()` from a scheduler or maintenance job.
+- Choose providers based on the domains your application uses. For example, schedules require an adapter that implements the `schedules` domain.
+
+## Configuration scope
+
+Storage can be configured at the Mastra instance level or at the agent level.
+
+### Instance-level storage
+
+Instance-level storage is shared by agents, workflows, observability, evals, schedules, and other runtime features registered on the same Mastra instance.
+
+**PostgreSQL**:
+
+```typescript
+import { Mastra } from '@mastra/core'
+import { PostgresStore } from '@mastra/pg'
+
+export const mastra = new Mastra({
+  storage: new PostgresStore({
+    id: 'mastra-storage',
+    connectionString: process.env.DATABASE_URL,
+  }),
+})
+```
+
+**MongoDB**:
+
+```typescript
+import { Mastra } from '@mastra/core'
+import { MongoDBStore } from '@mastra/mongodb'
+
+export const mastra = new Mastra({
+  storage: new MongoDBStore({
+    id: 'mastra-storage',
+    uri: process.env.MONGODB_URI,
+    dbName: process.env.MONGODB_DB_NAME,
+  }),
+})
+```
+
+Use instance-level storage when most runtime domains can share the same database.
+
+### Agent-level storage
+
+Agent-level storage is configured on a `Memory` instance. It overrides instance-level storage for that agent's memory data only.
+
+```typescript
+import { Agent } from '@mastra/core/agent'
+import { Memory } from '@mastra/memory'
+import { PostgresStore } from '@mastra/pg'
+
+export const supportAgent = new Agent({
+  id: 'support-agent',
+  name: 'Support agent',
+  instructions: 'Answer customer support questions.',
+  model: 'openai/gpt-5.5',
+  memory: new Memory({
+    storage: new PostgresStore({
+      id: 'support-agent-storage',
+      connectionString: process.env.SUPPORT_AGENT_DATABASE_URL,
+    }),
+  }),
+})
+```
+
+Use agent-level storage when an agent needs an isolated memory boundary or a different memory backend.
+
+## Composite storage
+
+[`MastraCompositeStore`](https://mastra.ai/reference/storage/composite) routes domains to different backends. Use it when one database isn't the right fit for every domain.
+
+The following example uses libSQL as the default store and routes workflow state to PostgreSQL:
+
+```typescript
+import { Mastra } from '@mastra/core'
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { LibSQLStore } from '@mastra/libsql'
+import { WorkflowsPG } from '@mastra/pg'
+
+export const mastra = new Mastra({
+  storage: new MastraCompositeStore({
+    id: 'composite-storage',
+    default: new LibSQLStore({
+      id: 'default-storage',
+      url: 'file:./mastra.db',
+    }),
+    domains: {
+      workflows: new WorkflowsPG({
+        connectionString: process.env.DATABASE_URL,
+      }),
+    },
+  }),
+})
+```
+
+You can also route `observability` to a dedicated analytics backend. See [observability storage](https://mastra.ai/docs/observability/storage) for an observability-specific example.
+
+## Supported providers
+
+Each provider page includes installation instructions, configuration parameters, and usage examples:
+
+- [libSQL](https://mastra.ai/reference/storage/libsql)
+- [PostgreSQL](https://mastra.ai/reference/storage/postgresql)
+- [MongoDB](https://mastra.ai/reference/storage/mongodb)
+- [Upstash](https://mastra.ai/reference/storage/upstash)
+- [Redis](https://mastra.ai/reference/storage/redis)
+- [Cloudflare D1](https://mastra.ai/reference/storage/cloudflare-d1)
+- [Cloudflare KV & Durable Objects](https://mastra.ai/reference/storage/cloudflare)
+- [Convex](https://mastra.ai/reference/storage/convex)
+- [DynamoDB](https://mastra.ai/reference/storage/dynamodb)
+- [LanceDB](https://mastra.ai/reference/storage/lance)
+- [Microsoft SQL Server](https://mastra.ai/reference/storage/mssql)
+- [Google Cloud Spanner](https://mastra.ai/reference/storage/spanner)
+
+> **Tip:** libSQL is the fastest path for local development because it doesn't require running a separate database server.
+
+## Next steps
+
+- [Composite storage](https://mastra.ai/reference/storage/composite)
+- [Storage retention](https://mastra.ai/reference/storage/retention)
+- [Storage schemas](https://mastra.ai/reference/storage/overview)
+- [Memory](https://mastra.ai/docs/memory/overview)
+- [Observability storage](https://mastra.ai/docs/observability/storage)

package/dist/docs/references/reference-storage-composite.md

@@ -0,0 +1,337 @@
+> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
+
+# Composite storage
+
+`MastraCompositeStore` can compose storage domains from different providers. Use it when you need different databases for different purposes. For example, use LibSQL for memory and PostgreSQL for workflows.
+
+## Installation
+
+`MastraCompositeStore` is included in `@mastra/core`:
+
+**npm**:
+
+```bash
+npm install @mastra/core@latest
+```
+
+**pnpm**:
+
+```bash
+pnpm add @mastra/core@latest
+```
+
+**Yarn**:
+
+```bash
+yarn add @mastra/core@latest
+```
+
+**Bun**:
+
+```bash
+bun add @mastra/core@latest
+```
+
+You'll also need to install the storage providers you want to compose:
+
+**npm**:
+
+```bash
+npm install @mastra/pg@latest @mastra/libsql@latest @mastra/mongodb@latest
+```
+
+**pnpm**:
+
+```bash
+pnpm add @mastra/pg@latest @mastra/libsql@latest @mastra/mongodb@latest
+```
+
+**Yarn**:
+
+```bash
+yarn add @mastra/pg@latest @mastra/libsql@latest @mastra/mongodb@latest
+```
+
+**Bun**:
+
+```bash
+bun add @mastra/pg@latest @mastra/libsql@latest @mastra/mongodb@latest
+```
+
+## Storage domains
+
+Mastra organizes storage into domains, each handling a specific type of data. Each domain can be backed by a different storage adapter, and domain classes are exported from each storage package.
+
+| Domain          | Description                                                                                                                                                                           |
+| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `memory`        | Conversation persistence for agents. Stores threads (conversation sessions), messages, resources (user identities), and working memory (persistent context across conversations).     |
+| `workflows`     | Workflow execution state. When workflows suspend for human input, external events, or scheduled resumption, their state is persisted here to enable resumption after server restarts. |
+| `scores`        | Evaluation results from Mastra's evals system. Scores and metrics are persisted here for analysis and comparison over time.                                                           |
+| `observability` | Telemetry data including traces and spans. Agent interactions, tool calls, and LLM requests generate spans collected into traces for debugging and performance analysis.              |
+| `agents`        | Agent configurations for stored agents. Enables agents to be defined and updated at runtime without code deployments.                                                                 |
+| `datasets`      | Evaluation datasets used for experiment runs. Stores dataset definitions, schemas, and versioned items.                                                                               |
+| `experiments`   | Experiment runs and per-item experiment results linked to datasets and targets.                                                                                                       |
+
+> **Note:** `MastraCompositeStore` accepts all of the domain keys above, but storage adapter support varies by package. You can mix adapters per domain, but only for domains implemented and exported by those adapters. For example, `memory: new MemoryLibSQL(...)` and `workflows: new WorkflowsPG(...)` is valid because both packages export those domain classes.
+
+## Usage
+
+### Basic composition
+
+Import domain classes directly from each store package and compose them:
+
+```typescript
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { WorkflowsPG, ScoresPG } from '@mastra/pg'
+import { MemoryLibSQL } from '@mastra/libsql'
+import { Mastra } from '@mastra/core'
+
+export const mastra = new Mastra({
+  storage: new MastraCompositeStore({
+    id: 'composite',
+    domains: {
+      memory: new MemoryLibSQL({ url: 'file:./local.db' }),
+      workflows: new WorkflowsPG({ connectionString: process.env.DATABASE_URL }),
+      scores: new ScoresPG({ connectionString: process.env.DATABASE_URL }),
+    },
+  }),
+})
+```
+
+### With a default storage
+
+Use `default` to specify a fallback storage, then override specific domains:
+
+```typescript
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { PostgresStore } from '@mastra/pg'
+import { MemoryLibSQL } from '@mastra/libsql'
+import { Mastra } from '@mastra/core'
+
+const pgStore = new PostgresStore({
+  id: 'pg',
+  connectionString: process.env.DATABASE_URL,
+})
+
+export const mastra = new Mastra({
+  storage: new MastraCompositeStore({
+    id: 'composite',
+    default: pgStore,
+    domains: {
+      memory: new MemoryLibSQL({ url: 'file:./local.db' }),
+    },
+  }),
+})
+```
+
+### Mixed backends
+
+Use domain classes from each storage package to route different domains to different backends. The following example stores memory and workflow state in MongoDB, then routes observability to ClickHouse:
+
+```typescript
+import { Mastra } from '@mastra/core'
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { ObservabilityStorageClickhouse } from '@mastra/clickhouse'
+import { MemoryStorageMongoDB, WorkflowsStorageMongoDB } from '@mastra/mongodb'
+
+export const mastra = new Mastra({
+  storage: new MastraCompositeStore({
+    id: 'composite',
+    domains: {
+      memory: new MemoryStorageMongoDB({
+        uri: process.env.MONGODB_URI,
+        dbName: 'mastra_memory',
+      }),
+      workflows: new WorkflowsStorageMongoDB({
+        uri: process.env.MONGODB_URI,
+        dbName: 'mastra_workflows',
+      }),
+      observability: new ObservabilityStorageClickhouse({
+        url: process.env.CLICKHOUSE_URL,
+        username: process.env.CLICKHOUSE_USERNAME,
+        password: process.env.CLICKHOUSE_PASSWORD,
+      }),
+    },
+  }),
+})
+```
+
+### Disabling a domain
+
+Set a domain to `false` to disable it. A disabled domain doesn't fall back to `default`, so data for that domain isn't persisted:
+
+```typescript
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { PostgresStore } from '@mastra/pg'
+import { Mastra } from '@mastra/core'
+
+const pgStore = new PostgresStore({
+  id: 'pg',
+  connectionString: process.env.DATABASE_URL,
+})
+
+export const mastra = new Mastra({
+  storage: new MastraCompositeStore({
+    id: 'composite',
+    default: pgStore,
+    domains: {
+      // don't persist traces and spans
+      observability: false,
+    },
+  }),
+})
+```
+
+## Options
+
+**id** (`string`): Unique identifier for this storage instance.
+
+**default** (`MastraCompositeStore`): Default storage adapter. Domains not explicitly specified in domains will use this storage's domains as fallbacks.
+
+**disableInit** (`boolean`): When true, automatic initialization is disabled. You must call init() explicitly.
+
+**domains** (`object`): Individual domain overrides. Each domain can come from a different storage adapter. These take precedence over both editor and default storage. Set a domain to false to disable it entirely; a disabled domain does not fall back to editor or default.
+
+**domains.memory** (`MemoryStorage`): Storage for threads, messages, and resources.
+
+**domains.workflows** (`WorkflowsStorage`): Storage for workflow snapshots.
+
+**domains.scores** (`ScoresStorage`): Storage for evaluation scores.
+
+**domains.observability** (`ObservabilityStorage`): Storage for traces and spans.
+
+**domains.agents** (`AgentsStorage`): Storage for stored agent configurations.
+
+**domains.datasets** (`DatasetsStorage`): Storage for dataset metadata, dataset items, and dataset versions.
+
+**domains.experiments** (`ExperimentsStorage`): Storage for experiment runs and per-item experiment results.
+
+## Initialization
+
+`MastraCompositeStore` initializes each configured domain independently. When passed to the Mastra class, `init()` is called automatically:
+
+```typescript
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { MemoryPG, WorkflowsPG, ScoresPG } from '@mastra/pg'
+import { Mastra } from '@mastra/core'
+
+const storage = new MastraCompositeStore({
+  id: 'composite',
+  domains: {
+    memory: new MemoryPG({ connectionString: process.env.DATABASE_URL }),
+    workflows: new WorkflowsPG({ connectionString: process.env.DATABASE_URL }),
+    scores: new ScoresPG({ connectionString: process.env.DATABASE_URL }),
+  },
+})
+
+export const mastra = new Mastra({
+  storage, // init() called automatically
+})
+```
+
+If using storage directly, call `init()` explicitly:
+
+```typescript
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { MemoryPG } from '@mastra/pg'
+
+const storage = new MastraCompositeStore({
+  id: 'composite',
+  domains: {
+    memory: new MemoryPG({ connectionString: process.env.DATABASE_URL }),
+  },
+})
+
+await storage.init()
+
+// Access domain-specific stores via getStore()
+const memoryStore = await storage.getStore('memory')
+const thread = await memoryStore?.getThreadById({ threadId: '...' })
+```
+
+## Use cases
+
+### Separate databases for different workloads
+
+Use a local database for development while keeping production data in a managed service:
+
+```typescript
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { MemoryPG, WorkflowsPG, ScoresPG } from '@mastra/pg'
+import { MemoryLibSQL } from '@mastra/libsql'
+
+const storage = new MastraCompositeStore({
+  id: 'composite',
+  domains: {
+    // Use local SQLite for development, PostgreSQL for production
+    memory:
+      process.env.NODE_ENV === 'development'
+        ? new MemoryLibSQL({ url: 'file:./dev.db' })
+        : new MemoryPG({ connectionString: process.env.DATABASE_URL }),
+    workflows: new WorkflowsPG({ connectionString: process.env.DATABASE_URL }),
+    scores: new ScoresPG({ connectionString: process.env.DATABASE_URL }),
+  },
+})
+```
+
+### Specialized storage for observability
+
+Observability data can quickly overwhelm general-purpose databases in production. A single agent interaction can generate hundreds of spans, and high-traffic applications can produce thousands of traces per day.
+
+**[ClickHouse](https://mastra.ai/reference/storage/clickhouse)** is recommended for production observability because it's optimized for high-volume, write-heavy analytics workloads. Use composite storage to route observability to ClickHouse while keeping other data in your primary database:
+
+```typescript
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { MemoryPG, WorkflowsPG, ScoresPG } from '@mastra/pg'
+import { ObservabilityStorageClickhouseVNext } from '@mastra/clickhouse'
+
+const storage = new MastraCompositeStore({
+  id: 'composite',
+  domains: {
+    memory: new MemoryPG({ connectionString: process.env.DATABASE_URL }),
+    workflows: new WorkflowsPG({ connectionString: process.env.DATABASE_URL }),
+    scores: new ScoresPG({ connectionString: process.env.DATABASE_URL }),
+    observability: new ObservabilityStorageClickhouseVNext({
+      url: process.env.CLICKHOUSE_URL,
+      username: process.env.CLICKHOUSE_USERNAME,
+      password: process.env.CLICKHOUSE_PASSWORD,
+    }),
+  },
+})
+```
+
+> **Note:** `ObservabilityStorageClickhouseVNext` is the current observability domain implementation. The legacy `ObservabilityStorageClickhouse` class is also exported and remains supported for projects that haven't migrated. See the [ClickHouse storage reference](https://mastra.ai/reference/storage/clickhouse) for details.
+
+### Replicated ClickHouse for multi-replica clusters
+
+For self-managed ClickHouse clusters with multiple replicas, set `replication` so Mastra emits `ReplicatedMergeTree` engines and applies `ON CLUSTER` to its DDL:
+
+```typescript
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { MemoryPG, WorkflowsPG, ScoresPG } from '@mastra/pg'
+import { ObservabilityStorageClickhouseVNext } from '@mastra/clickhouse'
+
+const storage = new MastraCompositeStore({
+  id: 'composite',
+  domains: {
+    memory: new MemoryPG({ connectionString: process.env.DATABASE_URL }),
+    workflows: new WorkflowsPG({ connectionString: process.env.DATABASE_URL }),
+    scores: new ScoresPG({ connectionString: process.env.DATABASE_URL }),
+    observability: new ObservabilityStorageClickhouseVNext({
+      url: process.env.CLICKHOUSE_URL,
+      username: process.env.CLICKHOUSE_USERNAME,
+      password: process.env.CLICKHOUSE_PASSWORD,
+      replication: {
+        cluster: 'production_cluster',
+        // Optional (defaults shown):
+        // zookeeperPath: '/clickhouse/tables/{shard}/{database}/{table}',
+        // replicaName: '{replica}',
+      },
+    }),
+  },
+})
+```
+
+Don't set `replication` on ClickHouse Cloud. Cloud rewrites `MergeTree` to `SharedMergeTree` server-side. See the [ClickHouse storage reference](https://mastra.ai/reference/storage/clickhouse) for the full config shape and operator notes.
+
+> **Info:** This approach is also required when using storage providers that don't support observability (like Convex, DynamoDB, or Cloudflare). See the [MastraStorageExporter documentation](https://mastra.ai/docs/observability/integrations/exporters/mastra-storage) for the full list of supported providers.