@mastra/core 1.7.0 → 1.8.0

package/dist/docs/references/reference-storage-cloudflare-d1.md

@@ -0,0 +1,218 @@
+# Cloudflare D1 Storage
+
+The Cloudflare D1 storage implementation provides a serverless SQL database solution using Cloudflare D1, supporting relational operations and transactional consistency.
+
+> **Observability Not Supported:** Cloudflare D1 storage **does not support the observability domain**. Traces from the `DefaultExporter` cannot be persisted to D1, and Mastra Studio's observability features won't work with D1 as your only storage provider. To enable observability, use [composite storage](https://mastra.ai/reference/storage/composite) to route observability data to a supported provider like ClickHouse or PostgreSQL.
+
+> **Row Size Limit:** Cloudflare D1 enforces a **1 MiB maximum row size**. This limit can be exceeded when storing messages with base64-encoded attachments such as images. See [Handling large attachments](https://mastra.ai/docs/memory/storage) for workarounds including uploading attachments to external storage.
+
+## Installation
+
+**npm**:
+
+```bash
+npm install @mastra/cloudflare-d1@latest
+```
+
+**pnpm**:
+
+```bash
+pnpm add @mastra/cloudflare-d1@latest
+```
+
+**Yarn**:
+
+```bash
+yarn add @mastra/cloudflare-d1@latest
+```
+
+**Bun**:
+
+```bash
+bun add @mastra/cloudflare-d1@latest
+```
+
+## Usage
+
+### Using with Cloudflare Workers
+
+When using D1Store in a Cloudflare Worker, you need to access the D1 binding from the worker's `env` parameter at runtime. The `D1Database` in your type definition is only for TypeScript type checking—the actual binding is provided by the Workers runtime.
+
+```typescript
+import { D1Store } from '@mastra/cloudflare-d1'
+import { Mastra } from '@mastra/core'
+import { CloudflareDeployer } from '@mastra/deployer-cloudflare'
+
+type Env = {
+  D1Database: D1Database // TypeScript type definition
+}
+
+// Factory function to create Mastra with D1 binding
+function createMastra(env: Env) {
+  const storage = new D1Store({
+    binding: env.D1Database, // ✅ Access the actual binding from env
+    tablePrefix: 'dev_', // Optional: isolate tables per environment
+  })
+
+  return new Mastra({
+    storage,
+    deployer: new CloudflareDeployer({
+      name: 'my-worker',
+      d1_databases: [
+        {
+          binding: 'D1Database', // Must match the property name in Env type
+          database_name: 'your-database-name',
+          database_id: 'your-database-id',
+        },
+      ],
+    }),
+  })
+}
+
+// Cloudflare Worker export
+export default {
+  async fetch(request: Request, env: Env, ctx: ExecutionContext) {
+    const mastra = createMastra(env)
+
+    // Your handler logic here
+    return new Response('Hello from Mastra with D1!')
+  },
+}
+```
+
+> **Important: Understanding D1 Bindings:** In the `Env` type definition, `D1Database: D1Database` serves two purposes:
+>
+> - The **property name** (`D1Database`) must match the `binding` name in your `wrangler.toml`
+> - The **type** (`: D1Database`) is from `@cloudflare/workers-types` for TypeScript type checking
+>
+> At runtime, Cloudflare Workers provides the actual D1 database instance via `env.D1Database`. You cannot use `D1Database` directly outside of the worker's context.
+
+### Using with REST API
+
+For non-Workers environments (Node.js, serverless functions, etc.), use the REST API approach:
+
+```typescript
+import { D1Store } from '@mastra/cloudflare-d1'
+
+const storage = new D1Store({
+  accountId: process.env.CLOUDFLARE_ACCOUNT_ID!, // Cloudflare Account ID
+  databaseId: process.env.CLOUDFLARE_D1_DATABASE_ID!, // D1 Database ID
+  apiToken: process.env.CLOUDFLARE_API_TOKEN!, // Cloudflare API Token
+  tablePrefix: 'dev_', // Optional: isolate tables per environment
+})
+```
+
+### Wrangler Configuration
+
+Add the D1 database binding to your `wrangler.toml`:
+
+```toml
+[[d1_databases]]
+binding = "D1Database"  # Must match the property name in your Env type
+database_name = "your-database-name"
+database_id = "your-database-id"
+```
+
+Or in `wrangler.jsonc`:
+
+```jsonc
+{
+  "d1_databases": [
+    {
+      "binding": "D1Database",
+      "database_name": "your-database-name",
+      "database_id": "your-database-id",
+    },
+  ],
+}
+```
+
+## Parameters
+
+**binding?:** (`D1Database`): Cloudflare D1 Workers binding (for Workers runtime)
+
+**accountId?:** (`string`): Cloudflare Account ID (for REST API)
+
+**databaseId?:** (`string`): Cloudflare D1 Database ID (for REST API)
+
+**apiToken?:** (`string`): Cloudflare API Token (for REST API)
+
+**tablePrefix?:** (`string`): Optional prefix for all table names (useful for environment isolation)
+
+## Additional Notes
+
+### Schema Management
+
+The storage implementation handles schema creation and updates automatically. It creates the following tables:
+
+- `threads`: Stores conversation threads
+- `messages`: Stores individual messages
+- `metadata`: Stores additional metadata for threads and messages
+
+### Initialization
+
+When you pass storage to the Mastra class, `init()` is called automatically before any storage operation:
+
+```typescript
+import { Mastra } from '@mastra/core'
+import { D1Store } from '@mastra/cloudflare-d1'
+
+type Env = {
+  D1Database: D1Database
+}
+
+// In a Cloudflare Worker
+export default {
+  async fetch(request: Request, env: Env, ctx: ExecutionContext) {
+    const storage = new D1Store({
+      binding: env.D1Database, // ✅ Use env.D1Database
+    })
+
+    const mastra = new Mastra({
+      storage, // init() is called automatically
+    })
+
+    // Your handler logic here
+    return new Response('Success')
+  },
+}
+```
+
+If you're using storage directly without Mastra, you must call `init()` explicitly to create the tables:
+
+```typescript
+import { D1Store } from '@mastra/cloudflare-d1'
+
+type Env = {
+  D1Database: D1Database
+}
+
+// In a Cloudflare Worker
+export default {
+  async fetch(request: Request, env: Env, ctx: ExecutionContext) {
+    const storage = new D1Store({
+      id: 'd1-storage',
+      binding: env.D1Database, // ✅ Use env.D1Database
+    })
+
+    // Required when using storage directly
+    await storage.init()
+
+    // Access domain-specific stores via getStore()
+    const memoryStore = await storage.getStore('memory')
+    const thread = await memoryStore?.getThreadById({ threadId: '...' })
+
+    return new Response('Success')
+  },
+}
+```
+
+> **Warning:** If `init()` is not called, tables won't be created and storage operations will fail silently or throw errors.
+
+### Transactions & Consistency
+
+Cloudflare D1 provides transactional guarantees for single-row operations. This means that multiple operations can be executed as a single, all-or-nothing unit of work.
+
+### Table Creation & Migrations
+
+Tables are created automatically when storage is initialized (and can be isolated per environment using the `tablePrefix` option), but advanced schema changes—such as adding columns, changing data types, or modifying indexes—require manual migration and careful planning to avoid data loss.

package/dist/docs/references/reference-storage-composite.md

@@ -0,0 +1,235 @@
+# Composite Storage
+
+`MastraCompositeStore` can compose storage domains from different providers. Use it when you need different databases for different purposes. For example, use LibSQL for memory and PostgreSQL for workflows.
+
+## Installation
+
+`MastraCompositeStore` is included in `@mastra/core`:
+
+**npm**:
+
+```bash
+npm install @mastra/core@latest
+```
+
+**pnpm**:
+
+```bash
+pnpm add @mastra/core@latest
+```
+
+**Yarn**:
+
+```bash
+yarn add @mastra/core@latest
+```
+
+**Bun**:
+
+```bash
+bun add @mastra/core@latest
+```
+
+You'll also need to install the storage providers you want to compose:
+
+**npm**:
+
+```bash
+npm install @mastra/pg@latest @mastra/libsql@latest
+```
+
+**pnpm**:
+
+```bash
+pnpm add @mastra/pg@latest @mastra/libsql@latest
+```
+
+**Yarn**:
+
+```bash
+yarn add @mastra/pg@latest @mastra/libsql@latest
+```
+
+**Bun**:
+
+```bash
+bun add @mastra/pg@latest @mastra/libsql@latest
+```
+
+## Storage domains
+
+Mastra organizes storage into five specialized domains, each handling a specific type of data. Each domain can be backed by a different storage adapter, and domain classes are exported from each storage package.
+
+| Domain          | Description                                                                                                                                                                           |
+| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `memory`        | Conversation persistence for agents. Stores threads (conversation sessions), messages, resources (user identities), and working memory (persistent context across conversations).     |
+| `workflows`     | Workflow execution state. When workflows suspend for human input, external events, or scheduled resumption, their state is persisted here to enable resumption after server restarts. |
+| `scores`        | Evaluation results from Mastra's evals system. Scores and metrics are persisted here for analysis and comparison over time.                                                           |
+| `observability` | Telemetry data including traces and spans. Agent interactions, tool calls, and LLM requests generate spans collected into traces for debugging and performance analysis.              |
+| `agents`        | Agent configurations for stored agents. Enables agents to be defined and updated at runtime without code deployments.                                                                 |
+
+## Usage
+
+### Basic composition
+
+Import domain classes directly from each store package and compose them:
+
+```typescript
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { WorkflowsPG, ScoresPG } from '@mastra/pg'
+import { MemoryLibSQL } from '@mastra/libsql'
+import { Mastra } from '@mastra/core'
+
+export const mastra = new Mastra({
+  storage: new MastraCompositeStore({
+    id: 'composite',
+    domains: {
+      memory: new MemoryLibSQL({ url: 'file:./local.db' }),
+      workflows: new WorkflowsPG({ connectionString: process.env.DATABASE_URL }),
+      scores: new ScoresPG({ connectionString: process.env.DATABASE_URL }),
+    },
+  }),
+})
+```
+
+### With a default storage
+
+Use `default` to specify a fallback storage, then override specific domains:
+
+```typescript
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { PostgresStore } from '@mastra/pg'
+import { MemoryLibSQL } from '@mastra/libsql'
+import { Mastra } from '@mastra/core'
+
+const pgStore = new PostgresStore({
+  id: 'pg',
+  connectionString: process.env.DATABASE_URL,
+})
+
+export const mastra = new Mastra({
+  storage: new MastraCompositeStore({
+    id: 'composite',
+    default: pgStore,
+    domains: {
+      memory: new MemoryLibSQL({ url: 'file:./local.db' }),
+    },
+  }),
+})
+```
+
+## Options
+
+**id:** (`string`): Unique identifier for this storage instance.
+
+**default?:** (`MastraCompositeStore`): Default storage adapter. Domains not explicitly specified in \`domains\` will use this storage's domains as fallbacks.
+
+**domains?:** (`object`): Individual domain overrides. Each domain can come from a different storage adapter. These take precedence over the default storage.
+
+**domains.memory?:** (`MemoryStorage`): Storage for threads, messages, and resources.
+
+**domains.workflows?:** (`WorkflowsStorage`): Storage for workflow snapshots.
+
+**domains.scores?:** (`ScoresStorage`): Storage for evaluation scores.
+
+**domains.observability?:** (`ObservabilityStorage`): Storage for traces and spans.
+
+**domains.agents?:** (`AgentsStorage`): Storage for stored agent configurations.
+
+**disableInit?:** (`boolean`): When true, automatic initialization is disabled. You must call init() explicitly.
+
+## Initialization
+
+`MastraCompositeStore` initializes each configured domain independently. When passed to the Mastra class, `init()` is called automatically:
+
+```typescript
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { MemoryPG, WorkflowsPG, ScoresPG } from '@mastra/pg'
+import { Mastra } from '@mastra/core'
+
+const storage = new MastraCompositeStore({
+  id: 'composite',
+  domains: {
+    memory: new MemoryPG({ connectionString: process.env.DATABASE_URL }),
+    workflows: new WorkflowsPG({ connectionString: process.env.DATABASE_URL }),
+    scores: new ScoresPG({ connectionString: process.env.DATABASE_URL }),
+  },
+})
+
+export const mastra = new Mastra({
+  storage, // init() called automatically
+})
+```
+
+If using storage directly, call `init()` explicitly:
+
+```typescript
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { MemoryPG } from '@mastra/pg'
+
+const storage = new MastraCompositeStore({
+  id: 'composite',
+  domains: {
+    memory: new MemoryPG({ connectionString: process.env.DATABASE_URL }),
+  },
+})
+
+await storage.init()
+
+// Access domain-specific stores via getStore()
+const memoryStore = await storage.getStore('memory')
+const thread = await memoryStore?.getThreadById({ threadId: '...' })
+```
+
+## Use cases
+
+### Separate databases for different workloads
+
+Use a local database for development while keeping production data in a managed service:
+
+```typescript
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { MemoryPG, WorkflowsPG, ScoresPG } from '@mastra/pg'
+import { MemoryLibSQL } from '@mastra/libsql'
+
+const storage = new MastraCompositeStore({
+  id: 'composite',
+  domains: {
+    // Use local SQLite for development, PostgreSQL for production
+    memory:
+      process.env.NODE_ENV === 'development'
+        ? new MemoryLibSQL({ url: 'file:./dev.db' })
+        : new MemoryPG({ connectionString: process.env.DATABASE_URL }),
+    workflows: new WorkflowsPG({ connectionString: process.env.DATABASE_URL }),
+    scores: new ScoresPG({ connectionString: process.env.DATABASE_URL }),
+  },
+})
+```
+
+### Specialized storage for observability
+
+Observability data can quickly overwhelm general-purpose databases in production. A single agent interaction can generate hundreds of spans, and high-traffic applications can produce thousands of traces per day.
+
+**ClickHouse** is recommended for production observability because it's optimized for high-volume, write-heavy analytics workloads. Use composite storage to route observability to ClickHouse while keeping other data in your primary database:
+
+```typescript
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { MemoryPG, WorkflowsPG, ScoresPG } from '@mastra/pg'
+import { ObservabilityStorageClickhouse } from '@mastra/clickhouse'
+
+const storage = new MastraCompositeStore({
+  id: 'composite',
+  domains: {
+    memory: new MemoryPG({ connectionString: process.env.DATABASE_URL }),
+    workflows: new WorkflowsPG({ connectionString: process.env.DATABASE_URL }),
+    scores: new ScoresPG({ connectionString: process.env.DATABASE_URL }),
+    observability: new ObservabilityStorageClickhouse({
+      url: process.env.CLICKHOUSE_URL,
+      username: process.env.CLICKHOUSE_USERNAME,
+      password: process.env.CLICKHOUSE_PASSWORD,
+    }),
+  },
+})
+```
+
+> **Info:** This approach is also required when using storage providers that don't support observability (like Convex, DynamoDB, or Cloudflare). See the [DefaultExporter documentation](https://mastra.ai/docs/observability/tracing/exporters/default) for the full list of supported providers.

package/dist/docs/references/reference-storage-lance.md

@@ -0,0 +1,131 @@
+# LanceDB Storage
+
+The LanceDB storage implementation provides a high-performance storage solution using the LanceDB database system, which excels at handling both traditional data storage and vector operations.
+
+> **Observability Not Supported:** LanceDB storage **does not support the observability domain**. Traces from the `DefaultExporter` cannot be persisted to LanceDB, and Mastra Studio's observability features won't work with LanceDB as your only storage provider. To enable observability, use [composite storage](https://mastra.ai/reference/storage/composite) to route observability data to a supported provider like ClickHouse or PostgreSQL.
+
+## Installation
+
+**npm**:
+
+```bash
+npm install @mastra/lance@latest
+```
+
+**pnpm**:
+
+```bash
+pnpm add @mastra/lance@latest
+```
+
+**Yarn**:
+
+```bash
+yarn add @mastra/lance@latest
+```
+
+**Bun**:
+
+```bash
+bun add @mastra/lance@latest
+```
+
+## Usage
+
+### Basic Storage Usage
+
+```typescript
+import { LanceStorage } from '@mastra/lance'
+
+// Connect to a local database
+const storage = await LanceStorage.create('my-storage', '/path/to/db')
+
+// Connect to a LanceDB cloud database
+const storage = await LanceStorage.create('my-storage', 'db://host:port')
+
+// Connect to a cloud database with custom options
+const storage = await LanceStorage.create('my-storage', 's3://bucket/db', {
+  storageOptions: { timeout: '60s' },
+})
+```
+
+## Parameters
+
+### LanceStorage.create()
+
+**name:** (`string`): Name identifier for the storage instance
+
+**uri:** (`string`): URI to connect to the LanceDB database. Can be a local path, cloud DB URL, or S3 bucket URL
+
+**options?:** (`ConnectionOptions`): Connection options for LanceDB, such as timeout settings, authentication, etc.
+
+## Additional Notes
+
+### Schema Management
+
+The LanceStorage implementation automatically handles schema creation and updates. It maps Mastra's schema types to Apache Arrow data types, which are used by LanceDB internally:
+
+- `text`, `uuid` → Utf8
+- `int`, `integer` → Int32
+- `float` → Float32
+- `jsonb`, `json` → Utf8 (serialized)
+- `binary` → Binary
+
+### Initialization
+
+When you pass storage to the Mastra class, `init()` is called automatically before any storage operation:
+
+```typescript
+import { Mastra } from '@mastra/core'
+import { LanceStorage } from '@mastra/lance'
+
+const storage = await LanceStorage.create('my-storage', '/path/to/db')
+
+const mastra = new Mastra({
+  storage, // init() is called automatically
+})
+```
+
+If you're using storage directly without Mastra, you must call `init()` explicitly to create the tables:
+
+```typescript
+import { LanceStorage } from '@mastra/lance'
+
+const storage = await LanceStorage.create('my-storage', '/path/to/db')
+
+// Required when using storage directly
+await storage.init()
+
+// Access domain-specific stores via getStore()
+const memoryStore = await storage.getStore('memory')
+const thread = await memoryStore?.getThreadById({ threadId: '...' })
+```
+
+> **Warning:** If `init()` is not called, tables won't be created and storage operations will fail silently or throw errors.
+
+### Deployment Options
+
+LanceDB storage can be configured for different deployment scenarios:
+
+- **Local Development**: Use a local file path for development and testing
+  ```text
+  /path/to/db
+  ```
+- **Cloud Deployment**: Connect to a hosted LanceDB instance
+  ```text
+  db://host:port
+  ```
+- **S3 Storage**: Use Amazon S3 for scalable cloud storage
+  ```text
+  s3://bucket/db
+  ```
+
+### Table Management
+
+LanceStorage provides methods for managing tables:
+
+- Create tables with custom schemas
+- Drop tables
+- Clear tables (delete all records)
+- Load records by key
+- Insert single and batch records