@mastra/libsql 1.0.0 → 1.1.0-alpha.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +77 -0
- package/dist/docs/README.md +2 -2
- package/dist/docs/SKILL.md +2 -2
- package/dist/docs/SOURCE_MAP.json +1 -1
- package/dist/docs/agents/01-agent-memory.md +8 -8
- package/dist/docs/agents/02-networks.md +1 -1
- package/dist/docs/agents/03-agent-approval.md +2 -2
- package/dist/docs/agents/04-network-approval.md +2 -2
- package/dist/docs/core/01-reference.md +7 -7
- package/dist/docs/guides/01-ai-sdk.md +9 -30
- package/dist/docs/memory/01-overview.md +22 -53
- package/dist/docs/memory/02-storage.md +115 -87
- package/dist/docs/memory/03-message-history.md +249 -0
- package/dist/docs/memory/{03-working-memory.md → 04-working-memory.md} +22 -1
- package/dist/docs/memory/{04-semantic-recall.md → 05-semantic-recall.md} +45 -22
- package/dist/docs/memory/{05-memory-processors.md → 06-memory-processors.md} +4 -4
- package/dist/docs/memory/{06-reference.md → 07-reference.md} +11 -11
- package/dist/docs/observability/01-overview.md +13 -4
- package/dist/docs/observability/02-default.md +44 -7
- package/dist/docs/rag/01-retrieval.md +4 -4
- package/dist/docs/storage/01-reference.md +31 -17
- package/dist/docs/vectors/01-reference.md +4 -4
- package/dist/docs/workflows/01-snapshots.md +14 -14
- package/dist/index.cjs +271 -1
- package/dist/index.cjs.map +1 -1
- package/dist/index.js +272 -2
- package/dist/index.js.map +1 -1
- package/dist/storage/domains/agents/index.d.ts +10 -1
- package/dist/storage/domains/agents/index.d.ts.map +1 -1
- package/dist/storage/domains/observability/index.d.ts +2 -5
- package/dist/storage/domains/observability/index.d.ts.map +1 -1
- package/package.json +8 -8
|
@@ -37,9 +37,32 @@ const agent = new Agent({
|
|
|
37
37
|
});
|
|
38
38
|
```
|
|
39
39
|
|
|
40
|
+
## Using the recall() Method
|
|
41
|
+
|
|
42
|
+
While `listMessages` retrieves messages by thread ID with basic pagination, [`recall()`](https://mastra.ai/reference/memory/recall) adds support for **semantic search**. When you need to find messages by meaning rather than just recency, use `recall()` with a `vectorSearchString`:
|
|
43
|
+
|
|
44
|
+
```typescript
|
|
45
|
+
const memory = await agent.getMemory();
|
|
46
|
+
|
|
47
|
+
// Basic recall - similar to listMessages
|
|
48
|
+
const { messages } = await memory!.recall({
|
|
49
|
+
threadId: "thread-123",
|
|
50
|
+
perPage: 50,
|
|
51
|
+
});
|
|
52
|
+
|
|
53
|
+
// Semantic recall - find messages by meaning
|
|
54
|
+
const { messages: relevantMessages } = await memory!.recall({
|
|
55
|
+
threadId: "thread-123",
|
|
56
|
+
vectorSearchString: "What did we discuss about the project deadline?",
|
|
57
|
+
threadConfig: {
|
|
58
|
+
semanticRecall: true,
|
|
59
|
+
},
|
|
60
|
+
});
|
|
61
|
+
```
|
|
62
|
+
|
|
40
63
|
## Storage configuration
|
|
41
64
|
|
|
42
|
-
Semantic recall relies on a [storage and vector db](https://mastra.ai/reference/
|
|
65
|
+
Semantic recall relies on a [storage and vector db](https://mastra.ai/reference/memory/memory-class) to store messages and their embeddings.
|
|
43
66
|
|
|
44
67
|
```ts {8-16}
|
|
45
68
|
import { Memory } from "@mastra/memory";
|
|
@@ -64,23 +87,23 @@ const agent = new Agent({
|
|
|
64
87
|
|
|
65
88
|
Each vector store page below includes installation instructions, configuration parameters, and usage examples:
|
|
66
89
|
|
|
67
|
-
- [Astra](https://mastra.ai/reference/
|
|
68
|
-
- [Chroma](https://mastra.ai/reference/
|
|
69
|
-
- [Cloudflare Vectorize](https://mastra.ai/reference/
|
|
70
|
-
- [Convex](https://mastra.ai/reference/
|
|
71
|
-
- [Couchbase](https://mastra.ai/reference/
|
|
72
|
-
- [DuckDB](https://mastra.ai/reference/
|
|
73
|
-
- [Elasticsearch](https://mastra.ai/reference/
|
|
74
|
-
- [LanceDB](https://mastra.ai/reference/
|
|
75
|
-
- [libSQL](https://mastra.ai/reference/
|
|
76
|
-
- [MongoDB](https://mastra.ai/reference/
|
|
77
|
-
- [OpenSearch](https://mastra.ai/reference/
|
|
78
|
-
- [Pinecone](https://mastra.ai/reference/
|
|
79
|
-
- [PostgreSQL](https://mastra.ai/reference/
|
|
80
|
-
- [Qdrant](https://mastra.ai/reference/
|
|
81
|
-
- [S3 Vectors](https://mastra.ai/reference/
|
|
82
|
-
- [Turbopuffer](https://mastra.ai/reference/
|
|
83
|
-
- [Upstash](https://mastra.ai/reference/
|
|
90
|
+
- [Astra](https://mastra.ai/reference/vectors/astra)
|
|
91
|
+
- [Chroma](https://mastra.ai/reference/vectors/chroma)
|
|
92
|
+
- [Cloudflare Vectorize](https://mastra.ai/reference/vectors/vectorize)
|
|
93
|
+
- [Convex](https://mastra.ai/reference/vectors/convex)
|
|
94
|
+
- [Couchbase](https://mastra.ai/reference/vectors/couchbase)
|
|
95
|
+
- [DuckDB](https://mastra.ai/reference/vectors/duckdb)
|
|
96
|
+
- [Elasticsearch](https://mastra.ai/reference/vectors/elasticsearch)
|
|
97
|
+
- [LanceDB](https://mastra.ai/reference/vectors/lance)
|
|
98
|
+
- [libSQL](https://mastra.ai/reference/vectors/libsql)
|
|
99
|
+
- [MongoDB](https://mastra.ai/reference/vectors/mongodb)
|
|
100
|
+
- [OpenSearch](https://mastra.ai/reference/vectors/opensearch)
|
|
101
|
+
- [Pinecone](https://mastra.ai/reference/vectors/pinecone)
|
|
102
|
+
- [PostgreSQL](https://mastra.ai/reference/vectors/pg)
|
|
103
|
+
- [Qdrant](https://mastra.ai/reference/vectors/qdrant)
|
|
104
|
+
- [S3 Vectors](https://mastra.ai/reference/vectors/s3vectors)
|
|
105
|
+
- [Turbopuffer](https://mastra.ai/reference/vectors/turbopuffer)
|
|
106
|
+
- [Upstash](https://mastra.ai/reference/vectors/upstash)
|
|
84
107
|
|
|
85
108
|
## Recall configuration
|
|
86
109
|
|
|
@@ -106,7 +129,7 @@ const agent = new Agent({
|
|
|
106
129
|
|
|
107
130
|
## Embedder configuration
|
|
108
131
|
|
|
109
|
-
Semantic recall relies on an [embedding model](https://mastra.ai/reference/
|
|
132
|
+
Semantic recall relies on an [embedding model](https://mastra.ai/reference/memory/memory-class) to convert messages into embeddings. Mastra supports embedding models through the model router using `provider/model` strings, or you can use any [embedding model](https://sdk.vercel.ai/docs/ai-sdk-core/embeddings) compatible with the AI SDK.
|
|
110
133
|
|
|
111
134
|
#### Using the Model Router (Recommended)
|
|
112
135
|
|
|
@@ -127,7 +150,7 @@ const agent = new Agent({
|
|
|
127
150
|
Supported embedding models:
|
|
128
151
|
|
|
129
152
|
- **OpenAI**: `text-embedding-3-small`, `text-embedding-3-large`, `text-embedding-ada-002`
|
|
130
|
-
- **Google**: `gemini-embedding-001
|
|
153
|
+
- **Google**: `gemini-embedding-001`
|
|
131
154
|
|
|
132
155
|
The model router automatically handles API key detection from environment variables (`OPENAI_API_KEY`, `GOOGLE_GENERATIVE_AI_API_KEY`).
|
|
133
156
|
|
|
@@ -152,7 +175,7 @@ const agent = new Agent({
|
|
|
152
175
|
To use FastEmbed (a local embedding model), install `@mastra/fastembed`:
|
|
153
176
|
|
|
154
177
|
```bash npm2yarn
|
|
155
|
-
npm install @mastra/fastembed@
|
|
178
|
+
npm install @mastra/fastembed@latest
|
|
156
179
|
```
|
|
157
180
|
|
|
158
181
|
Then configure it in your memory:
|
|
@@ -205,7 +228,7 @@ const agent = new Agent({
|
|
|
205
228
|
});
|
|
206
229
|
```
|
|
207
230
|
|
|
208
|
-
For detailed information about index configuration options and performance tuning, see the [PgVector configuration guide](https://mastra.ai/reference/
|
|
231
|
+
For detailed information about index configuration options and performance tuning, see the [PgVector configuration guide](https://mastra.ai/reference/vectors/pg#index-configuration-guide).
|
|
209
232
|
|
|
210
233
|
## Disabling
|
|
211
234
|
|
|
@@ -6,7 +6,7 @@ Memory processors transform and filter messages as they pass through an agent wi
|
|
|
6
6
|
|
|
7
7
|
When memory is enabled on an agent, Mastra adds memory processors to the agent's processor pipeline. These processors retrieve message history, working memory, and semantically relevant messages, then persist new messages after the model responds.
|
|
8
8
|
|
|
9
|
-
Memory processors are [processors](https://mastra.ai/docs/
|
|
9
|
+
Memory processors are [processors](https://mastra.ai/docs/agents/processors) that operate specifically on memory-related messages and state.
|
|
10
10
|
|
|
11
11
|
## Built-in Memory Processors
|
|
12
12
|
|
|
@@ -311,8 +311,8 @@ Both scenarios are safe - guardrails prevent inappropriate content from being pe
|
|
|
311
311
|
|
|
312
312
|
## Related documentation
|
|
313
313
|
|
|
314
|
-
- [Processors](https://mastra.ai/docs/
|
|
315
|
-
- [Guardrails](https://mastra.ai/docs/
|
|
316
|
-
- [Memory Overview](https://mastra.ai/docs/
|
|
314
|
+
- [Processors](https://mastra.ai/docs/agents/processors) - General processor concepts and custom processor creation
|
|
315
|
+
- [Guardrails](https://mastra.ai/docs/agents/guardrails) - Security and validation processors
|
|
316
|
+
- [Memory Overview](https://mastra.ai/docs/memory/overview) - Memory types and configuration
|
|
317
317
|
|
|
318
318
|
When creating custom processors avoid mutating the input `messages` array or its objects directly.
|
|
@@ -120,14 +120,14 @@ export const agent = new Agent({
|
|
|
120
120
|
|
|
121
121
|
### Related
|
|
122
122
|
|
|
123
|
-
- [Getting Started with Memory](https://mastra.ai/docs/
|
|
124
|
-
- [Semantic Recall](https://mastra.ai/docs/
|
|
125
|
-
- [Working Memory](https://mastra.ai/docs/
|
|
126
|
-
- [Memory Processors](https://mastra.ai/docs/
|
|
127
|
-
- [createThread](https://mastra.ai/reference/
|
|
128
|
-
- [recall](https://mastra.ai/reference/
|
|
129
|
-
- [getThreadById](https://mastra.ai/reference/
|
|
130
|
-
- [listThreads](https://mastra.ai/reference/
|
|
131
|
-
- [deleteMessages](https://mastra.ai/reference/
|
|
132
|
-
- [cloneThread](https://mastra.ai/reference/
|
|
133
|
-
- [Clone Utility Methods](https://mastra.ai/reference/
|
|
123
|
+
- [Getting Started with Memory](https://mastra.ai/docs/memory/overview)
|
|
124
|
+
- [Semantic Recall](https://mastra.ai/docs/memory/semantic-recall)
|
|
125
|
+
- [Working Memory](https://mastra.ai/docs/memory/working-memory)
|
|
126
|
+
- [Memory Processors](https://mastra.ai/docs/memory/memory-processors)
|
|
127
|
+
- [createThread](https://mastra.ai/reference/memory/createThread)
|
|
128
|
+
- [recall](https://mastra.ai/reference/memory/recall)
|
|
129
|
+
- [getThreadById](https://mastra.ai/reference/memory/getThreadById)
|
|
130
|
+
- [listThreads](https://mastra.ai/reference/memory/listThreads)
|
|
131
|
+
- [deleteMessages](https://mastra.ai/reference/memory/deleteMessages)
|
|
132
|
+
- [cloneThread](https://mastra.ai/reference/memory/cloneThread)
|
|
133
|
+
- [Clone Utility Methods](https://mastra.ai/reference/memory/clone-utilities)
|
|
@@ -15,6 +15,12 @@ Specialized tracing for AI operations that captures:
|
|
|
15
15
|
- **Workflow steps**: Branching logic, parallel execution, and step outputs
|
|
16
16
|
- **Automatic instrumentation**: Tracing with decorators
|
|
17
17
|
|
|
18
|
+
## Storage Requirements
|
|
19
|
+
|
|
20
|
+
The `DefaultExporter` persists traces to your configured storage backend. Not all storage providers support observability—for the full list, see [Storage Provider Support](https://mastra.ai/docs/observability/tracing/exporters/default#storage-provider-support).
|
|
21
|
+
|
|
22
|
+
For production environments with high traffic, we recommend using **ClickHouse** for the observability domain via [composite storage](https://mastra.ai/reference/storage/composite). See [Production Recommendations](https://mastra.ai/docs/observability/tracing/exporters/default#production-recommendations) for details.
|
|
23
|
+
|
|
18
24
|
## Quick Start
|
|
19
25
|
|
|
20
26
|
Configure Observability in your Mastra instance:
|
|
@@ -53,12 +59,15 @@ export const mastra = new Mastra({
|
|
|
53
59
|
});
|
|
54
60
|
```
|
|
55
61
|
|
|
62
|
+
> **Serverless environments**
|
|
63
|
+
The `file:./mastra.db` storage URL uses the local filesystem, which doesn't work in serverless environments like Vercel, AWS Lambda, or Cloudflare Workers. For serverless deployments, use external storage. See the [Vercel deployment guide](https://mastra.ai/guides/deployment/vercel-deployer#observability) for a complete example.
|
|
64
|
+
|
|
56
65
|
With this basic setup, you will see Traces and Logs in both Studio and in Mastra Cloud.
|
|
57
66
|
|
|
58
|
-
We also support various external tracing providers like MLflow, Langfuse, Braintrust, and any OpenTelemetry-compatible platform (Datadog, New Relic, SigNoz, etc.). See more about this in the [Tracing](https://mastra.ai/docs/
|
|
67
|
+
We also support various external tracing providers like MLflow, Langfuse, Braintrust, and any OpenTelemetry-compatible platform (Datadog, New Relic, SigNoz, etc.). See more about this in the [Tracing](https://mastra.ai/docs/observability/tracing/overview) documentation.
|
|
59
68
|
|
|
60
69
|
## What's Next?
|
|
61
70
|
|
|
62
|
-
- **[Set up Tracing](https://mastra.ai/docs/
|
|
63
|
-
- **[Configure Logging](https://mastra.ai/docs/
|
|
64
|
-
- **[API Reference](https://mastra.ai/reference/
|
|
71
|
+
- **[Set up Tracing](https://mastra.ai/docs/observability/tracing/overview)**: Configure tracing for your application
|
|
72
|
+
- **[Configure Logging](https://mastra.ai/docs/observability/logging)**: Add structured logging
|
|
73
|
+
- **[API Reference](https://mastra.ai/reference/observability/tracing/instances)**: Detailed configuration options
|
|
@@ -4,6 +4,9 @@
|
|
|
4
4
|
|
|
5
5
|
The `DefaultExporter` persists traces to your configured storage backend, making them accessible through Studio. It's automatically enabled when using the default observability configuration and requires no external services.
|
|
6
6
|
|
|
7
|
+
> **Production Observability**
|
|
8
|
+
Observability data can quickly overwhelm general-purpose databases in production. For high-traffic applications, we recommend using **ClickHouse** for the observability storage domain via [composite storage](https://mastra.ai/reference/storage/composite). See [Production Recommendations](#production-recommendations) for details.
|
|
9
|
+
|
|
7
10
|
## Configuration
|
|
8
11
|
|
|
9
12
|
### Prerequisites
|
|
@@ -110,14 +113,30 @@ new DefaultExporter({
|
|
|
110
113
|
|
|
111
114
|
## Storage Provider Support
|
|
112
115
|
|
|
113
|
-
Different storage providers support different tracing strategies.
|
|
116
|
+
Different storage providers support different tracing strategies. Some providers support observability for production workloads, while others are intended primarily for local development.
|
|
114
117
|
|
|
115
118
|
If you set the strategy to `'auto'`, the `DefaultExporter` automatically selects the optimal strategy for the storage provider. If you set the strategy to a mode that the storage provider doesn't support, you will get an error message.
|
|
116
119
|
|
|
117
|
-
|
|
120
|
+
### Providers with Observability Support
|
|
121
|
+
|
|
122
|
+
| Storage Provider | Preferred Strategy | Supported Strategies | Recommended Use |
|
|
118
123
|
| ----------------------------------------------- | ------------------ | ----------------------------------------- | ------------------------------------- |
|
|
119
|
-
| **
|
|
120
|
-
| **[PostgreSQL](https://mastra.ai/reference/
|
|
124
|
+
| **ClickHouse** (`@mastra/clickhouse`) | insert-only | insert-only | Production (high-volume) |
|
|
125
|
+
| **[PostgreSQL](https://mastra.ai/reference/storage/postgresql)** | batch-with-updates | batch-with-updates, insert-only | Production (low volume) |
|
|
126
|
+
| **[MSSQL](https://mastra.ai/reference/storage/mssql)** | batch-with-updates | batch-with-updates, insert-only | Production (low volume) |
|
|
127
|
+
| **[MongoDB](https://mastra.ai/reference/storage/mongodb)** | batch-with-updates | batch-with-updates, insert-only | Production (low volume) |
|
|
128
|
+
| **[libSQL](https://mastra.ai/reference/storage/libsql)** | batch-with-updates | batch-with-updates, insert-only | Default storage, good for development |
|
|
129
|
+
|
|
130
|
+
### Providers without Observability Support
|
|
131
|
+
|
|
132
|
+
The following storage providers **do not support** the observability domain. If you're using one of these providers and need observability, use [composite storage](https://mastra.ai/reference/storage/composite) to route observability data to a supported provider:
|
|
133
|
+
|
|
134
|
+
- [Convex](https://mastra.ai/reference/storage/convex)
|
|
135
|
+
- [DynamoDB](https://mastra.ai/reference/storage/dynamodb)
|
|
136
|
+
- [Cloudflare D1](https://mastra.ai/reference/storage/cloudflare-d1)
|
|
137
|
+
- [Cloudflare Durable Objects](https://mastra.ai/reference/storage/cloudflare)
|
|
138
|
+
- [Upstash](https://mastra.ai/reference/storage/upstash)
|
|
139
|
+
- [LanceDB](https://mastra.ai/reference/storage/lance)
|
|
121
140
|
|
|
122
141
|
### Strategy Benefits
|
|
123
142
|
|
|
@@ -125,6 +144,23 @@ If you set the strategy to `'auto'`, the `DefaultExporter` automatically selects
|
|
|
125
144
|
- **batch-with-updates**: 10-100x throughput improvement, full span lifecycle
|
|
126
145
|
- **insert-only**: Additional 70% reduction in database operations, perfect for analytics
|
|
127
146
|
|
|
147
|
+
## Production Recommendations
|
|
148
|
+
|
|
149
|
+
Observability data grows quickly in production environments. A single agent interaction can generate hundreds of spans, and high-traffic applications can produce thousands of traces per day. Most general-purpose databases aren't optimized for this write-heavy, append-only workload.
|
|
150
|
+
|
|
151
|
+
### Recommended: ClickHouse for High-Volume Production
|
|
152
|
+
|
|
153
|
+
[ClickHouse](https://mastra.ai/reference/storage/composite#specialized-storage-for-observability) is a columnar database designed for high-volume analytics workloads. It's the recommended choice for production observability because:
|
|
154
|
+
|
|
155
|
+
- **Optimized for writes**: Handles millions of inserts per second
|
|
156
|
+
- **Efficient compression**: Reduces storage costs for trace data
|
|
157
|
+
- **Fast queries**: Columnar storage enables quick trace lookups and aggregations
|
|
158
|
+
- **Time-series native**: Built-in support for time-based data retention and partitioning
|
|
159
|
+
|
|
160
|
+
### Using Composite Storage
|
|
161
|
+
|
|
162
|
+
If you're using a provider without observability support (like Convex or DynamoDB) or want to optimize performance, use [composite storage](https://mastra.ai/reference/storage/composite#specialized-storage-for-observability) to route observability data to ClickHouse while keeping other data in your primary database.
|
|
163
|
+
|
|
128
164
|
## Batching Behavior
|
|
129
165
|
|
|
130
166
|
### Flush Triggers
|
|
@@ -172,6 +208,7 @@ new DefaultExporter({
|
|
|
172
208
|
|
|
173
209
|
## Related
|
|
174
210
|
|
|
175
|
-
- [Tracing Overview](https://mastra.ai/docs/
|
|
176
|
-
- [CloudExporter](https://mastra.ai/docs/
|
|
177
|
-
- [Storage
|
|
211
|
+
- [Tracing Overview](https://mastra.ai/docs/observability/tracing/overview)
|
|
212
|
+
- [CloudExporter](https://mastra.ai/docs/observability/tracing/exporters/cloud)
|
|
213
|
+
- [Composite Storage](https://mastra.ai/reference/storage/composite) - Combine multiple storage providers
|
|
214
|
+
- [Storage Configuration](https://mastra.ai/docs/memory/storage)
|
|
@@ -73,7 +73,7 @@ Filter results based on metadata fields to narrow down the search space. This ap
|
|
|
73
73
|
|
|
74
74
|
This is useful when you have documents from different sources, time periods, or with specific attributes. Mastra provides a unified MongoDB-style query syntax that works across all supported vector stores.
|
|
75
75
|
|
|
76
|
-
For detailed information about available operators and syntax, see the [Metadata Filters Reference](https://mastra.ai/reference/
|
|
76
|
+
For detailed information about available operators and syntax, see the [Metadata Filters Reference](https://mastra.ai/reference/rag/metadata-filters).
|
|
77
77
|
|
|
78
78
|
Basic filtering examples:
|
|
79
79
|
|
|
@@ -264,7 +264,7 @@ await pineconeQueryTool.execute(
|
|
|
264
264
|
);
|
|
265
265
|
```
|
|
266
266
|
|
|
267
|
-
For detailed configuration options and advanced usage, see the [Vector Query Tool Reference](https://mastra.ai/reference/
|
|
267
|
+
For detailed configuration options and advanced usage, see the [Vector Query Tool Reference](https://mastra.ai/reference/tools/vector-query-tool).
|
|
268
268
|
|
|
269
269
|
### Vector Store Prompts
|
|
270
270
|
|
|
@@ -543,6 +543,6 @@ const relevanceProvider = new ZeroEntropyRelevanceScorer("zerank-1");
|
|
|
543
543
|
|
|
544
544
|
The re-ranked results combine vector similarity with semantic understanding to improve retrieval quality.
|
|
545
545
|
|
|
546
|
-
For more details about re-ranking, see the [rerank()](https://mastra.ai/reference/
|
|
546
|
+
For more details about re-ranking, see the [rerank()](https://mastra.ai/reference/rag/rerankWithScorer) method.
|
|
547
547
|
|
|
548
|
-
For graph-based retrieval that follows connections between chunks, see the [GraphRAG](https://mastra.ai/docs/
|
|
548
|
+
For graph-based retrieval that follows connections between chunks, see the [GraphRAG](https://mastra.ai/docs/rag/graph-rag) documentation.
|
|
@@ -15,14 +15,14 @@
|
|
|
15
15
|
|
|
16
16
|
`MastraCompositeStore` is included in `@mastra/core`:
|
|
17
17
|
|
|
18
|
-
```bash
|
|
19
|
-
npm install @mastra/core@
|
|
18
|
+
```bash npm2yarn
|
|
19
|
+
npm install @mastra/core@latest
|
|
20
20
|
```
|
|
21
21
|
|
|
22
22
|
You'll also need to install the storage providers you want to compose:
|
|
23
23
|
|
|
24
|
-
```bash
|
|
25
|
-
npm install @mastra/pg@
|
|
24
|
+
```bash npm2yarn
|
|
25
|
+
npm install @mastra/pg@latest @mastra/libsql@latest
|
|
26
26
|
```
|
|
27
27
|
|
|
28
28
|
## Storage domains
|
|
@@ -159,7 +159,9 @@ const storage = new MastraCompositeStore({
|
|
|
159
159
|
|
|
160
160
|
### Specialized storage for observability
|
|
161
161
|
|
|
162
|
-
|
|
162
|
+
Observability data can quickly overwhelm general-purpose databases in production. A single agent interaction can generate hundreds of spans, and high-traffic applications can produce thousands of traces per day.
|
|
163
|
+
|
|
164
|
+
**ClickHouse** is recommended for production observability because it's optimized for high-volume, write-heavy analytics workloads. Use composite storage to route observability to ClickHouse while keeping other data in your primary database:
|
|
163
165
|
|
|
164
166
|
```typescript
|
|
165
167
|
import { MastraCompositeStore } from "@mastra/core/storage";
|
|
@@ -181,6 +183,10 @@ const storage = new MastraCompositeStore({
|
|
|
181
183
|
});
|
|
182
184
|
```
|
|
183
185
|
|
|
186
|
+
> **Note:**
|
|
187
|
+
|
|
188
|
+
This approach is also required when using storage providers that don't support observability (like Convex, DynamoDB, or Cloudflare). See the [DefaultExporter documentation](https://mastra.ai/docs/observability/tracing/exporters/default#storage-provider-support) for the full list of supported providers.
|
|
189
|
+
|
|
184
190
|
---
|
|
185
191
|
|
|
186
192
|
## Reference: DynamoDB Storage
|
|
@@ -189,24 +195,26 @@ const storage = new MastraCompositeStore({
|
|
|
189
195
|
|
|
190
196
|
The DynamoDB storage implementation provides a scalable and performant NoSQL database solution for Mastra, leveraging a single-table design pattern with [ElectroDB](https://electrodb.dev/).
|
|
191
197
|
|
|
198
|
+
> **Observability Not Supported**
|
|
199
|
+
DynamoDB storage **does not support the observability domain**. Traces from the `DefaultExporter` cannot be persisted to DynamoDB, and Mastra Studio's observability features won't work with DynamoDB as your only storage provider. To enable observability, use [composite storage](https://mastra.ai/reference/storage/composite#specialized-storage-for-observability) to route observability data to a supported provider like ClickHouse or PostgreSQL.
|
|
200
|
+
|
|
201
|
+
> **Item Size Limit**
|
|
202
|
+
DynamoDB enforces a **400 KB maximum item size**. This limit can be exceeded when storing messages with base64-encoded attachments such as images. See [Handling large attachments](https://mastra.ai/docs/memory/storage#handling-large-attachments) for workarounds including uploading attachments to external storage.
|
|
203
|
+
|
|
192
204
|
## Features
|
|
193
205
|
|
|
194
206
|
- Efficient single-table design for all Mastra storage needs
|
|
195
207
|
- Based on ElectroDB for type-safe DynamoDB access
|
|
196
208
|
- Support for AWS credentials, regions, and endpoints
|
|
197
209
|
- Compatible with AWS DynamoDB Local for development
|
|
198
|
-
- Stores Thread, Message,
|
|
210
|
+
- Stores Thread, Message, Eval, and Workflow data
|
|
199
211
|
- Optimized for serverless environments
|
|
200
212
|
- Configurable TTL (Time To Live) for automatic data expiration per entity type
|
|
201
213
|
|
|
202
214
|
## Installation
|
|
203
215
|
|
|
204
|
-
```bash
|
|
205
|
-
npm install @mastra/dynamodb@
|
|
206
|
-
# or
|
|
207
|
-
pnpm add @mastra/dynamodb@beta
|
|
208
|
-
# or
|
|
209
|
-
yarn add @mastra/dynamodb@beta
|
|
216
|
+
```bash npm2yarn
|
|
217
|
+
npm install @mastra/dynamodb@latest
|
|
210
218
|
```
|
|
211
219
|
|
|
212
220
|
## Prerequisites
|
|
@@ -438,14 +446,14 @@ This implementation uses a single-table design pattern with ElectroDB, which off
|
|
|
438
446
|
|
|
439
447
|
[libSQL](https://docs.turso.tech/libsql) is an open-source, SQLite-compatible database that supports both local and remote deployments. It can be used to store message history, workflow snapshots, traces, and eval scores.
|
|
440
448
|
|
|
441
|
-
For vectors like semantic recall or traditional RAG, use [libSQL Vector](https://mastra.ai/reference/
|
|
449
|
+
For vectors like semantic recall or traditional RAG, use [libSQL Vector](https://mastra.ai/reference/vectors/libsql) which covers embeddings and vector search.
|
|
442
450
|
|
|
443
451
|
## Installation
|
|
444
452
|
|
|
445
453
|
Storage providers must be installed as separate packages:
|
|
446
454
|
|
|
447
|
-
```bash
|
|
448
|
-
npm install @mastra/libsql@
|
|
455
|
+
```bash npm2yarn
|
|
456
|
+
npm install @mastra/libsql@latest
|
|
449
457
|
```
|
|
450
458
|
|
|
451
459
|
## Usage
|
|
@@ -508,7 +516,7 @@ In-memory storage resets when the process changes. Only suitable for development
|
|
|
508
516
|
|
|
509
517
|
## Initialization
|
|
510
518
|
|
|
511
|
-
When you pass storage to the Mastra class, `init()` is called automatically to create the [core schema](https://mastra.ai/reference/
|
|
519
|
+
When you pass storage to the Mastra class, `init()` is called automatically to create the [core schema](https://mastra.ai/reference/storage/overview#core-schema):
|
|
512
520
|
|
|
513
521
|
```typescript
|
|
514
522
|
import { Mastra } from "@mastra/core";
|
|
@@ -539,4 +547,10 @@ await storage.init();
|
|
|
539
547
|
// Access domain-specific stores via getStore()
|
|
540
548
|
const memoryStore = await storage.getStore('memory');
|
|
541
549
|
const thread = await memoryStore?.getThreadById({ threadId: "..." });
|
|
542
|
-
```
|
|
550
|
+
```
|
|
551
|
+
|
|
552
|
+
## Observability
|
|
553
|
+
|
|
554
|
+
libSQL supports observability and is ideal for local development. Use the `realtime` [tracing strategy](https://mastra.ai/docs/observability/tracing/exporters/default#tracing-strategies) for immediate visibility while debugging.
|
|
555
|
+
|
|
556
|
+
For production environments with higher trace volumes, consider using [PostgreSQL](https://mastra.ai/reference/storage/postgresql) or [ClickHouse via composite storage](https://mastra.ai/reference/storage/composite#specialized-storage-for-observability).
|
|
@@ -14,8 +14,8 @@ It's part of the `@mastra/libsql` package and offers efficient vector similarity
|
|
|
14
14
|
|
|
15
15
|
## Installation
|
|
16
16
|
|
|
17
|
-
```bash
|
|
18
|
-
npm install @mastra/libsql@
|
|
17
|
+
```bash npm2yarn
|
|
18
|
+
npm install @mastra/libsql@latest
|
|
19
19
|
```
|
|
20
20
|
|
|
21
21
|
## Usage
|
|
@@ -168,8 +168,8 @@ Embeddings are numeric vectors used by memory's `semanticRecall` to retrieve rel
|
|
|
168
168
|
|
|
169
169
|
Install `fastembed` to get started:
|
|
170
170
|
|
|
171
|
-
```bash
|
|
172
|
-
npm install @mastra/fastembed@
|
|
171
|
+
```bash npm2yarn
|
|
172
|
+
npm install @mastra/fastembed@latest
|
|
173
173
|
```
|
|
174
174
|
|
|
175
175
|
Add the following to your agent:
|
|
@@ -85,9 +85,9 @@ Snapshots are saved to the configured storage system. By default, they use libSQ
|
|
|
85
85
|
|
|
86
86
|
Read more about:
|
|
87
87
|
|
|
88
|
-
- [libSQL Storage](https://mastra.ai/reference/
|
|
89
|
-
- [Upstash Storage](https://mastra.ai/reference/
|
|
90
|
-
- [PostgreSQL Storage](https://mastra.ai/reference/
|
|
88
|
+
- [libSQL Storage](https://mastra.ai/reference/storage/libsql)
|
|
89
|
+
- [Upstash Storage](https://mastra.ai/reference/storage/upstash)
|
|
90
|
+
- [PostgreSQL Storage](https://mastra.ai/reference/storage/postgresql)
|
|
91
91
|
|
|
92
92
|
### Saving snapshots
|
|
93
93
|
|
|
@@ -139,13 +139,13 @@ export const mastra = new Mastra({
|
|
|
139
139
|
});
|
|
140
140
|
```
|
|
141
141
|
|
|
142
|
-
- [libSQL Storage](https://mastra.ai/reference/
|
|
143
|
-
- [PostgreSQL Storage](https://mastra.ai/reference/
|
|
144
|
-
- [MongoDB Storage](https://mastra.ai/reference/
|
|
145
|
-
- [Upstash Storage](https://mastra.ai/reference/
|
|
146
|
-
- [Cloudflare D1](https://mastra.ai/reference/
|
|
147
|
-
- [DynamoDB](https://mastra.ai/reference/
|
|
148
|
-
- [More storage providers](https://mastra.ai/docs/
|
|
142
|
+
- [libSQL Storage](https://mastra.ai/reference/storage/libsql)
|
|
143
|
+
- [PostgreSQL Storage](https://mastra.ai/reference/storage/postgresql)
|
|
144
|
+
- [MongoDB Storage](https://mastra.ai/reference/storage/mongodb)
|
|
145
|
+
- [Upstash Storage](https://mastra.ai/reference/storage/upstash)
|
|
146
|
+
- [Cloudflare D1](https://mastra.ai/reference/storage/cloudflare-d1)
|
|
147
|
+
- [DynamoDB](https://mastra.ai/reference/storage/dynamodb)
|
|
148
|
+
- [More storage providers](https://mastra.ai/docs/memory/storage)
|
|
149
149
|
|
|
150
150
|
## Best practices
|
|
151
151
|
|
|
@@ -234,7 +234,7 @@ if (result.status === "suspended") {
|
|
|
234
234
|
|
|
235
235
|
## Related
|
|
236
236
|
|
|
237
|
-
- [Control Flow](https://mastra.ai/docs/
|
|
238
|
-
- [Suspend & Resume](https://mastra.ai/docs/
|
|
239
|
-
- [Time Travel](https://mastra.ai/docs/
|
|
240
|
-
- [Human-in-the-loop](https://mastra.ai/docs/
|
|
237
|
+
- [Control Flow](https://mastra.ai/docs/workflows/control-flow)
|
|
238
|
+
- [Suspend & Resume](https://mastra.ai/docs/workflows/suspend-and-resume)
|
|
239
|
+
- [Time Travel](https://mastra.ai/docs/workflows/time-travel)
|
|
240
|
+
- [Human-in-the-loop](https://mastra.ai/docs/workflows/human-in-the-loop)
|