@mastra/pg 1.0.0 → 1.1.0-alpha.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +102 -0
- package/dist/docs/README.md +1 -1
- package/dist/docs/SKILL.md +1 -1
- package/dist/docs/SOURCE_MAP.json +1 -1
- package/dist/docs/memory/01-storage.md +115 -87
- package/dist/docs/memory/02-working-memory.md +22 -1
- package/dist/docs/memory/03-semantic-recall.md +45 -22
- package/dist/docs/memory/04-reference.md +11 -11
- package/dist/docs/processors/01-reference.md +6 -7
- package/dist/docs/rag/02-vector-databases.md +1 -1
- package/dist/docs/rag/03-retrieval.md +4 -4
- package/dist/docs/rag/04-reference.md +10 -10
- package/dist/docs/storage/01-reference.md +107 -30
- package/dist/docs/tools/01-reference.md +2 -2
- package/dist/docs/vectors/01-reference.md +2 -2
- package/dist/index.cjs +305 -82
- package/dist/index.cjs.map +1 -1
- package/dist/index.js +306 -83
- package/dist/index.js.map +1 -1
- package/dist/storage/domains/agents/index.d.ts +11 -1
- package/dist/storage/domains/agents/index.d.ts.map +1 -1
- package/dist/storage/domains/observability/index.d.ts +1 -1
- package/dist/storage/domains/observability/index.d.ts.map +1 -1
- package/package.json +8 -8
|
@@ -75,7 +75,7 @@ export const agent = new Agent({
|
|
|
75
75
|
|
|
76
76
|
## Related
|
|
77
77
|
|
|
78
|
-
- [Guardrails](https://mastra.ai/docs/
|
|
78
|
+
- [Guardrails](https://mastra.ai/docs/agents/guardrails)
|
|
79
79
|
|
|
80
80
|
---
|
|
81
81
|
|
|
@@ -172,7 +172,7 @@ When `scope` is set to `'resource'`, the processor can recall messages from othe
|
|
|
172
172
|
|
|
173
173
|
## Related
|
|
174
174
|
|
|
175
|
-
- [Guardrails](https://mastra.ai/docs/
|
|
175
|
+
- [Guardrails](https://mastra.ai/docs/agents/guardrails)
|
|
176
176
|
|
|
177
177
|
---
|
|
178
178
|
|
|
@@ -276,10 +276,9 @@ const processor = new WorkingMemory({
|
|
|
276
276
|
- Thread metadata (`scope: 'thread'`)
|
|
277
277
|
- Resource record (`scope: 'resource'`)
|
|
278
278
|
3. Resolves the template (from provider, options, or default)
|
|
279
|
-
4. Generates system instructions
|
|
280
|
-
-
|
|
281
|
-
-
|
|
282
|
-
- Current working memory data
|
|
279
|
+
4. Generates system instructions based on mode:
|
|
280
|
+
- **Normal mode**: Includes guidelines for storing/updating information, template structure, and current data
|
|
281
|
+
- **Read-only mode** (`readOnly: true`): Includes only the current data as context without update instructions
|
|
283
282
|
5. Adds the instruction as a system message with `source: 'memory'` tag
|
|
284
283
|
|
|
285
284
|
### Working memory updates
|
|
@@ -294,4 +293,4 @@ If no template is provided, the processor uses a default markdown template with
|
|
|
294
293
|
|
|
295
294
|
## Related
|
|
296
295
|
|
|
297
|
-
- [Guardrails](https://mastra.ai/docs/
|
|
296
|
+
- [Guardrails](https://mastra.ai/docs/agents/guardrails)
|
|
@@ -376,7 +376,7 @@ The dimension size must match the output dimension of your chosen embedding mode
|
|
|
376
376
|
|
|
377
377
|
- OpenAI text-embedding-3-small: 1536 dimensions (or custom, e.g., 256)
|
|
378
378
|
- Cohere embed-multilingual-v3: 1024 dimensions
|
|
379
|
-
- Google
|
|
379
|
+
- Google gemini-embedding-001: 768 dimensions (or custom)
|
|
380
380
|
|
|
381
381
|
> **Note:**
|
|
382
382
|
Index dimensions cannot be changed after creation. To use a different model, delete and recreate the index with the new dimension size.
|
|
@@ -73,7 +73,7 @@ Filter results based on metadata fields to narrow down the search space. This ap
|
|
|
73
73
|
|
|
74
74
|
This is useful when you have documents from different sources, time periods, or with specific attributes. Mastra provides a unified MongoDB-style query syntax that works across all supported vector stores.
|
|
75
75
|
|
|
76
|
-
For detailed information about available operators and syntax, see the [Metadata Filters Reference](https://mastra.ai/reference/
|
|
76
|
+
For detailed information about available operators and syntax, see the [Metadata Filters Reference](https://mastra.ai/reference/rag/metadata-filters).
|
|
77
77
|
|
|
78
78
|
Basic filtering examples:
|
|
79
79
|
|
|
@@ -264,7 +264,7 @@ await pineconeQueryTool.execute(
|
|
|
264
264
|
);
|
|
265
265
|
```
|
|
266
266
|
|
|
267
|
-
For detailed configuration options and advanced usage, see the [Vector Query Tool Reference](https://mastra.ai/reference/
|
|
267
|
+
For detailed configuration options and advanced usage, see the [Vector Query Tool Reference](https://mastra.ai/reference/tools/vector-query-tool).
|
|
268
268
|
|
|
269
269
|
### Vector Store Prompts
|
|
270
270
|
|
|
@@ -543,6 +543,6 @@ const relevanceProvider = new ZeroEntropyRelevanceScorer("zerank-1");
|
|
|
543
543
|
|
|
544
544
|
The re-ranked results combine vector similarity with semantic understanding to improve retrieval quality.
|
|
545
545
|
|
|
546
|
-
For more details about re-ranking, see the [rerank()](https://mastra.ai/reference/
|
|
546
|
+
For more details about re-ranking, see the [rerank()](https://mastra.ai/reference/rag/rerankWithScorer) method.
|
|
547
547
|
|
|
548
|
-
For graph-based retrieval that follows connections between chunks, see the [GraphRAG](https://mastra.ai/docs/
|
|
548
|
+
For graph-based retrieval that follows connections between chunks, see the [GraphRAG](https://mastra.ai/docs/rag/graph-rag) documentation.
|
|
@@ -357,13 +357,13 @@ const results = await store.query({
|
|
|
357
357
|
|
|
358
358
|
## Related
|
|
359
359
|
|
|
360
|
-
- [Astra](https://mastra.ai/reference/
|
|
361
|
-
- [Chroma](https://mastra.ai/reference/
|
|
362
|
-
- [Cloudflare Vectorize](https://mastra.ai/reference/
|
|
363
|
-
- [libSQL](https://mastra.ai/reference/
|
|
364
|
-
- [MongoDB](https://mastra.ai/reference/
|
|
365
|
-
- [PgStore](https://mastra.ai/reference/
|
|
366
|
-
- [Pinecone](https://mastra.ai/reference/
|
|
367
|
-
- [Qdrant](https://mastra.ai/reference/
|
|
368
|
-
- [Upstash](https://mastra.ai/reference/
|
|
369
|
-
- [Amazon S3 Vectors](https://mastra.ai/reference/
|
|
360
|
+
- [Astra](https://mastra.ai/reference/vectors/astra)
|
|
361
|
+
- [Chroma](https://mastra.ai/reference/vectors/chroma)
|
|
362
|
+
- [Cloudflare Vectorize](https://mastra.ai/reference/vectors/vectorize)
|
|
363
|
+
- [libSQL](https://mastra.ai/reference/vectors/libsql)
|
|
364
|
+
- [MongoDB](https://mastra.ai/reference/vectors/mongodb)
|
|
365
|
+
- [PgStore](https://mastra.ai/reference/vectors/pg)
|
|
366
|
+
- [Pinecone](https://mastra.ai/reference/vectors/pinecone)
|
|
367
|
+
- [Qdrant](https://mastra.ai/reference/vectors/qdrant)
|
|
368
|
+
- [Upstash](https://mastra.ai/reference/vectors/upstash)
|
|
369
|
+
- [Amazon S3 Vectors](https://mastra.ai/reference/vectors/s3vectors)
|
|
@@ -15,14 +15,14 @@
|
|
|
15
15
|
|
|
16
16
|
`MastraCompositeStore` is included in `@mastra/core`:
|
|
17
17
|
|
|
18
|
-
```bash
|
|
19
|
-
npm install @mastra/core@
|
|
18
|
+
```bash npm2yarn
|
|
19
|
+
npm install @mastra/core@latest
|
|
20
20
|
```
|
|
21
21
|
|
|
22
22
|
You'll also need to install the storage providers you want to compose:
|
|
23
23
|
|
|
24
|
-
```bash
|
|
25
|
-
npm install @mastra/pg@
|
|
24
|
+
```bash npm2yarn
|
|
25
|
+
npm install @mastra/pg@latest @mastra/libsql@latest
|
|
26
26
|
```
|
|
27
27
|
|
|
28
28
|
## Storage domains
|
|
@@ -159,7 +159,9 @@ const storage = new MastraCompositeStore({
|
|
|
159
159
|
|
|
160
160
|
### Specialized storage for observability
|
|
161
161
|
|
|
162
|
-
|
|
162
|
+
Observability data can quickly overwhelm general-purpose databases in production. A single agent interaction can generate hundreds of spans, and high-traffic applications can produce thousands of traces per day.
|
|
163
|
+
|
|
164
|
+
**ClickHouse** is recommended for production observability because it's optimized for high-volume, write-heavy analytics workloads. Use composite storage to route observability to ClickHouse while keeping other data in your primary database:
|
|
163
165
|
|
|
164
166
|
```typescript
|
|
165
167
|
import { MastraCompositeStore } from "@mastra/core/storage";
|
|
@@ -181,6 +183,10 @@ const storage = new MastraCompositeStore({
|
|
|
181
183
|
});
|
|
182
184
|
```
|
|
183
185
|
|
|
186
|
+
> **Note:**
|
|
187
|
+
|
|
188
|
+
This approach is also required when using storage providers that don't support observability (like Convex, DynamoDB, or Cloudflare). See the [DefaultExporter documentation](https://mastra.ai/docs/observability/tracing/exporters/default#storage-provider-support) for the full list of supported providers.
|
|
189
|
+
|
|
184
190
|
---
|
|
185
191
|
|
|
186
192
|
## Reference: DynamoDB Storage
|
|
@@ -189,24 +195,26 @@ const storage = new MastraCompositeStore({
|
|
|
189
195
|
|
|
190
196
|
The DynamoDB storage implementation provides a scalable and performant NoSQL database solution for Mastra, leveraging a single-table design pattern with [ElectroDB](https://electrodb.dev/).
|
|
191
197
|
|
|
198
|
+
> **Observability Not Supported**
|
|
199
|
+
DynamoDB storage **does not support the observability domain**. Traces from the `DefaultExporter` cannot be persisted to DynamoDB, and Mastra Studio's observability features won't work with DynamoDB as your only storage provider. To enable observability, use [composite storage](https://mastra.ai/reference/storage/composite#specialized-storage-for-observability) to route observability data to a supported provider like ClickHouse or PostgreSQL.
|
|
200
|
+
|
|
201
|
+
> **Item Size Limit**
|
|
202
|
+
DynamoDB enforces a **400 KB maximum item size**. This limit can be exceeded when storing messages with base64-encoded attachments such as images. See [Handling large attachments](https://mastra.ai/docs/memory/storage#handling-large-attachments) for workarounds including uploading attachments to external storage.
|
|
203
|
+
|
|
192
204
|
## Features
|
|
193
205
|
|
|
194
206
|
- Efficient single-table design for all Mastra storage needs
|
|
195
207
|
- Based on ElectroDB for type-safe DynamoDB access
|
|
196
208
|
- Support for AWS credentials, regions, and endpoints
|
|
197
209
|
- Compatible with AWS DynamoDB Local for development
|
|
198
|
-
- Stores Thread, Message,
|
|
210
|
+
- Stores Thread, Message, Eval, and Workflow data
|
|
199
211
|
- Optimized for serverless environments
|
|
200
212
|
- Configurable TTL (Time To Live) for automatic data expiration per entity type
|
|
201
213
|
|
|
202
214
|
## Installation
|
|
203
215
|
|
|
204
|
-
```bash
|
|
205
|
-
npm install @mastra/dynamodb@
|
|
206
|
-
# or
|
|
207
|
-
pnpm add @mastra/dynamodb@beta
|
|
208
|
-
# or
|
|
209
|
-
yarn add @mastra/dynamodb@beta
|
|
216
|
+
```bash npm2yarn
|
|
217
|
+
npm install @mastra/dynamodb@latest
|
|
210
218
|
```
|
|
211
219
|
|
|
212
220
|
## Prerequisites
|
|
@@ -440,8 +448,8 @@ The PostgreSQL storage implementation provides a production-ready storage soluti
|
|
|
440
448
|
|
|
441
449
|
## Installation
|
|
442
450
|
|
|
443
|
-
```bash
|
|
444
|
-
npm install @mastra/pg@
|
|
451
|
+
```bash npm2yarn
|
|
452
|
+
npm install @mastra/pg@latest
|
|
445
453
|
```
|
|
446
454
|
|
|
447
455
|
## Usage
|
|
@@ -463,22 +471,26 @@ You can instantiate `PostgresStore` in the following ways:
|
|
|
463
471
|
|
|
464
472
|
```ts
|
|
465
473
|
import { PostgresStore } from "@mastra/pg";
|
|
474
|
+
import { Pool } from "pg";
|
|
466
475
|
|
|
467
|
-
// Using a connection string
|
|
476
|
+
// Using a connection string
|
|
468
477
|
const store1 = new PostgresStore({
|
|
469
478
|
id: 'pg-storage-1',
|
|
470
479
|
connectionString: "postgresql://user:password@localhost:5432/mydb",
|
|
471
480
|
});
|
|
472
481
|
|
|
473
|
-
// Using a connection string with
|
|
482
|
+
// Using a connection string with pool options
|
|
474
483
|
const store2 = new PostgresStore({
|
|
475
484
|
id: 'pg-storage-2',
|
|
476
485
|
connectionString: "postgresql://user:password@localhost:5432/mydb",
|
|
477
|
-
schemaName: "custom_schema",
|
|
486
|
+
schemaName: "custom_schema",
|
|
487
|
+
max: 30, // Max pool connections
|
|
488
|
+
idleTimeoutMillis: 60000, // Idle timeout
|
|
489
|
+
ssl: { rejectUnauthorized: false },
|
|
478
490
|
});
|
|
479
491
|
|
|
480
492
|
// Using individual connection parameters
|
|
481
|
-
const
|
|
493
|
+
const store3 = new PostgresStore({
|
|
482
494
|
id: 'pg-storage-3',
|
|
483
495
|
host: "localhost",
|
|
484
496
|
port: 5432,
|
|
@@ -487,14 +499,16 @@ const store4 = new PostgresStore({
|
|
|
487
499
|
password: "password",
|
|
488
500
|
});
|
|
489
501
|
|
|
490
|
-
//
|
|
491
|
-
const
|
|
502
|
+
// Using a pre-configured pg.Pool (recommended for pool reuse)
|
|
503
|
+
const existingPool = new Pool({
|
|
504
|
+
connectionString: "postgresql://user:password@localhost:5432/mydb",
|
|
505
|
+
max: 20,
|
|
506
|
+
// ... your custom pool configuration
|
|
507
|
+
});
|
|
508
|
+
|
|
509
|
+
const store4 = new PostgresStore({
|
|
492
510
|
id: 'pg-storage-4',
|
|
493
|
-
|
|
494
|
-
port: 5432,
|
|
495
|
-
database: "mydb",
|
|
496
|
-
user: "user",
|
|
497
|
-
password: "password",
|
|
511
|
+
pool: existingPool,
|
|
498
512
|
schemaName: "custom_schema", // optional
|
|
499
513
|
});
|
|
500
514
|
```
|
|
@@ -513,6 +527,14 @@ The storage implementation handles schema creation and updates automatically. It
|
|
|
513
527
|
- `mastra_scorers`: Stores scoring and evaluation data
|
|
514
528
|
- `mastra_resources`: Stores resource working memory data
|
|
515
529
|
|
|
530
|
+
### Observability
|
|
531
|
+
|
|
532
|
+
PostgreSQL supports observability and can handle low trace volumes. Throughput capacity depends on deployment factors such as hardware, schema design, indexing, and retention policies, and should be validated for your specific environment. For high-volume production environments, consider:
|
|
533
|
+
|
|
534
|
+
- Using the `insert-only` [tracing strategy](https://mastra.ai/docs/observability/tracing/exporters/default#tracing-strategies) to reduce database write operations
|
|
535
|
+
- Setting up table partitioning for efficient data retention
|
|
536
|
+
- Migrating observability to [ClickHouse via composite storage](https://mastra.ai/reference/storage/composite#specialized-storage-for-observability) if you need to scale further
|
|
537
|
+
|
|
516
538
|
### Initialization
|
|
517
539
|
|
|
518
540
|
When you pass storage to the Mastra class, `init()` is called automatically before any storage operation:
|
|
@@ -552,19 +574,74 @@ const thread = await memoryStore?.getThreadById({ threadId: "..." });
|
|
|
552
574
|
> **Note:**
|
|
553
575
|
If `init()` is not called, tables won't be created and storage operations will fail silently or throw errors.
|
|
554
576
|
|
|
577
|
+
### Using an Existing Pool
|
|
578
|
+
|
|
579
|
+
If you already have a `pg.Pool` in your application (e.g., shared with an ORM or for Row Level Security), you can pass it directly to `PostgresStore`:
|
|
580
|
+
|
|
581
|
+
```typescript
|
|
582
|
+
import { Pool } from "pg";
|
|
583
|
+
import { PostgresStore } from "@mastra/pg";
|
|
584
|
+
|
|
585
|
+
// Your existing pool (shared across your application)
|
|
586
|
+
const pool = new Pool({
|
|
587
|
+
connectionString: process.env.DATABASE_URL,
|
|
588
|
+
max: 20,
|
|
589
|
+
});
|
|
590
|
+
|
|
591
|
+
const storage = new PostgresStore({
|
|
592
|
+
id: "shared-storage",
|
|
593
|
+
pool: pool,
|
|
594
|
+
});
|
|
595
|
+
```
|
|
596
|
+
|
|
597
|
+
**Pool lifecycle behavior:**
|
|
598
|
+
|
|
599
|
+
- When you **provide a pool**: Mastra uses your pool but does **not** close it when `store.close()` is called. You manage the pool lifecycle.
|
|
600
|
+
- When Mastra **creates a pool**: Mastra owns the pool and will close it when `store.close()` is called.
|
|
601
|
+
|
|
555
602
|
### Direct Database and Pool Access
|
|
556
603
|
|
|
557
|
-
`PostgresStore` exposes
|
|
604
|
+
`PostgresStore` exposes the underlying database client and pool for advanced use cases:
|
|
605
|
+
|
|
606
|
+
```typescript
|
|
607
|
+
store.db; // DbClient - query interface with helpers (any, one, tx, etc.)
|
|
608
|
+
store.pool; // pg.Pool - the underlying connection pool
|
|
609
|
+
```
|
|
610
|
+
|
|
611
|
+
**Using `store.db` for queries:**
|
|
558
612
|
|
|
559
613
|
```typescript
|
|
560
|
-
|
|
561
|
-
store.
|
|
614
|
+
// Execute queries with helper methods
|
|
615
|
+
const users = await store.db.any("SELECT * FROM users WHERE active = $1", [true]);
|
|
616
|
+
const user = await store.db.one("SELECT * FROM users WHERE id = $1", [userId]);
|
|
617
|
+
const maybeUser = await store.db.oneOrNone("SELECT * FROM users WHERE email = $1", [email]);
|
|
618
|
+
|
|
619
|
+
// Use transactions
|
|
620
|
+
const result = await store.db.tx(async (t) => {
|
|
621
|
+
await t.none("INSERT INTO logs (message) VALUES ($1)", ["Started"]);
|
|
622
|
+
const data = await t.any("SELECT * FROM items");
|
|
623
|
+
return data;
|
|
624
|
+
});
|
|
625
|
+
```
|
|
626
|
+
|
|
627
|
+
**Using `store.pool` directly:**
|
|
628
|
+
|
|
629
|
+
```typescript
|
|
630
|
+
// Get a client for manual connection management
|
|
631
|
+
const client = await store.pool.connect();
|
|
632
|
+
try {
|
|
633
|
+
await client.query("SET LOCAL app.user_id = $1", [userId]);
|
|
634
|
+
const result = await client.query("SELECT * FROM protected_table");
|
|
635
|
+
return result.rows;
|
|
636
|
+
} finally {
|
|
637
|
+
client.release();
|
|
638
|
+
}
|
|
562
639
|
```
|
|
563
640
|
|
|
564
|
-
|
|
641
|
+
When using these fields:
|
|
565
642
|
|
|
566
643
|
- You are responsible for proper connection and transaction handling.
|
|
567
|
-
- Closing the store (`store.close()`) will destroy the
|
|
644
|
+
- Closing the store (`store.close()`) will destroy the pool only if Mastra created it.
|
|
568
645
|
- Direct access bypasses any additional logic or validation provided by PostgresStore methods.
|
|
569
646
|
|
|
570
647
|
This approach is intended for advanced scenarios where low-level access is required.
|
|
@@ -340,8 +340,8 @@ const response = await agent.generate(
|
|
|
340
340
|
|
|
341
341
|
For more information on request context, please see:
|
|
342
342
|
|
|
343
|
-
- [Agent Request Context](https://mastra.ai/docs/
|
|
344
|
-
- [Request Context](https://mastra.ai/docs/
|
|
343
|
+
- [Agent Request Context](https://mastra.ai/docs/server/request-context)
|
|
344
|
+
- [Request Context](https://mastra.ai/docs/server/request-context#accessing-values-with-tools)
|
|
345
345
|
|
|
346
346
|
## Usage Without a Mastra Server
|
|
347
347
|
|
|
@@ -263,8 +263,8 @@ Embeddings are numeric vectors used by memory's `semanticRecall` to retrieve rel
|
|
|
263
263
|
|
|
264
264
|
Install `fastembed` to get started:
|
|
265
265
|
|
|
266
|
-
```bash
|
|
267
|
-
npm install @mastra/fastembed@
|
|
266
|
+
```bash npm2yarn
|
|
267
|
+
npm install @mastra/fastembed@latest
|
|
268
268
|
```
|
|
269
269
|
|
|
270
270
|
Add the following to your agent:
|