@mastra/pg 1.2.0-alpha.0 → 1.3.0-alpha.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +180 -0
- package/dist/docs/SKILL.md +28 -25
- package/dist/docs/{SOURCE_MAP.json → assets/SOURCE_MAP.json} +1 -1
- package/dist/docs/{memory/03-semantic-recall.md → references/docs-memory-semantic-recall.md} +33 -17
- package/dist/docs/{memory/01-storage.md → references/docs-memory-storage.md} +29 -39
- package/dist/docs/{memory/02-working-memory.md → references/docs-memory-working-memory.md} +16 -27
- package/dist/docs/{rag/01-overview.md → references/docs-rag-overview.md} +2 -4
- package/dist/docs/{rag/03-retrieval.md → references/docs-rag-retrieval.md} +26 -53
- package/dist/docs/{rag/02-vector-databases.md → references/docs-rag-vector-databases.md} +198 -202
- package/dist/docs/{memory/04-reference.md → references/reference-memory-memory-class.md} +28 -14
- package/dist/docs/references/reference-processors-message-history-processor.md +85 -0
- package/dist/docs/references/reference-processors-semantic-recall-processor.md +123 -0
- package/dist/docs/references/reference-processors-working-memory-processor.md +154 -0
- package/dist/docs/{rag/04-reference.md → references/reference-rag-metadata-filters.md} +26 -179
- package/dist/docs/references/reference-storage-composite.md +235 -0
- package/dist/docs/references/reference-storage-dynamodb.md +282 -0
- package/dist/docs/references/reference-storage-postgresql.md +529 -0
- package/dist/docs/{tools/01-reference.md → references/reference-tools-vector-query-tool.md} +137 -118
- package/dist/docs/{vectors/01-reference.md → references/reference-vectors-pg.md} +115 -14
- package/dist/index.cjs +1998 -217
- package/dist/index.cjs.map +1 -1
- package/dist/index.js +1998 -219
- package/dist/index.js.map +1 -1
- package/dist/storage/db/constraint-utils.d.ts +16 -0
- package/dist/storage/db/constraint-utils.d.ts.map +1 -0
- package/dist/storage/db/index.d.ts.map +1 -1
- package/dist/storage/domains/agents/index.d.ts +9 -12
- package/dist/storage/domains/agents/index.d.ts.map +1 -1
- package/dist/storage/domains/memory/index.d.ts +7 -1
- package/dist/storage/domains/memory/index.d.ts.map +1 -1
- package/dist/storage/domains/prompt-blocks/index.d.ts +33 -0
- package/dist/storage/domains/prompt-blocks/index.d.ts.map +1 -0
- package/dist/storage/domains/scorer-definitions/index.d.ts +33 -0
- package/dist/storage/domains/scorer-definitions/index.d.ts.map +1 -0
- package/dist/storage/index.d.ts +3 -1
- package/dist/storage/index.d.ts.map +1 -1
- package/package.json +5 -6
- package/dist/docs/README.md +0 -36
- package/dist/docs/processors/01-reference.md +0 -296
- package/dist/docs/storage/01-reference.md +0 -905
|
@@ -1,13 +1,4 @@
|
|
|
1
|
-
#
|
|
2
|
-
|
|
3
|
-
> API reference for tools - 1 entries
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
## Reference: createVectorQueryTool()
|
|
9
|
-
|
|
10
|
-
> Documentation for the Vector Query Tool in Mastra, which facilitates semantic search over vector stores with filtering and reranking capabilities.
|
|
1
|
+
# createVectorQueryTool()
|
|
11
2
|
|
|
12
3
|
The `createVectorQueryTool()` function creates a tool for semantic search over vector stores. It supports filtering, reranking, database-specific configurations, and integrates with various vector store backends.
|
|
13
4
|
|
|
@@ -26,24 +17,56 @@ const queryTool = createVectorQueryTool({
|
|
|
26
17
|
|
|
27
18
|
## Parameters
|
|
28
19
|
|
|
29
|
-
> **Note:**
|
|
20
|
+
> **Note:** **Parameter Requirements:** Most fields can be set at creation as defaults. Some fields can be overridden at runtime via the request context or input. If a required field is missing from both creation and runtime, an error will be thrown. Note that `model`, `id`, and `description` can only be set at creation time.
|
|
21
|
+
|
|
22
|
+
**id?:** (`string`): Custom ID for the tool. By default: 'VectorQuery {vectorStoreName} {indexName} Tool'. (Set at creation only.)
|
|
23
|
+
|
|
24
|
+
**description?:** (`string`): Custom description for the tool. By default: 'Access the knowledge base to find information needed to answer user questions' (Set at creation only.)
|
|
25
|
+
|
|
26
|
+
**model:** (`EmbeddingModel`): Embedding model to use for vector search. (Set at creation only.)
|
|
27
|
+
|
|
28
|
+
**vectorStoreName:** (`string`): Name of the vector store to query. (Can be set at creation or overridden at runtime.)
|
|
30
29
|
|
|
31
|
-
**
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
30
|
+
**indexName:** (`string`): Name of the index within the vector store. (Can be set at creation or overridden at runtime.)
|
|
31
|
+
|
|
32
|
+
**enableFilter?:** (`boolean`): Enable filtering of results based on metadata. (Set at creation only, but will be automatically enabled if a filter is provided in the request context.) (Default: `false`)
|
|
33
|
+
|
|
34
|
+
**includeVectors?:** (`boolean`): Include the embedding vectors in the results. (Can be set at creation or overridden at runtime.) (Default: `false`)
|
|
35
|
+
|
|
36
|
+
**includeSources?:** (`boolean`): Include the full retrieval objects in the results. (Can be set at creation or overridden at runtime.) (Default: `true`)
|
|
37
|
+
|
|
38
|
+
**reranker?:** (`RerankConfig`): Options for reranking results. (Can be set at creation or overridden at runtime.)
|
|
39
|
+
|
|
40
|
+
**databaseConfig?:** (`DatabaseConfig`): Database-specific configuration options for optimizing queries. (Can be set at creation or overridden at runtime.)
|
|
41
|
+
|
|
42
|
+
**providerOptions?:** (`Record<string, Record<string, any>>`): Provider-specific options for the embedding model (e.g., outputDimensionality). \*\*Important\*\*: Only works with AI SDK EmbeddingModelV2 models. For V1 models, configure options when creating the model itself.
|
|
43
|
+
|
|
44
|
+
**vectorStore?:** (`MastraVector | VectorStoreResolver`): Direct vector store instance or a resolver function for dynamic selection. Use a function for multi-tenant applications where the vector store is selected based on request context. When provided, \`vectorStoreName\` becomes optional.
|
|
36
45
|
|
|
37
46
|
### DatabaseConfig
|
|
38
47
|
|
|
39
48
|
The `DatabaseConfig` type allows you to specify database-specific configurations that are automatically applied to query operations. This enables you to take advantage of unique features and optimizations offered by different vector stores.
|
|
40
49
|
|
|
50
|
+
**pinecone?:** (`PineconeConfig`): objectnamespace?:stringPinecone namespace for organizing vectorssparseVector?:{ indices: number\[]; values: number\[]; }Sparse vector for hybrid search
|
|
51
|
+
|
|
52
|
+
**pgvector?:** (`PgVectorConfig`): objectminScore?:numberMinimum similarity score threshold for resultsef?:numberHNSW search parameter - controls accuracy vs speed tradeoffprobes?:numberIVFFlat probe parameter - number of cells to visit during search
|
|
53
|
+
|
|
54
|
+
**chroma?:** (`ChromaConfig`): objectwhere?:Record\<string, any>Metadata filtering conditionswhereDocument?:Record\<string, any>Document content filtering conditions
|
|
55
|
+
|
|
41
56
|
### RerankConfig
|
|
42
57
|
|
|
58
|
+
**model:** (`MastraLanguageModel`): Language model to use for reranking
|
|
59
|
+
|
|
60
|
+
**options?:** (`RerankerOptions`): objectweights?:WeightConfigWeights for scoring components (semantic: 0.4, vector: 0.4, position: 0.2)topK?:numberNumber of top results to return
|
|
61
|
+
|
|
43
62
|
## Returns
|
|
44
63
|
|
|
45
64
|
The tool returns an object with:
|
|
46
65
|
|
|
66
|
+
**relevantContext:** (`string`): Combined text from the most relevant document chunks
|
|
67
|
+
|
|
68
|
+
**sources:** (`QueryResult[]`): Array of full retrieval result objects. Each object contains all information needed to reference the original document, chunk, and similarity score.
|
|
69
|
+
|
|
47
70
|
### QueryResult object structure
|
|
48
71
|
|
|
49
72
|
```typescript
|
|
@@ -95,7 +118,7 @@ This agent-driven approach:
|
|
|
95
118
|
- Implements vector store-specific filter syntax
|
|
96
119
|
- Translates query terms to filter operators
|
|
97
120
|
|
|
98
|
-
For detailed filter syntax and store-specific capabilities, see the [Metadata Filters](
|
|
121
|
+
For detailed filter syntax and store-specific capabilities, see the [Metadata Filters](https://mastra.ai/reference/rag/metadata-filters) documentation.
|
|
99
122
|
|
|
100
123
|
For an example of how agent-driven filtering works, see the [Agent-Driven Metadata Filtering](https://github.com/mastra-ai/mastra/tree/main/examples/basics/rag/filter-rag) example.
|
|
101
124
|
|
|
@@ -147,122 +170,118 @@ This example shows how to customize the tool description for a specific use case
|
|
|
147
170
|
|
|
148
171
|
The `databaseConfig` parameter allows you to leverage unique features and optimizations specific to each vector database. These configurations are automatically applied during query execution.
|
|
149
172
|
|
|
150
|
-
|
|
173
|
+
**Pinecone**:
|
|
151
174
|
|
|
152
|
-
|
|
175
|
+
### Pinecone Configuration
|
|
153
176
|
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
}
|
|
166
|
-
}
|
|
177
|
+
```typescript
|
|
178
|
+
const pineconeQueryTool = createVectorQueryTool({
|
|
179
|
+
vectorStoreName: "pinecone",
|
|
180
|
+
indexName: "docs",
|
|
181
|
+
model: new ModelRouterEmbeddingModel("openai/text-embedding-3-small"),
|
|
182
|
+
databaseConfig: {
|
|
183
|
+
pinecone: {
|
|
184
|
+
namespace: "production", // Organize vectors by environment
|
|
185
|
+
sparseVector: { // Enable hybrid search
|
|
186
|
+
indices: [0, 1, 2, 3],
|
|
187
|
+
values: [0.1, 0.2, 0.15, 0.05]
|
|
167
188
|
}
|
|
168
|
-
}
|
|
169
|
-
|
|
189
|
+
}
|
|
190
|
+
}
|
|
191
|
+
});
|
|
192
|
+
```
|
|
170
193
|
|
|
171
|
-
|
|
172
|
-
- **Namespace**: Isolate different data sets within the same index
|
|
173
|
-
- **Sparse Vector**: Combine dense and sparse embeddings for improved search quality
|
|
174
|
-
- **Use Cases**: Multi-tenant applications, hybrid semantic search
|
|
194
|
+
**Pinecone Features:**
|
|
175
195
|
|
|
176
|
-
|
|
196
|
+
- **Namespace**: Isolate different data sets within the same index
|
|
197
|
+
- **Sparse Vector**: Combine dense and sparse embeddings for improved search quality
|
|
198
|
+
- **Use Cases**: Multi-tenant applications, hybrid semantic search
|
|
177
199
|
|
|
178
|
-
|
|
200
|
+
**pgVector**:
|
|
179
201
|
|
|
180
|
-
|
|
202
|
+
### pgVector Configuration
|
|
181
203
|
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
204
|
+
```typescript
|
|
205
|
+
const pgVectorQueryTool = createVectorQueryTool({
|
|
206
|
+
vectorStoreName: "postgres",
|
|
207
|
+
indexName: "embeddings",
|
|
208
|
+
model: new ModelRouterEmbeddingModel("openai/text-embedding-3-small"),
|
|
209
|
+
databaseConfig: {
|
|
210
|
+
pgvector: {
|
|
211
|
+
minScore: 0.7, // Only return results above 70% similarity
|
|
212
|
+
ef: 200, // Higher value = better accuracy, slower search
|
|
213
|
+
probes: 10 // For IVFFlat: more probes = better recall
|
|
214
|
+
}
|
|
215
|
+
}
|
|
216
|
+
});
|
|
217
|
+
```
|
|
196
218
|
|
|
197
|
-
|
|
198
|
-
- **minScore**: Filter out low-quality matches
|
|
199
|
-
- **ef (HNSW)**: Control accuracy vs speed for HNSW indexes
|
|
200
|
-
- **probes (IVFFlat)**: Control recall vs speed for IVFFlat indexes
|
|
201
|
-
- **Use Cases**: Performance tuning, quality filtering
|
|
219
|
+
**pgVector Features:**
|
|
202
220
|
|
|
203
|
-
|
|
221
|
+
- **minScore**: Filter out low-quality matches
|
|
222
|
+
- **ef (HNSW)**: Control accuracy vs speed for HNSW indexes
|
|
223
|
+
- **probes (IVFFlat)**: Control recall vs speed for IVFFlat indexes
|
|
224
|
+
- **Use Cases**: Performance tuning, quality filtering
|
|
204
225
|
|
|
205
|
-
|
|
226
|
+
**Chroma**:
|
|
206
227
|
|
|
207
|
-
|
|
228
|
+
### Chroma Configuration
|
|
208
229
|
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
}
|
|
223
|
-
}
|
|
230
|
+
```typescript
|
|
231
|
+
const chromaQueryTool = createVectorQueryTool({
|
|
232
|
+
vectorStoreName: "chroma",
|
|
233
|
+
indexName: "documents",
|
|
234
|
+
model: new ModelRouterEmbeddingModel("openai/text-embedding-3-small"),
|
|
235
|
+
databaseConfig: {
|
|
236
|
+
chroma: {
|
|
237
|
+
where: { // Metadata filtering
|
|
238
|
+
"category": "technical",
|
|
239
|
+
"status": "published"
|
|
240
|
+
},
|
|
241
|
+
whereDocument: { // Document content filtering
|
|
242
|
+
"$contains": "API"
|
|
224
243
|
}
|
|
225
|
-
}
|
|
226
|
-
|
|
244
|
+
}
|
|
245
|
+
}
|
|
246
|
+
});
|
|
247
|
+
```
|
|
227
248
|
|
|
228
|
-
|
|
229
|
-
- **where**: Filter by metadata fields
|
|
230
|
-
- **whereDocument**: Filter by document content
|
|
231
|
-
- **Use Cases**: Advanced filtering, content-based search
|
|
249
|
+
**Chroma Features:**
|
|
232
250
|
|
|
233
|
-
|
|
251
|
+
- **where**: Filter by metadata fields
|
|
252
|
+
- **whereDocument**: Filter by document content
|
|
253
|
+
- **Use Cases**: Advanced filtering, content-based search
|
|
234
254
|
|
|
235
|
-
|
|
255
|
+
**Multiple Configs**:
|
|
236
256
|
|
|
237
|
-
|
|
257
|
+
### Multiple Database Configurations
|
|
238
258
|
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
+
```typescript
|
|
260
|
+
// Configure for multiple databases (useful for dynamic stores)
|
|
261
|
+
const multiDbQueryTool = createVectorQueryTool({
|
|
262
|
+
vectorStoreName: "dynamic-store", // Will be set at runtime
|
|
263
|
+
indexName: "docs",
|
|
264
|
+
model: new ModelRouterEmbeddingModel("openai/text-embedding-3-small"),
|
|
265
|
+
databaseConfig: {
|
|
266
|
+
pinecone: {
|
|
267
|
+
namespace: "default"
|
|
268
|
+
},
|
|
269
|
+
pgvector: {
|
|
270
|
+
minScore: 0.8,
|
|
271
|
+
ef: 150
|
|
272
|
+
},
|
|
273
|
+
chroma: {
|
|
274
|
+
where: { "type": "documentation" }
|
|
275
|
+
}
|
|
276
|
+
}
|
|
277
|
+
});
|
|
278
|
+
```
|
|
259
279
|
|
|
260
|
-
|
|
261
|
-
- Support multiple vector stores with one tool
|
|
262
|
-
- Database-specific optimizations are automatically applied
|
|
263
|
-
- Flexible deployment scenarios
|
|
280
|
+
**Multi-Config Benefits:**
|
|
264
281
|
|
|
265
|
-
|
|
282
|
+
- Support multiple vector stores with one tool
|
|
283
|
+
- Database-specific optimizations are automatically applied
|
|
284
|
+
- Flexible deployment scenarios
|
|
266
285
|
|
|
267
286
|
### Runtime Configuration Override
|
|
268
287
|
|
|
@@ -341,13 +360,13 @@ const response = await agent.generate(
|
|
|
341
360
|
For more information on request context, please see:
|
|
342
361
|
|
|
343
362
|
- [Agent Request Context](https://mastra.ai/docs/server/request-context)
|
|
344
|
-
- [Request Context](https://mastra.ai/docs/server/request-context
|
|
363
|
+
- [Request Context](https://mastra.ai/docs/server/request-context)
|
|
345
364
|
|
|
346
365
|
## Usage Without a Mastra Server
|
|
347
366
|
|
|
348
367
|
The tool can be used by itself to retrieve documents matching a query:
|
|
349
368
|
|
|
350
|
-
```typescript
|
|
369
|
+
```typescript
|
|
351
370
|
import { RequestContext } from "@mastra/core/request-context";
|
|
352
371
|
import { createVectorQueryTool } from "@mastra/rag";
|
|
353
372
|
import { PgVector } from "@mastra/pg";
|
|
@@ -377,7 +396,7 @@ console.log(queryResult.sources);
|
|
|
377
396
|
|
|
378
397
|
For multi-tenant applications where each tenant has isolated data (e.g., separate PostgreSQL schemas), you can pass a resolver function instead of a static vector store instance. The function receives the request context and can return the appropriate vector store for the current tenant:
|
|
379
398
|
|
|
380
|
-
```typescript
|
|
399
|
+
```typescript
|
|
381
400
|
import { createVectorQueryTool, VectorStoreResolver } from "@mastra/rag";
|
|
382
401
|
import { PgVector } from "@mastra/pg";
|
|
383
402
|
|
|
@@ -387,7 +406,7 @@ const vectorStoreCache = new Map<string, PgVector>();
|
|
|
387
406
|
// Resolver function that returns the correct vector store based on tenant
|
|
388
407
|
const vectorStoreResolver: VectorStoreResolver = async ({ requestContext }) => {
|
|
389
408
|
const tenantId = requestContext?.get("tenantId");
|
|
390
|
-
|
|
409
|
+
|
|
391
410
|
if (!tenantId) {
|
|
392
411
|
throw new Error("tenantId is required in request context");
|
|
393
412
|
}
|
|
@@ -436,5 +455,5 @@ The tool is created with:
|
|
|
436
455
|
|
|
437
456
|
## Related
|
|
438
457
|
|
|
439
|
-
- [rerank()](
|
|
440
|
-
- [createGraphRAGTool](
|
|
458
|
+
- [rerank()](https://mastra.ai/reference/rag/rerank)
|
|
459
|
+
- [createGraphRAGTool](https://mastra.ai/reference/tools/graph-rag-tool)
|
|
@@ -1,18 +1,30 @@
|
|
|
1
|
-
#
|
|
1
|
+
# PG Vector Store
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
The PgVector class provides vector search using [PostgreSQL](https://www.postgresql.org/) with [pgvector](https://github.com/pgvector/pgvector) extension. It provides robust vector similarity search capabilities within your existing PostgreSQL database.
|
|
4
4
|
|
|
5
|
+
## Constructor Options
|
|
5
6
|
|
|
6
|
-
|
|
7
|
+
**connectionString?:** (`string`): PostgreSQL connection URL
|
|
7
8
|
|
|
8
|
-
|
|
9
|
+
**host?:** (`string`): PostgreSQL server host
|
|
9
10
|
|
|
10
|
-
|
|
11
|
+
**port?:** (`number`): PostgreSQL server port
|
|
11
12
|
|
|
12
|
-
|
|
13
|
-
It provides robust vector similarity search capabilities within your existing PostgreSQL database.
|
|
13
|
+
**database?:** (`string`): PostgreSQL database name
|
|
14
14
|
|
|
15
|
-
|
|
15
|
+
**user?:** (`string`): PostgreSQL user
|
|
16
|
+
|
|
17
|
+
**password?:** (`string`): PostgreSQL password
|
|
18
|
+
|
|
19
|
+
**ssl?:** (`boolean | ConnectionOptions`): Enable SSL or provide custom SSL configuration
|
|
20
|
+
|
|
21
|
+
**schemaName?:** (`string`): The name of the schema you want the vector store to use. Will use the default schema if not provided.
|
|
22
|
+
|
|
23
|
+
**max?:** (`number`): Maximum number of pool connections (default: 20)
|
|
24
|
+
|
|
25
|
+
**idleTimeoutMillis?:** (`number`): Idle connection timeout in milliseconds (default: 30000)
|
|
26
|
+
|
|
27
|
+
**pgPoolOptions?:** (`PoolConfig`): Additional pg pool configuration options
|
|
16
28
|
|
|
17
29
|
## Constructor Examples
|
|
18
30
|
|
|
@@ -60,28 +72,68 @@ const vectorStore = new PgVector({
|
|
|
60
72
|
|
|
61
73
|
### createIndex()
|
|
62
74
|
|
|
75
|
+
**indexName:** (`string`): Name of the index to create
|
|
76
|
+
|
|
77
|
+
**dimension:** (`number`): Vector dimension (must match your embedding model)
|
|
78
|
+
|
|
79
|
+
**metric?:** (`'cosine' | 'euclidean' | 'dotproduct'`): Distance metric for similarity search (Default: `cosine`)
|
|
80
|
+
|
|
81
|
+
**indexConfig?:** (`IndexConfig`): Index configuration (Default: `{ type: 'ivfflat' }`)
|
|
82
|
+
|
|
83
|
+
**buildIndex?:** (`boolean`): Whether to build the index (Default: `true`)
|
|
84
|
+
|
|
63
85
|
#### IndexConfig
|
|
64
86
|
|
|
87
|
+
**type:** (`'flat' | 'hnsw' | 'ivfflat'`): stringflat:flatSequential scan (no index) that performs exhaustive search.ivfflat:ivfflatClusters vectors into lists for approximate search.hnsw:hnswGraph-based index offering fast search times and high recall. (Default: `ivfflat`)
|
|
88
|
+
|
|
89
|
+
**ivf?:** (`IVFConfig`): objectlists?:numberNumber of lists. If not specified, automatically calculated based on dataset size. (Minimum 100, Maximum 4000)
|
|
90
|
+
|
|
91
|
+
**hnsw?:** (`HNSWConfig`): objectm?:numberMaximum number of connections per node (default: 8)efConstruction?:numberBuild-time complexity (default: 32)
|
|
92
|
+
|
|
65
93
|
#### Memory Requirements
|
|
66
94
|
|
|
67
95
|
HNSW indexes require significant shared memory during construction. For 100K vectors:
|
|
68
96
|
|
|
69
|
-
- Small dimensions (64d):
|
|
70
|
-
- Medium dimensions (256d):
|
|
71
|
-
- Large dimensions (384d+):
|
|
97
|
+
- Small dimensions (64d): \~60MB with default settings
|
|
98
|
+
- Medium dimensions (256d): \~180MB with default settings
|
|
99
|
+
- Large dimensions (384d+): \~250MB+ with default settings
|
|
72
100
|
|
|
73
101
|
Higher M values or efConstruction values will increase memory requirements significantly. Adjust your system's shared memory limits if needed.
|
|
74
102
|
|
|
75
103
|
### upsert()
|
|
76
104
|
|
|
105
|
+
**indexName:** (`string`): Name of the index to upsert vectors into
|
|
106
|
+
|
|
107
|
+
**vectors:** (`number[][]`): Array of embedding vectors
|
|
108
|
+
|
|
109
|
+
**metadata?:** (`Record<string, any>[]`): Metadata for each vector
|
|
110
|
+
|
|
111
|
+
**ids?:** (`string[]`): Optional vector IDs (auto-generated if not provided)
|
|
112
|
+
|
|
77
113
|
### query()
|
|
78
114
|
|
|
115
|
+
**indexName:** (`string`): Name of the index to query
|
|
116
|
+
|
|
117
|
+
**vector:** (`number[]`): Query vector
|
|
118
|
+
|
|
119
|
+
**topK?:** (`number`): Number of results to return (Default: `10`)
|
|
120
|
+
|
|
121
|
+
**filter?:** (`Record<string, any>`): Metadata filters
|
|
122
|
+
|
|
123
|
+
**includeVector?:** (`boolean`): Whether to include the vector in the result (Default: `false`)
|
|
124
|
+
|
|
125
|
+
**minScore?:** (`number`): Minimum similarity score threshold (Default: `0`)
|
|
126
|
+
|
|
127
|
+
**options?:** (`{ ef?: number; probes?: number }`): objectef?:numberHNSW search parameterprobes?:numberIVF search parameter
|
|
128
|
+
|
|
79
129
|
### listIndexes()
|
|
80
130
|
|
|
81
131
|
Returns an array of index names as strings.
|
|
82
132
|
|
|
83
133
|
### describeIndex()
|
|
84
134
|
|
|
135
|
+
**indexName:** (`string`): Name of the index to describe
|
|
136
|
+
|
|
85
137
|
Returns:
|
|
86
138
|
|
|
87
139
|
```typescript
|
|
@@ -101,10 +153,20 @@ interface PGIndexStats {
|
|
|
101
153
|
|
|
102
154
|
### deleteIndex()
|
|
103
155
|
|
|
156
|
+
**indexName:** (`string`): Name of the index to delete
|
|
157
|
+
|
|
104
158
|
### updateVector()
|
|
105
159
|
|
|
106
160
|
Update a single vector by ID or by metadata filter. Either `id` or `filter` must be provided, but not both.
|
|
107
161
|
|
|
162
|
+
**indexName:** (`string`): Name of the index containing the vector
|
|
163
|
+
|
|
164
|
+
**id?:** (`string`): ID of the vector to update (mutually exclusive with filter)
|
|
165
|
+
|
|
166
|
+
**filter?:** (`Record<string, any>`): Metadata filter to identify vector(s) to update (mutually exclusive with id)
|
|
167
|
+
|
|
168
|
+
**update:** (`{ vector?: number[]; metadata?: Record<string, any>; }`): Object containing the vector and/or metadata to update
|
|
169
|
+
|
|
108
170
|
Updates an existing vector by ID or filter. At least one of vector or metadata must be provided in the update object.
|
|
109
171
|
|
|
110
172
|
```typescript
|
|
@@ -130,6 +192,10 @@ await pgVector.updateVector({
|
|
|
130
192
|
|
|
131
193
|
### deleteVector()
|
|
132
194
|
|
|
195
|
+
**indexName:** (`string`): Name of the index containing the vector
|
|
196
|
+
|
|
197
|
+
**id:** (`string`): ID of the vector to delete
|
|
198
|
+
|
|
133
199
|
Deletes a single vector by ID from the specified index.
|
|
134
200
|
|
|
135
201
|
```typescript
|
|
@@ -140,12 +206,24 @@ await pgVector.deleteVector({ indexName: "my_vectors", id: "vector123" });
|
|
|
140
206
|
|
|
141
207
|
Delete multiple vectors by IDs or by metadata filter. Either `ids` or `filter` must be provided, but not both.
|
|
142
208
|
|
|
209
|
+
**indexName:** (`string`): Name of the index containing the vectors to delete
|
|
210
|
+
|
|
211
|
+
**ids?:** (`string[]`): Array of vector IDs to delete (mutually exclusive with filter)
|
|
212
|
+
|
|
213
|
+
**filter?:** (`Record<string, any>`): Metadata filter to identify vectors to delete (mutually exclusive with ids)
|
|
214
|
+
|
|
143
215
|
### disconnect()
|
|
144
216
|
|
|
145
217
|
Closes the database connection pool. Should be called when done using the store.
|
|
146
218
|
|
|
147
219
|
### buildIndex()
|
|
148
220
|
|
|
221
|
+
**indexName:** (`string`): Name of the index to define
|
|
222
|
+
|
|
223
|
+
**metric?:** (`'cosine' | 'euclidean' | 'dotproduct'`): Distance metric for similarity search (Default: `cosine`)
|
|
224
|
+
|
|
225
|
+
**indexConfig:** (`IndexConfig`): Configuration for the index type and parameters
|
|
226
|
+
|
|
149
227
|
Builds or rebuilds an index with specified metric and configuration. Will drop any existing index before creating the new one.
|
|
150
228
|
|
|
151
229
|
```typescript
|
|
@@ -216,10 +294,13 @@ try {
|
|
|
216
294
|
#### HNSW Tuning
|
|
217
295
|
|
|
218
296
|
- **m parameter**:
|
|
297
|
+
|
|
219
298
|
- 8-16: Moderate accuracy, lower memory
|
|
220
299
|
- 16-32: High accuracy, moderate memory
|
|
221
300
|
- 32-64: Very high accuracy, high memory
|
|
301
|
+
|
|
222
302
|
- **efConstruction**:
|
|
303
|
+
|
|
223
304
|
- 32-64: Fast build, good quality
|
|
224
305
|
- 64-128: Slower build, better quality
|
|
225
306
|
- 128-256: Slowest build, best quality
|
|
@@ -263,13 +344,33 @@ Embeddings are numeric vectors used by memory's `semanticRecall` to retrieve rel
|
|
|
263
344
|
|
|
264
345
|
Install `fastembed` to get started:
|
|
265
346
|
|
|
266
|
-
|
|
347
|
+
**npm**:
|
|
348
|
+
|
|
349
|
+
```bash
|
|
267
350
|
npm install @mastra/fastembed@latest
|
|
268
351
|
```
|
|
269
352
|
|
|
353
|
+
**pnpm**:
|
|
354
|
+
|
|
355
|
+
```bash
|
|
356
|
+
pnpm add @mastra/fastembed@latest
|
|
357
|
+
```
|
|
358
|
+
|
|
359
|
+
**Yarn**:
|
|
360
|
+
|
|
361
|
+
```bash
|
|
362
|
+
yarn add @mastra/fastembed@latest
|
|
363
|
+
```
|
|
364
|
+
|
|
365
|
+
**Bun**:
|
|
366
|
+
|
|
367
|
+
```bash
|
|
368
|
+
bun add @mastra/fastembed@latest
|
|
369
|
+
```
|
|
370
|
+
|
|
270
371
|
Add the following to your agent:
|
|
271
372
|
|
|
272
|
-
```typescript
|
|
373
|
+
```typescript
|
|
273
374
|
import { Memory } from "@mastra/memory";
|
|
274
375
|
import { Agent } from "@mastra/core/agent";
|
|
275
376
|
import { PostgresStore, PgVector } from "@mastra/pg";
|
|
@@ -304,4 +405,4 @@ export const pgAgent = new Agent({
|
|
|
304
405
|
|
|
305
406
|
## Related
|
|
306
407
|
|
|
307
|
-
- [Metadata Filters](
|
|
408
|
+
- [Metadata Filters](https://mastra.ai/reference/rag/metadata-filters)
|