@mastra/s3vectors 1.0.2-alpha.0 → 1.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +9 -0
- package/dist/docs/SKILL.md +4 -4
- package/dist/docs/assets/SOURCE_MAP.json +1 -1
- package/dist/docs/references/docs-rag-retrieval.md +4 -4
- package/dist/docs/references/docs-rag-vector-databases.md +11 -11
- package/dist/docs/references/reference-vectors-s3vectors.md +17 -17
- package/package.json +5 -5
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,14 @@
|
|
|
1
1
|
# @mastra/s3vectors
|
|
2
2
|
|
|
3
|
+
## 1.0.2
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- dependencies updates: ([#14061](https://github.com/mastra-ai/mastra/pull/14061))
|
|
8
|
+
- Updated dependency [`@aws-sdk/client-s3vectors@^3.1004.0` ↗︎](https://www.npmjs.com/package/@aws-sdk/client-s3vectors/v/3.1004.0) (from `^3.901.0`, in `dependencies`)
|
|
9
|
+
- Updated dependencies [[`4f71b43`](https://github.com/mastra-ai/mastra/commit/4f71b436a4a6b8839842d8da47b57b84509af56c), [`a070277`](https://github.com/mastra-ai/mastra/commit/a07027766ce195ba74d0783116d894cbab25d44c), [`b628b91`](https://github.com/mastra-ai/mastra/commit/b628b9128b372c0f54214d902b07279f03443900), [`332c014`](https://github.com/mastra-ai/mastra/commit/332c014e076b81edf7fe45b58205882726415e90), [`6b63153`](https://github.com/mastra-ai/mastra/commit/6b63153878ea841c0f4ce632ba66bb33e57e9c1b), [`4246e34`](https://github.com/mastra-ai/mastra/commit/4246e34cec9c26636d0965942268e6d07c346671), [`b8837ee`](https://github.com/mastra-ai/mastra/commit/b8837ee77e2e84197609762bfabd8b3da326d30c), [`866cc2c`](https://github.com/mastra-ai/mastra/commit/866cc2cb1f0e3b314afab5194f69477fada745d1), [`5d950f7`](https://github.com/mastra-ai/mastra/commit/5d950f7bf426a215a1808f0abef7de5c8336ba1c), [`28c85b1`](https://github.com/mastra-ai/mastra/commit/28c85b184fc32b40f7f160483c982da6d388ecbd), [`e9a08fb`](https://github.com/mastra-ai/mastra/commit/e9a08fbef1ada7e50e961e2f54f55e8c10b4a45c), [`1d0a8a8`](https://github.com/mastra-ai/mastra/commit/1d0a8a8acf33203d5744fc429b090ad8598aa8ed), [`631ffd8`](https://github.com/mastra-ai/mastra/commit/631ffd82fed108648b448b28e6a90e38c5f53bf5), [`6bcbf8a`](https://github.com/mastra-ai/mastra/commit/6bcbf8a6774d5a53b21d61db8a45ce2593ca1616), [`aae2295`](https://github.com/mastra-ai/mastra/commit/aae2295838a2d329ad6640829e87934790ffe5b8), [`aa61f29`](https://github.com/mastra-ai/mastra/commit/aa61f29ff8095ce46a4ae16e46c4d8c79b2b685b), [`7ff3714`](https://github.com/mastra-ai/mastra/commit/7ff37148515439bb3be009a60e02c3e363299760), [`18c3a90`](https://github.com/mastra-ai/mastra/commit/18c3a90c9e48cf69500e308affeb8eba5860b2af), [`41d79a1`](https://github.com/mastra-ai/mastra/commit/41d79a14bd8cb6de1e2565fd0a04786bae2f211b), [`f35487b`](https://github.com/mastra-ai/mastra/commit/f35487bb2d46c636e22aa71d90025613ae38235a), [`6dc2192`](https://github.com/mastra-ai/mastra/commit/6dc21921aef0f0efab15cd0805fa3d18f277a76f), [`eeb3a3f`](https://github.com/mastra-ai/mastra/commit/eeb3a3f43aca10cf49479eed2a84b7d9ecea02ba), [`e673376`](https://github.com/mastra-ai/mastra/commit/e6733763ad1321aa7e5ae15096b9c2104f93b1f3), [`05f8d90`](https://github.com/mastra-ai/mastra/commit/05f8d9009290ce6aa03428b3add635268615db85), [`b2204c9`](https://github.com/mastra-ai/mastra/commit/b2204c98a42848bbfb6f0440f005dc2b6354f1cd), [`a1bf1e3`](https://github.com/mastra-ai/mastra/commit/a1bf1e385ed4c0ef6f11b56c5887442970d127f2), [`b6f647a`](https://github.com/mastra-ai/mastra/commit/b6f647ae2388e091f366581595feb957e37d5b40), [`0c57b8b`](https://github.com/mastra-ai/mastra/commit/0c57b8b0a69a97b5a4ae3f79be6c610f29f3cf7b), [`b081f27`](https://github.com/mastra-ai/mastra/commit/b081f272cf411716e1d6bd72ceac4bcee2657b19), [`4b8da97`](https://github.com/mastra-ai/mastra/commit/4b8da97a5ce306e97869df6c39535d9069e563db), [`0c09eac`](https://github.com/mastra-ai/mastra/commit/0c09eacb1926f64cfdc9ae5c6d63385cf8c9f72c), [`6b9b93d`](https://github.com/mastra-ai/mastra/commit/6b9b93d6f459d1ba6e36f163abf62a085ddb3d64), [`31b6067`](https://github.com/mastra-ai/mastra/commit/31b6067d0cc3ab10e1b29c36147f3b5266bc714a), [`797ac42`](https://github.com/mastra-ai/mastra/commit/797ac4276de231ad2d694d9aeca75980f6cd0419), [`0bc289e`](https://github.com/mastra-ai/mastra/commit/0bc289e2d476bf46c5b91c21969e8d0c6864691c), [`9b75a06`](https://github.com/mastra-ai/mastra/commit/9b75a06e53ebb0b950ba7c1e83a0142047185f46), [`4c3a1b1`](https://github.com/mastra-ai/mastra/commit/4c3a1b122ea083e003d71092f30f3b31680b01c0), [`256df35`](https://github.com/mastra-ai/mastra/commit/256df3571d62beb3ad4971faa432927cc140e603), [`85cc3b3`](https://github.com/mastra-ai/mastra/commit/85cc3b3b6f32ae4b083c26498f50d5b250ba944b), [`97ea28c`](https://github.com/mastra-ai/mastra/commit/97ea28c746e9e4147d56047bbb1c4a92417a3fec), [`d567299`](https://github.com/mastra-ai/mastra/commit/d567299cf81e02bd9d5221d4bc05967d6c224161), [`716ffe6`](https://github.com/mastra-ai/mastra/commit/716ffe68bed81f7c2690bc8581b9e140f7bf1c3d), [`8296332`](https://github.com/mastra-ai/mastra/commit/8296332de21c16e3dfc3d0b2d615720a6dc88f2f), [`4df2116`](https://github.com/mastra-ai/mastra/commit/4df211619dd922c047d396ca41cd7027c8c4c8e7), [`2219c1a`](https://github.com/mastra-ai/mastra/commit/2219c1acbd21da116da877f0036ffb985a9dd5a3), [`17c4145`](https://github.com/mastra-ai/mastra/commit/17c4145166099354545582335b5252bdfdfd908b)]:
|
|
10
|
+
- @mastra/core@1.11.0
|
|
11
|
+
|
|
3
12
|
## 1.0.2-alpha.0
|
|
4
13
|
|
|
5
14
|
### Patch Changes
|
package/dist/docs/SKILL.md
CHANGED
|
@@ -3,7 +3,7 @@ name: mastra-s3vectors
|
|
|
3
3
|
description: Documentation for @mastra/s3vectors. Use when working with @mastra/s3vectors APIs, configuration, or implementation.
|
|
4
4
|
metadata:
|
|
5
5
|
package: "@mastra/s3vectors"
|
|
6
|
-
version: "1.0.2
|
|
6
|
+
version: "1.0.2"
|
|
7
7
|
---
|
|
8
8
|
|
|
9
9
|
## When to use
|
|
@@ -16,12 +16,12 @@ Read the individual reference documents for detailed explanations and code examp
|
|
|
16
16
|
|
|
17
17
|
### Docs
|
|
18
18
|
|
|
19
|
-
- [Retrieval,
|
|
20
|
-
- [Storing
|
|
19
|
+
- [Retrieval, semantic search, reranking](references/docs-rag-retrieval.md) - Guide on retrieval processes in Mastra's RAG systems, including semantic search, filtering, and re-ranking.
|
|
20
|
+
- [Storing embeddings in a vector database](references/docs-rag-vector-databases.md) - Guide on vector storage options in Mastra, including embedded and dedicated vector databases for similarity search.
|
|
21
21
|
|
|
22
22
|
### Reference
|
|
23
23
|
|
|
24
|
-
- [Reference: Amazon S3
|
|
24
|
+
- [Reference: Amazon S3 vectors store](references/reference-vectors-s3vectors.md) - Documentation for the S3Vectors class in Mastra, which provides vector search using Amazon S3 Vectors (Preview).
|
|
25
25
|
|
|
26
26
|
|
|
27
27
|
Read [assets/SOURCE_MAP.json](assets/SOURCE_MAP.json) for source code references.
|
|
@@ -1,10 +1,10 @@
|
|
|
1
|
-
# Retrieval in RAG
|
|
1
|
+
# Retrieval in RAG systems
|
|
2
2
|
|
|
3
3
|
After storing embeddings, you need to retrieve relevant chunks to answer user queries.
|
|
4
4
|
|
|
5
5
|
Mastra provides flexible retrieval options with support for semantic search, filtering, and re-ranking.
|
|
6
6
|
|
|
7
|
-
## How
|
|
7
|
+
## How retrieval works
|
|
8
8
|
|
|
9
9
|
1. The user's query is converted to an embedding using the same model used for document embeddings
|
|
10
10
|
2. This embedding is compared to stored embeddings using vector similarity
|
|
@@ -14,7 +14,7 @@ Mastra provides flexible retrieval options with support for semantic search, fil
|
|
|
14
14
|
- Re-ranked for better relevance
|
|
15
15
|
- Processed through a knowledge graph
|
|
16
16
|
|
|
17
|
-
## Basic
|
|
17
|
+
## Basic retrieval
|
|
18
18
|
|
|
19
19
|
The simplest approach is direct semantic search. This method uses vector similarity to find chunks that are semantically similar to the query:
|
|
20
20
|
|
|
@@ -63,7 +63,7 @@ Results include both the text content and a similarity score:
|
|
|
63
63
|
]
|
|
64
64
|
```
|
|
65
65
|
|
|
66
|
-
## Advanced
|
|
66
|
+
## Advanced retrieval options
|
|
67
67
|
|
|
68
68
|
### Metadata Filtering
|
|
69
69
|
|
|
@@ -1,8 +1,8 @@
|
|
|
1
|
-
# Storing
|
|
1
|
+
# Storing embeddings in a vector database
|
|
2
2
|
|
|
3
3
|
After generating embeddings, you need to store them in a database that supports vector similarity search. Mastra provides a consistent interface for storing and querying embeddings across various vector databases.
|
|
4
4
|
|
|
5
|
-
## Supported
|
|
5
|
+
## Supported databases
|
|
6
6
|
|
|
7
7
|
**MongoDB**:
|
|
8
8
|
|
|
@@ -234,7 +234,7 @@ await store.upsert({
|
|
|
234
234
|
})
|
|
235
235
|
```
|
|
236
236
|
|
|
237
|
-
**
|
|
237
|
+
**Elasticsearch**:
|
|
238
238
|
|
|
239
239
|
```ts
|
|
240
240
|
import { ElasticSearchVector } from '@mastra/elasticsearch'
|
|
@@ -337,7 +337,7 @@ await store.upsert({
|
|
|
337
337
|
})
|
|
338
338
|
```
|
|
339
339
|
|
|
340
|
-
## Using
|
|
340
|
+
## Using vector storage
|
|
341
341
|
|
|
342
342
|
Once initialized, all vector stores share the same interface for creating indexes, upserting embeddings, and querying.
|
|
343
343
|
|
|
@@ -355,9 +355,9 @@ await store.createIndex({
|
|
|
355
355
|
|
|
356
356
|
The dimension size must match the output dimension of your chosen embedding model. Common dimension sizes are:
|
|
357
357
|
|
|
358
|
-
- OpenAI text-embedding-3-small
|
|
359
|
-
- Cohere embed-multilingual-v3
|
|
360
|
-
- Google gemini-embedding-001
|
|
358
|
+
- `OpenAI text-embedding-3-small`: 1536 dimensions (or custom, e.g., 256)
|
|
359
|
+
- `Cohere embed-multilingual-v3`: 1024 dimensions
|
|
360
|
+
- `Google gemini-embedding-001`: 768 dimensions (or custom)
|
|
361
361
|
|
|
362
362
|
> **Warning:** Index dimensions can't be changed after creation. To use a different model, delete and recreate the index with the new dimension size.
|
|
363
363
|
|
|
@@ -490,7 +490,7 @@ Index names must:
|
|
|
490
490
|
- Example: `My_Index` is not valid (contains uppercase letters)
|
|
491
491
|
- Example: `_myindex` is not valid (begins with underscore)
|
|
492
492
|
|
|
493
|
-
**
|
|
493
|
+
**Elasticsearch**:
|
|
494
494
|
|
|
495
495
|
Index names must:
|
|
496
496
|
|
|
@@ -543,7 +543,7 @@ The upsert operation:
|
|
|
543
543
|
- Creates new vectors if they don't exist
|
|
544
544
|
- Automatically handles batching for large datasets
|
|
545
545
|
|
|
546
|
-
## Adding
|
|
546
|
+
## Adding metadata
|
|
547
547
|
|
|
548
548
|
Vector stores support rich metadata (any JSON-serializable fields) for filtering and organization. Since metadata is stored with no fixed schema, use consistent field naming to avoid unexpected query results.
|
|
549
549
|
|
|
@@ -581,7 +581,7 @@ Key metadata considerations:
|
|
|
581
581
|
- Only include fields you plan to filter or sort by - extra fields add overhead
|
|
582
582
|
- Add timestamps (e.g., 'createdAt', 'lastUpdated') to track content freshness
|
|
583
583
|
|
|
584
|
-
## Deleting
|
|
584
|
+
## Deleting vectors
|
|
585
585
|
|
|
586
586
|
When building RAG applications, you often need to clean up stale vectors when documents are deleted or updated. Mastra provides the `deleteVectors` method that supports deleting vectors by metadata filters, making it straightforward to remove all embeddings associated with a specific document.
|
|
587
587
|
|
|
@@ -637,7 +637,7 @@ await store.deleteVectors({
|
|
|
637
637
|
})
|
|
638
638
|
```
|
|
639
639
|
|
|
640
|
-
## Best
|
|
640
|
+
## Best practices
|
|
641
641
|
|
|
642
642
|
- Create indexes before bulk insertions
|
|
643
643
|
- Use batch operations for large insertions (the upsert method handles batching automatically)
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Amazon S3
|
|
1
|
+
# Amazon S3 vectors store
|
|
2
2
|
|
|
3
3
|
The `S3Vectors` class provides vector search using [Amazon S3 Vectors (Preview)](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-vectors.html). It stores vectors in **vector buckets** and performs similarity search in **vector indexes**, with JSON-based metadata filters.
|
|
4
4
|
|
|
@@ -30,7 +30,7 @@ yarn add @mastra/s3vectors@latest
|
|
|
30
30
|
bun add @mastra/s3vectors@latest
|
|
31
31
|
```
|
|
32
32
|
|
|
33
|
-
## Usage
|
|
33
|
+
## Usage example
|
|
34
34
|
|
|
35
35
|
```typescript
|
|
36
36
|
import { S3Vectors } from '@mastra/s3vectors'
|
|
@@ -82,7 +82,7 @@ const results = await store.query({
|
|
|
82
82
|
await store.disconnect()
|
|
83
83
|
```
|
|
84
84
|
|
|
85
|
-
## Constructor
|
|
85
|
+
## Constructor options
|
|
86
86
|
|
|
87
87
|
**vectorBucketName** (`string`): Target S3 Vectors vector bucket name.
|
|
88
88
|
|
|
@@ -92,7 +92,7 @@ await store.disconnect()
|
|
|
92
92
|
|
|
93
93
|
## Methods
|
|
94
94
|
|
|
95
|
-
### createIndex()
|
|
95
|
+
### `createIndex()`
|
|
96
96
|
|
|
97
97
|
Creates a new vector index in the configured vector bucket. If the index already exists, the call validates the schema and becomes a no-op (existing metric and dimension are preserved).
|
|
98
98
|
|
|
@@ -102,7 +102,7 @@ Creates a new vector index in the configured vector bucket. If the index already
|
|
|
102
102
|
|
|
103
103
|
**metric** (`'cosine' | 'euclidean'`): Distance metric for similarity search. \`dotproduct\` is not supported by S3 Vectors. (Default: `cosine`)
|
|
104
104
|
|
|
105
|
-
### upsert()
|
|
105
|
+
### `upsert()`
|
|
106
106
|
|
|
107
107
|
Adds or replaces vectors (full-record put). If `ids` aren't provided, UUIDs are generated.
|
|
108
108
|
|
|
@@ -114,7 +114,7 @@ Adds or replaces vectors (full-record put). If `ids` aren't provided, UUIDs are
|
|
|
114
114
|
|
|
115
115
|
**ids** (`string[]`): Optional vector IDs (auto-generated if not provided)
|
|
116
116
|
|
|
117
|
-
### query()
|
|
117
|
+
### `query()`
|
|
118
118
|
|
|
119
119
|
Searches for nearest neighbors with optional metadata filtering.
|
|
120
120
|
|
|
@@ -130,7 +130,7 @@ Searches for nearest neighbors with optional metadata filtering.
|
|
|
130
130
|
|
|
131
131
|
> **Note:** Results include `score = 1/(1 + distance)` so that higher is better while preserving the underlying distance ranking.
|
|
132
132
|
|
|
133
|
-
### describeIndex()
|
|
133
|
+
### `describeIndex()`
|
|
134
134
|
|
|
135
135
|
Returns information about the index.
|
|
136
136
|
|
|
@@ -146,19 +146,19 @@ interface IndexStats {
|
|
|
146
146
|
}
|
|
147
147
|
```
|
|
148
148
|
|
|
149
|
-
### deleteIndex()
|
|
149
|
+
### `deleteIndex()`
|
|
150
150
|
|
|
151
151
|
Deletes an index and its data.
|
|
152
152
|
|
|
153
153
|
**indexName** (`string`): Index to delete.
|
|
154
154
|
|
|
155
|
-
### listIndexes()
|
|
155
|
+
### `listIndexes()`
|
|
156
156
|
|
|
157
157
|
Lists all indexes in the configured vector bucket.
|
|
158
158
|
|
|
159
159
|
Returns: `Promise<string[]>`
|
|
160
160
|
|
|
161
|
-
### updateVector()
|
|
161
|
+
### `updateVector()`
|
|
162
162
|
|
|
163
163
|
Updates a vector or metadata for a specific ID within an index.
|
|
164
164
|
|
|
@@ -172,7 +172,7 @@ Updates a vector or metadata for a specific ID within an index.
|
|
|
172
172
|
|
|
173
173
|
**update.metadata** (`Record<string, any>`): New metadata to update
|
|
174
174
|
|
|
175
|
-
### deleteVector()
|
|
175
|
+
### `deleteVector()`
|
|
176
176
|
|
|
177
177
|
Deletes a specific vector by ID.
|
|
178
178
|
|
|
@@ -180,11 +180,11 @@ Deletes a specific vector by ID.
|
|
|
180
180
|
|
|
181
181
|
**id** (`string`): ID to delete.
|
|
182
182
|
|
|
183
|
-
### disconnect()
|
|
183
|
+
### `disconnect()`
|
|
184
184
|
|
|
185
185
|
Closes the underlying AWS SDK HTTP handler to free sockets.
|
|
186
186
|
|
|
187
|
-
## Response
|
|
187
|
+
## Response types
|
|
188
188
|
|
|
189
189
|
Query results are returned in this format:
|
|
190
190
|
|
|
@@ -197,7 +197,7 @@ interface QueryResult {
|
|
|
197
197
|
}
|
|
198
198
|
```
|
|
199
199
|
|
|
200
|
-
## Filter
|
|
200
|
+
## Filter syntax
|
|
201
201
|
|
|
202
202
|
S3 Vectors supports a strict subset of operators and value types. The Mastra filter translator:
|
|
203
203
|
|
|
@@ -238,7 +238,7 @@ S3 Vectors supports a strict subset of operators and value types. The Mastra fil
|
|
|
238
238
|
|
|
239
239
|
> **Note:** If you set `nonFilterableMetadataKeys` at index creation, those keys are stored but **can't** be used in filters.
|
|
240
240
|
|
|
241
|
-
## Error
|
|
241
|
+
## Error handling
|
|
242
242
|
|
|
243
243
|
The store throws typed errors that can be caught:
|
|
244
244
|
|
|
@@ -256,7 +256,7 @@ try {
|
|
|
256
256
|
}
|
|
257
257
|
```
|
|
258
258
|
|
|
259
|
-
## Environment
|
|
259
|
+
## Environment variables
|
|
260
260
|
|
|
261
261
|
Typical environment variables when wiring your app:
|
|
262
262
|
|
|
@@ -264,7 +264,7 @@ Typical environment variables when wiring your app:
|
|
|
264
264
|
- `AWS_REGION`: AWS region for the S3 Vectors bucket.
|
|
265
265
|
- **AWS credentials**: via the standard AWS SDK provider chain (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_PROFILE`, etc.).
|
|
266
266
|
|
|
267
|
-
## Best
|
|
267
|
+
## Best practices
|
|
268
268
|
|
|
269
269
|
- Choose the metric (`cosine` or `euclidean`) to match your embedding model; `dotproduct` isn't supported.
|
|
270
270
|
- Keep **filterable** metadata small and structured (string/number/boolean). Store large text (e.g., `content`) as **non-filterable**.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@mastra/s3vectors",
|
|
3
|
-
"version": "1.0.2
|
|
3
|
+
"version": "1.0.2",
|
|
4
4
|
"description": "Amazon S3 Vectors store provider for Mastra",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"files": [
|
|
@@ -36,10 +36,10 @@
|
|
|
36
36
|
"tsup": "^8.5.1",
|
|
37
37
|
"typescript": "^5.9.3",
|
|
38
38
|
"vitest": "4.0.18",
|
|
39
|
-
"@internal/lint": "0.0.
|
|
40
|
-
"@internal/storage-test-utils": "0.0.
|
|
41
|
-
"@
|
|
42
|
-
"@
|
|
39
|
+
"@internal/lint": "0.0.67",
|
|
40
|
+
"@internal/storage-test-utils": "0.0.63",
|
|
41
|
+
"@internal/types-builder": "0.0.42",
|
|
42
|
+
"@mastra/core": "1.11.0"
|
|
43
43
|
},
|
|
44
44
|
"peerDependencies": {
|
|
45
45
|
"@mastra/core": ">=1.0.0-0 <2.0.0-0"
|