@mastra/pg 1.20.0-alpha.2 → 1.20.0-alpha.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (24) hide show
  1. package/CHANGELOG.md +30 -0
  2. package/dist/docs/SKILL.md +9 -7
  3. package/dist/docs/assets/SOURCE_MAP.json +1 -1
  4. package/dist/docs/references/docs-deployment-workers.md +1 -1
  5. package/dist/docs/references/docs-memory-semantic-recall.md +2 -1
  6. package/dist/docs/references/docs-memory-working-memory.md +1 -0
  7. package/dist/docs/references/docs-storage-overview.md +14 -12
  8. package/dist/docs/references/{reference-storage-dynamodb.md → integrations-databases-dynamodb.md} +1 -1
  9. package/dist/docs/references/integrations-databases-neon.md +220 -0
  10. package/dist/docs/references/{reference-storage-postgresql.md → integrations-databases-postgresql.md} +1 -1
  11. package/dist/docs/references/reference-rag-chunking-and-embedding.md +182 -0
  12. package/dist/docs/references/reference-rag-metadata-filters.md +13 -4
  13. package/dist/docs/references/{guides-rag-overview.md → reference-rag-overview.md} +2 -2
  14. package/dist/docs/references/{guides-rag-retrieval.md → reference-rag-retrieval.md} +18 -1
  15. package/dist/docs/references/{guides-rag-vector-databases.md → reference-rag-vector-databases.md} +42 -1
  16. package/dist/docs/references/reference-storage-composite.md +4 -4
  17. package/dist/docs/references/reference-storage-retention.md +4 -4
  18. package/dist/index.cjs +5 -4
  19. package/dist/index.cjs.map +1 -1
  20. package/dist/index.js +5 -4
  21. package/dist/index.js.map +1 -1
  22. package/dist/storage/domains/memory/index.d.ts +3 -2
  23. package/dist/storage/domains/memory/index.d.ts.map +1 -1
  24. package/package.json +3 -3
package/CHANGELOG.md CHANGED
@@ -1,5 +1,35 @@
1
1
  # @mastra/pg
2
2
 
3
+ ## 1.20.0-alpha.4
4
+
5
+ ### Patch Changes
6
+
7
+ - Fixed a crash where updating a thread without a title (for example during observational memory buffering) could write a null title and violate the database's not-null constraint when running a newer @mastra/memory against an older storage package. Memory now checks whether the connected storage adapter supports partial thread updates and backfills the existing title for older adapters, so mixed-version deployments keep working. See #21041 for the original title-clobbering fix this makes backward compatible. ([#21257](https://github.com/mastra-ai/mastra/pull/21257))
8
+
9
+ - Storage adapters now declare support for partial thread updates, letting newer @mastra/memory preserve existing thread titles instead of overwriting them, while remaining safe against older versions. ([#21257](https://github.com/mastra-ai/mastra/pull/21257))
10
+
11
+ - Updated dependencies [[`dc4a25d`](https://github.com/mastra-ai/mastra/commit/dc4a25d41af4e2fe97a816070eaec6aa963ab53b), [`dc4a25d`](https://github.com/mastra-ai/mastra/commit/dc4a25d41af4e2fe97a816070eaec6aa963ab53b)]:
12
+ - @mastra/core@1.58.0-alpha.15
13
+
14
+ ## 1.20.0-alpha.3
15
+
16
+ ### Patch Changes
17
+
18
+ - Fixed generated thread titles being clobbered during a turn ([#21041](https://github.com/mastra-ai/mastra/pull/21041))
19
+
20
+ `updateThread` required both `title` and `metadata`, so callers that only needed to
21
+ change metadata (message persistence, working memory, observational memory, channel
22
+ subscriptions) had to read the thread and pass its title back. When title generation
23
+ finished between that read and the write, the freshly generated title was overwritten
24
+ with the stale one.
25
+
26
+ `title` and `metadata` are now independently optional: omitting one leaves that column
27
+ untouched. Callers that only change metadata no longer send a title, and message
28
+ persistence no longer rewrites a thread row it just read.
29
+
30
+ - Updated dependencies [[`1c75e32`](https://github.com/mastra-ai/mastra/commit/1c75e32f7fc0b9fb6f548b4407feaec8a1440212), [`c47165c`](https://github.com/mastra-ai/mastra/commit/c47165c983c87594c6952f1fd2fa51a90205034c), [`e08e789`](https://github.com/mastra-ai/mastra/commit/e08e789c1bf4cd2fe46363f7a4728536ceccc9bd), [`35cc901`](https://github.com/mastra-ai/mastra/commit/35cc90102cf834a84827acaf9eee0b6d6d1e2a3b), [`a8b4cf0`](https://github.com/mastra-ai/mastra/commit/a8b4cf02823cffebc4751a53337dfacf097c1ae1), [`f33264f`](https://github.com/mastra-ai/mastra/commit/f33264f517ae603279afd5c4251e2b40f6dd3618), [`689f2c4`](https://github.com/mastra-ai/mastra/commit/689f2c4b6c0835fe455702b01d21daa8abcd9331), [`eeae63e`](https://github.com/mastra-ai/mastra/commit/eeae63e7fbe8e1f237adc69bca6e2ac13c5ca907), [`4c186a0`](https://github.com/mastra-ai/mastra/commit/4c186a017275f45e6ed4c09de0f89550e2d09e8c), [`b0fa077`](https://github.com/mastra-ai/mastra/commit/b0fa077bcbc9b08551846fe372a0d3d15b71ed72)]:
31
+ - @mastra/core@1.58.0-alpha.8
32
+
3
33
  ## 1.20.0-alpha.2
4
34
 
5
35
  ### Patch Changes
@@ -3,7 +3,7 @@ name: mastra-pg
3
3
  description: Documentation for @mastra/pg. Use when working with @mastra/pg APIs, configuration, or implementation.
4
4
  metadata:
5
5
  package: "@mastra/pg"
6
- version: "1.20.0-alpha.2"
6
+ version: "1.20.0-alpha.4"
7
7
  ---
8
8
 
9
9
  ## When to use
@@ -21,11 +21,11 @@ Read the individual reference documents for detailed explanations and code examp
21
21
  - [Working memory](references/docs-memory-working-memory.md) - Learn how to configure working memory in Mastra to store persistent user data, preferences.
22
22
  - [Storage overview](references/docs-storage-overview.md) - Configure storage for Mastra to persist runtime state across agents, workflows, observability, evals, schedules, and memory.
23
23
 
24
- ### Guides
24
+ ### Integrations
25
25
 
26
- - [RAG (Retrieval-Augmented Generation) in Mastra](references/guides-rag-overview.md) - Overview of Retrieval-Augmented Generation (RAG) in Mastra, detailing its capabilities for enhancing LLM outputs with relevant context.
27
- - [Retrieval, semantic search, reranking](references/guides-rag-retrieval.md) - Guide on retrieval processes in Mastra's RAG systems, including semantic search, filtering, and re-ranking.
28
- - [Storing embeddings in a vector database](references/guides-rag-vector-databases.md) - Guide on vector storage options in Mastra, including embedded and dedicated vector databases for similarity search.
26
+ - [DynamoDB](references/integrations-databases-dynamodb.md) - Documentation for the DynamoDB storage implementation in Mastra, using a single-table design with ElectroDB.
27
+ - [Neon Postgres](references/integrations-databases-neon.md) - Use Neon Postgres as a Mastra storage and vector backend.
28
+ - [PostgreSQL](references/integrations-databases-postgresql.md) - Documentation for the PostgreSQL storage implementation in Mastra.
29
29
 
30
30
  ### Reference
31
31
 
@@ -33,10 +33,12 @@ Read the individual reference documents for detailed explanations and code examp
33
33
  - [Reference: MessageHistory](references/reference-processors-message-history-processor.md) - Documentation for the MessageHistory processor in Mastra, which handles retrieval and persistence of conversation history.
34
34
  - [Reference: SemanticRecall](references/reference-processors-semantic-recall-processor.md) - Documentation for the SemanticRecall processor in Mastra, which enables semantic search over conversation history using vector embeddings.
35
35
  - [Reference: WorkingMemory](references/reference-processors-working-memory-processor.md) - Documentation for the WorkingMemory processor in Mastra, which injects persistent user/context data as system instructions.
36
+ - [Chunking and embedding documents](references/reference-rag-chunking-and-embedding.md) - Guide on chunking and embedding documents in Mastra for efficient processing and retrieval.
36
37
  - [Reference: Metadata filters](references/reference-rag-metadata-filters.md) - Documentation for metadata filtering capabilities in Mastra, which allow for precise querying of vector search results across different vector stores.
38
+ - [RAG (Retrieval-Augmented Generation) in Mastra](references/reference-rag-overview.md) - Overview of Retrieval-Augmented Generation (RAG) in Mastra, detailing its capabilities for enhancing LLM outputs with relevant context.
39
+ - [Retrieval, semantic search, reranking](references/reference-rag-retrieval.md) - Guide on retrieval processes in Mastra's RAG systems, including semantic search, filtering, and re-ranking.
40
+ - [Storing embeddings in a vector database](references/reference-rag-vector-databases.md) - Guide on vector storage options in Mastra, including embedded and dedicated vector databases for similarity search.
37
41
  - [Reference: Composite storage](references/reference-storage-composite.md) - Documentation for combining multiple storage backends in Mastra.
38
- - [Reference: DynamoDB storage](references/reference-storage-dynamodb.md) - Documentation for the DynamoDB storage implementation in Mastra, using a single-table design with ElectroDB.
39
- - [Reference: PostgreSQL storage](references/reference-storage-postgresql.md) - Documentation for the PostgreSQL storage implementation in Mastra.
40
42
  - [Reference: Storage retention (prune)](references/reference-storage-retention.md) - API reference for retention policies and prune() on Mastra storage.
41
43
  - [Reference: createVectorQueryTool()](references/reference-tools-vector-query-tool.md) - Documentation for the Vector Query Tool in Mastra, which facilitates semantic search over vector stores with filtering and reranking capabilities.
42
44
  - [Reference: PG vector store](references/reference-vectors-pg.md) - Documentation for the PgVector class in Mastra, which provides vector search using PostgreSQL with pgvector extension.
@@ -1,5 +1,5 @@
1
1
  {
2
- "version": "1.20.0-alpha.2",
2
+ "version": "1.20.0-alpha.4",
3
3
  "package": "@mastra/pg",
4
4
  "exports": {},
5
5
  "modules": {}
@@ -2,7 +2,7 @@
2
2
 
3
3
  # Workers
4
4
 
5
- > **Beta:** This feature is in beta. The API is stable enough for production use, but some details may change. See [known limitations](#known-limitations) for current gaps.
5
+ > **Beta:** Breaking changes may occur without a major version bump until the API is stable. See [known limitations](#known-limitations) for current gaps.
6
6
 
7
7
  Workers handle background processing outside the request-response cycle. Workflow step execution, cron-based scheduling, and long-running tool calls all run in workers, keeping the API responsive.
8
8
 
@@ -148,6 +148,7 @@ Each vector store page below includes installation instructions, configuration p
148
148
  - [libSQL](https://mastra.ai/reference/vectors/libsql)
149
149
  - [MongoDB](https://mastra.ai/reference/vectors/mongodb)
150
150
  - [OpenSearch](https://mastra.ai/reference/vectors/opensearch)
151
+ - [OracleDB](https://mastra.ai/reference/vectors/oracledb)
151
152
  - [Pinecone](https://mastra.ai/reference/vectors/pinecone)
152
153
  - [PostgreSQL](https://mastra.ai/reference/vectors/pg)
153
154
  - [Qdrant](https://mastra.ai/reference/vectors/qdrant)
@@ -180,7 +181,7 @@ const agent = new Agent({
180
181
  })
181
182
  ```
182
183
 
183
- > **Note:** `scope: 'resource'` is supported by the LibSQL, PostgreSQL, MongoDB, and Upstash storage adapters.
184
+ > **Note:** `scope: 'resource'` is supported by the LibSQL, OracleDB, PostgreSQL, MongoDB, and Upstash storage adapters.
184
185
 
185
186
  ### Metadata filtering
186
187
 
@@ -129,6 +129,7 @@ Resource-scoped working memory requires specific storage adapters that support t
129
129
 
130
130
  - **libSQL** (`@mastra/libsql`)
131
131
  - **PostgreSQL** (`@mastra/pg`)
132
+ - **OracleDB** (`@mastra/oracledb`)
132
133
  - **Upstash** (`@mastra/upstash`)
133
134
  - **MongoDB** (`@mastra/mongodb`)
134
135
 
@@ -190,18 +190,20 @@ You can also route `observability` to a dedicated analytics backend. See the [ob
190
190
 
191
191
  Each provider page includes installation instructions, configuration parameters, and usage examples:
192
192
 
193
- - [libSQL](https://mastra.ai/reference/storage/libsql)
194
- - [PostgreSQL](https://mastra.ai/reference/storage/postgresql)
195
- - [MongoDB](https://mastra.ai/reference/storage/mongodb)
196
- - [Upstash](https://mastra.ai/reference/storage/upstash)
197
- - [Redis](https://mastra.ai/reference/storage/redis)
198
- - [Cloudflare D1](https://mastra.ai/reference/storage/cloudflare-d1)
199
- - [Cloudflare KV & Durable Objects](https://mastra.ai/reference/storage/cloudflare)
200
- - [Convex](https://mastra.ai/reference/storage/convex)
201
- - [DynamoDB](https://mastra.ai/reference/storage/dynamodb)
202
- - [LanceDB](https://mastra.ai/reference/storage/lance)
203
- - [Microsoft SQL Server](https://mastra.ai/reference/storage/mssql)
204
- - [Google Cloud Spanner](https://mastra.ai/reference/storage/spanner)
193
+ - [Cloudflare D1](https://mastra.ai/integrations/databases/cloudflare-d1)
194
+ - [Cloudflare KV & Durable Objects](https://mastra.ai/integrations/databases/cloudflare-kv)
195
+ - [Convex](https://mastra.ai/integrations/databases/convex)
196
+ - [DynamoDB](https://mastra.ai/integrations/databases/dynamodb)
197
+ - [Google Cloud Spanner](https://mastra.ai/integrations/databases/spanner)
198
+ - [LanceDB](https://mastra.ai/integrations/databases/lancedb)
199
+ - [libSQL](https://mastra.ai/integrations/databases/libsql)
200
+ - [Microsoft SQL Server](https://mastra.ai/integrations/databases/mssql)
201
+ - [MongoDB](https://mastra.ai/integrations/databases/mongodb)
202
+ - [Neon Postgres](https://mastra.ai/integrations/databases/neon)
203
+ - [OracleDB](https://mastra.ai/integrations/databases/oracledb)
204
+ - [PostgreSQL](https://mastra.ai/integrations/databases/postgresql)
205
+ - [Redis](https://mastra.ai/integrations/databases/redis)
206
+ - [Upstash](https://mastra.ai/integrations/databases/upstash)
205
207
 
206
208
  > **Tip:** libSQL is the fastest path for local development because it doesn't require running a separate database server.
207
209
 
@@ -1,6 +1,6 @@
1
1
  > Discover all available pages from the documentation index: https://mastra.ai/llms.txt
2
2
 
3
- # DynamoDB storage
3
+ # DynamoDB
4
4
 
5
5
  The DynamoDB storage implementation provides a high-capacity and performant NoSQL database solution for Mastra, using a single-table design pattern with [ElectroDB](https://electrodb.dev/).
6
6
 
@@ -0,0 +1,220 @@
1
+ > Discover all available pages from the documentation index: https://mastra.ai/llms.txt
2
+
3
+ # Neon Postgres
4
+
5
+ [Neon](https://neon.com) is a managed PostgreSQL service. Mastra connects to Neon through [`PostgresStore`](https://mastra.ai/integrations/databases/postgresql), which uses the Node.js `pg` driver. Neon doesn't require a separate Mastra storage package.
6
+
7
+ ## Quickstart
8
+
9
+ Create a Neon project, then copy its [pooled connection string](https://neon.com/docs/connect/connect-from-any-app) into `DATABASE_URL`.
10
+
11
+ Install the PostgreSQL storage package:
12
+
13
+ **npm**:
14
+
15
+ ```bash
16
+ npm install @mastra/pg@latest
17
+ ```
18
+
19
+ **pnpm**:
20
+
21
+ ```bash
22
+ pnpm add @mastra/pg@latest
23
+ ```
24
+
25
+ **Yarn**:
26
+
27
+ ```bash
28
+ yarn add @mastra/pg@latest
29
+ ```
30
+
31
+ **Bun**:
32
+
33
+ ```bash
34
+ bun add @mastra/pg@latest
35
+ ```
36
+
37
+ Pass the pooled connection string to `PostgresStore`:
38
+
39
+ ```typescript
40
+ import { Mastra } from '@mastra/core/mastra'
41
+ import { PostgresStore } from '@mastra/pg'
42
+
43
+ export const mastra = new Mastra({
44
+ storage: new PostgresStore({
45
+ id: 'neon-storage',
46
+ connectionString: process.env.DATABASE_URL!,
47
+ }),
48
+ })
49
+ ```
50
+
51
+ On startup, Mastra calls `storage.init()` and creates its tables and indexes in Neon.
52
+
53
+ ## Set up Neon with the CLI
54
+
55
+ The Neon CLI requires Node.js 20.19 or newer.
56
+
57
+ Install the [Neon CLI](https://neon.com/docs/cli/install):
58
+
59
+ **npm**:
60
+
61
+ ```bash
62
+ npm install -g neon
63
+ ```
64
+
65
+ **pnpm**:
66
+
67
+ ```bash
68
+ pnpm add -g neon
69
+ ```
70
+
71
+ **Yarn**:
72
+
73
+ ```bash
74
+ yarn global add neon
75
+ ```
76
+
77
+ **Bun**:
78
+
79
+ ```bash
80
+ bun add --global neon
81
+ ```
82
+
83
+ From your project directory, link an existing Neon project or create one:
84
+
85
+ ```bash
86
+ neon link
87
+ ```
88
+
89
+ `neon link` signs you in and writes project context to `.neon`. It also pulls the current branch's environment variables. Run `neon env pull` again whenever you need to refresh them:
90
+
91
+ ```bash
92
+ neon env pull
93
+ ```
94
+
95
+ The command writes `DATABASE_URL` and `DATABASE_URL_UNPOOLED` to `.env.local` or to `.env` when that file already exists.
96
+
97
+ ### Add `neon.ts`
98
+
99
+ To manage branch settings in the repository, create a [`neon.ts`](https://neon.com/docs/reference/neon-ts) file:
100
+
101
+ ```bash
102
+ neon config init
103
+ ```
104
+
105
+ This installs `@neon/config` and `@neon/env` and creates a starter config. For example, the following policy expires new non-default branches after seven days:
106
+
107
+ ```typescript
108
+ import { defineConfig } from '@neon/config/v1'
109
+
110
+ export default defineConfig({
111
+ branch: branch => {
112
+ if (branch.isDefault) {
113
+ return {}
114
+ }
115
+
116
+ if (!branch.exists) {
117
+ return { ttl: '7d' }
118
+ }
119
+
120
+ return {}
121
+ },
122
+ })
123
+ ```
124
+
125
+ Apply the config:
126
+
127
+ ```bash
128
+ neon deploy
129
+ ```
130
+
131
+ ## Install Neon agent skills
132
+
133
+ Install the [`neon` and `neon-postgres` agent skills](https://neon.com/docs/ai/agent-skills) from your project directory:
134
+
135
+ **npm**:
136
+
137
+ ```bash
138
+ npx skills add neondatabase/agent-skills --skill neon --skill neon-postgres -y
139
+ ```
140
+
141
+ **pnpm**:
142
+
143
+ ```bash
144
+ pnpm dlx skills add neondatabase/agent-skills --skill neon --skill neon-postgres -y
145
+ ```
146
+
147
+ **Yarn**:
148
+
149
+ ```bash
150
+ yarn dlx skills add neondatabase/agent-skills --skill neon --skill neon-postgres -y
151
+ ```
152
+
153
+ **Bun**:
154
+
155
+ ```bash
156
+ bun x skills add neondatabase/agent-skills --skill neon --skill neon-postgres -y
157
+ ```
158
+
159
+ The skills give compatible coding agents Neon setup instructions and Postgres-specific guidance.
160
+
161
+ ## Production connections
162
+
163
+ Neon provides direct and pooled connection strings:
164
+
165
+ - `DATABASE_URL` uses Neon's pooled endpoint. Use it for normal application traffic.
166
+ - `DATABASE_URL_UNPOOLED` connects directly to Postgres. Use it for administrative tasks or schema changes that run outside the application.
167
+
168
+ For most applications, use the pooled `DATABASE_URL` shown in the quickstart and let Mastra initialize the schema at startup. `PostgresStore` supports initialization through a transaction-pooled connection.
169
+
170
+ ### Separate schema initialization
171
+
172
+ If the runtime database role shouldn't create tables or indexes, initialize the schema during deployment with the direct connection:
173
+
174
+ ```typescript
175
+ import { PostgresStore } from '@mastra/pg'
176
+
177
+ const storage = new PostgresStore({
178
+ id: 'neon-storage-init',
179
+ connectionString: process.env.DATABASE_URL_UNPOOLED!,
180
+ })
181
+
182
+ try {
183
+ await storage.init()
184
+ } finally {
185
+ await storage.close()
186
+ }
187
+ ```
188
+
189
+ After the schema exists, use the pooled connection at runtime and disable automatic initialization:
190
+
191
+ ```typescript
192
+ import { Mastra } from '@mastra/core/mastra'
193
+ import { PostgresStore } from '@mastra/pg'
194
+
195
+ export const mastra = new Mastra({
196
+ storage: new PostgresStore({
197
+ id: 'neon-storage',
198
+ connectionString: process.env.DATABASE_URL!,
199
+ disableInit: true,
200
+ }),
201
+ })
202
+ ```
203
+
204
+ See [Neon connection pooling](https://neon.com/docs/connect/connection-pooling) for details.
205
+
206
+ ## Vector search
207
+
208
+ Neon supports the [`pgvector` extension](https://neon.com/docs/extensions/pgvector). Use the direct `DATABASE_URL_UNPOOLED` connection while creating the extension and indexes. After setup, configure [`PgVector`](https://mastra.ai/reference/vectors/pg) with the pooled connection for runtime queries:
209
+
210
+ ```typescript
211
+ import { PgVector } from '@mastra/pg'
212
+
213
+ const vector = new PgVector({
214
+ id: 'neon-vector',
215
+ connectionString: process.env.DATABASE_URL!,
216
+ disableInit: true,
217
+ })
218
+ ```
219
+
220
+ Setting `disableInit: true` prevents the runtime process from creating or changing the extension, tables and indexes.
@@ -1,6 +1,6 @@
1
1
  > Discover all available pages from the documentation index: https://mastra.ai/llms.txt
2
2
 
3
- # PostgreSQL storage
3
+ # PostgreSQL
4
4
 
5
5
  The PostgreSQL storage implementation provides a production-ready storage solution using PostgreSQL databases.
6
6
 
@@ -0,0 +1,182 @@
1
+ > Discover all available pages from the documentation index: https://mastra.ai/llms.txt
2
+
3
+ # Chunking and embedding documents
4
+
5
+ Before processing, create a MDocument instance from your content. You can initialize it from different formats:
6
+
7
+ ```ts
8
+ import { MDocument } from '@mastra/rag'
9
+
10
+ const doc = MDocument.fromText('Your plain text content...')
11
+ const docFromHTML = MDocument.fromHTML('<html>Your HTML content...</html>')
12
+ const docFromMarkdown = MDocument.fromMarkdown('# Your Markdown content...')
13
+ const docFromJSON = MDocument.fromJSON(`{ "key": "value" }`)
14
+ ```
15
+
16
+ ## Document processing
17
+
18
+ Use `chunk` to split documents into manageable pieces. Mastra supports multiple chunking strategies optimized for different document types:
19
+
20
+ - `recursive`: Smart splitting based on content structure
21
+ - `character`: Simple character-based splits
22
+ - `token`: Token-aware splitting
23
+ - `markdown`: Markdown-aware splitting
24
+ - `semantic-markdown`: Markdown splitting based on related header families
25
+ - `html`: HTML structure-aware splitting
26
+ - `json`: JSON structure-aware splitting
27
+ - `latex`: LaTeX structure-aware splitting
28
+ - `sentence`: Sentence-aware splitting
29
+
30
+ > **Note:** Each strategy accepts different parameters optimized for its chunking approach.
31
+
32
+ Here's an example of how to use the `recursive` strategy:
33
+
34
+ ```ts
35
+ const chunks = await doc.chunk({
36
+ strategy: 'recursive',
37
+ maxSize: 512,
38
+ overlap: 50,
39
+ separators: ['\n'],
40
+ extract: {
41
+ metadata: true, // Optionally extract metadata
42
+ },
43
+ })
44
+ ```
45
+
46
+ For text where preserving sentence structure is important, here's an example of how to use the `sentence` strategy:
47
+
48
+ ```ts
49
+ const chunks = await doc.chunk({
50
+ strategy: 'sentence',
51
+ maxSize: 450,
52
+ minSize: 50,
53
+ overlap: 0,
54
+ sentenceEnders: ['.'],
55
+ })
56
+ ```
57
+
58
+ For markdown documents where preserving the semantic relationships between sections is important, here's an example of how to use the `semantic-markdown` strategy:
59
+
60
+ ```ts
61
+ const chunks = await doc.chunk({
62
+ strategy: 'semantic-markdown',
63
+ joinThreshold: 500,
64
+ modelName: 'gpt-5',
65
+ })
66
+ ```
67
+
68
+ > **Note:** Metadata extraction may use LLM calls, so ensure your API key is set.
69
+
70
+ We go deeper into chunking strategies in our [`chunk()` reference documentation](https://mastra.ai/reference/rag/chunk).
71
+
72
+ ## Embedding generation
73
+
74
+ Transform chunks into embeddings using your preferred provider. Mastra supports embedding models through the model router.
75
+
76
+ ### Using the Model Router
77
+
78
+ The simplest way is to use Mastra's model router with `provider/model` strings:
79
+
80
+ ```ts
81
+ import { ModelRouterEmbeddingModel } from '@mastra/core/llm'
82
+ import { embedMany } from 'ai'
83
+
84
+ const { embeddings } = await embedMany({
85
+ model: new ModelRouterEmbeddingModel('openai/text-embedding-3-small'),
86
+ values: chunks.map(chunk => chunk.text),
87
+ })
88
+ ```
89
+
90
+ Mastra supports OpenAI and Google embedding models. For a complete list of supported embedding models, see the [embeddings reference](https://mastra.ai/reference/rag/embeddings).
91
+
92
+ The model router automatically handles API key detection from environment variables.
93
+
94
+ The embedding functions return vectors, arrays of numbers representing the semantic meaning of your text, ready for similarity searches in your vector database.
95
+
96
+ ### Configuring Embedding Dimensions
97
+
98
+ Embedding models typically output vectors with a fixed number of dimensions (e.g., 1536 for OpenAI's `text-embedding-3-small`). Some models support reducing this dimensionality, which can help:
99
+
100
+ - Decrease storage requirements in vector databases
101
+ - Reduce computational costs for similarity searches
102
+
103
+ Here are some supported models:
104
+
105
+ OpenAI (text-embedding-3 models):
106
+
107
+ ```ts
108
+ import { ModelRouterEmbeddingModel } from '@mastra/core/llm'
109
+
110
+ const { embeddings } = await embedMany({
111
+ model: new ModelRouterEmbeddingModel('openai/text-embedding-3-small'),
112
+ options: {
113
+ dimensions: 256, // Only supported in text-embedding-3 and later
114
+ },
115
+ values: chunks.map(chunk => chunk.text),
116
+ })
117
+ ```
118
+
119
+ Google (text-embedding-001):
120
+
121
+ ```ts
122
+ const { embeddings } = await embedMany({
123
+ model: google('gemini-embedding-001', {
124
+ outputDimensionality: 256, // Truncates excessive values from the end
125
+ }),
126
+ values: chunks.map(chunk => chunk.text),
127
+ })
128
+ ```
129
+
130
+ > **Vector Database Compatibility:** When storing embeddings, the vector database index must be configured to match the output size of your embedding model. If the dimensions don't match, you may get errors or data corruption.
131
+
132
+ ## Example: Complete pipeline
133
+
134
+ Here's an example showing document processing, embedding generation, and vector storage:
135
+
136
+ ```ts
137
+ import { ModelRouterEmbeddingModel } from '@mastra/core/llm'
138
+ import { PgVector } from '@mastra/pg'
139
+ import { MDocument } from '@mastra/rag'
140
+ import { embedMany } from 'ai'
141
+
142
+ const doc = MDocument.fromText(`
143
+ Climate change poses significant challenges to global agriculture.
144
+ Rising temperatures and changing precipitation patterns affect crop yields.
145
+ `)
146
+
147
+ const chunks = await doc.chunk({
148
+ strategy: 'recursive',
149
+ maxSize: 256,
150
+ overlap: 50,
151
+ })
152
+
153
+ const { embeddings } = await embedMany({
154
+ model: new ModelRouterEmbeddingModel('openai/text-embedding-3-small'),
155
+ values: chunks.map(chunk => chunk.text),
156
+ })
157
+
158
+ const vectorStore = new PgVector({
159
+ id: 'pg-vector',
160
+ connectionString: process.env.POSTGRES_CONNECTION_STRING,
161
+ })
162
+
163
+ await vectorStore.createIndex({
164
+ indexName: 'embeddings',
165
+ dimension: 1536,
166
+ })
167
+
168
+ await vectorStore.upsert({
169
+ indexName: 'embeddings',
170
+ vectors: embeddings,
171
+ })
172
+ ```
173
+
174
+ For more examples of different chunking strategies and embedding configurations, see:
175
+
176
+ - [Chunk Reference](https://mastra.ai/reference/rag/chunk)
177
+ - [Embeddings Reference](https://mastra.ai/reference/rag/embeddings)
178
+
179
+ For more details on vector databases and embeddings, see:
180
+
181
+ - [Vector Databases](https://mastra.ai/reference/rag/vector-databases)
182
+ - [Embedding API Reference](https://mastra.ai/reference/rag/embeddings)
@@ -2,7 +2,7 @@
2
2
 
3
3
  # Metadata filters
4
4
 
5
- Mastra provides a unified metadata filtering syntax across all vector stores, based on MongoDB/Sift query syntax. Each vector store translates these filters into their native format.
5
+ Mastra provides a unified metadata filtering syntax across all vector stores, based on MongoDB/Sift query syntax. Each vector store translates these filters into its native query format. For example, PgVector uses PostgreSQL JSONB predicates, while OracleDB stores metadata as Oracle JSON and compiles filters to `JSON_VALUE`, `JSON_EXISTS`, `REGEXP_LIKE`, and `LIKE` predicates with bound values.
6
6
 
7
7
  ## Basic example
8
8
 
@@ -34,11 +34,11 @@ const results = await store.query({
34
34
 
35
35
  ### Array Operators
36
36
 
37
- `$in`Matches any value in array{ category: { $in: \["A", "B"] } }Supported by: All except Couchbase`$nin`Matches none of the values{ status: { $nin: \["deleted", "archived"] } }Supported by: All except Couchbase`$all`Matches arrays containing all elements{ tags: { $all: \["urgent", "high"] } }Supported by: Astra, Pinecone, Upstash, MongoDB`$elemMatch`Matches array elements meeting criteria{ scores: { $elemMatch: { $gt: 80 } } }Supported by: libSQL, PgVector, MongoDB
37
+ `$in`Matches any value in array{ category: { $in: \["A", "B"] } }Supported by: All except Couchbase`$nin`Matches none of the values{ status: { $nin: \["deleted", "archived"] } }Supported by: All except Couchbase`$all`Matches arrays containing all elements{ tags: { $all: \["urgent", "high"] } }Supported by: Astra, Pinecone, Upstash, MongoDB, OracleDB`$elemMatch`Matches array elements meeting criteria{ scores: { $elemMatch: { $gt: 80 } } }Supported by: libSQL, PgVector, MongoDB, OracleDB
38
38
 
39
39
  ### Logical Operators
40
40
 
41
- `$and`Logical AND{ $and: \[{ price: { $gt: 100 } }, { stock: { $gt: 0 } }] }Supported by: All except Vectorize, Couchbase`$or`Logical OR{ $or: \[{ status: "active" }, { priority: "high" }] }Supported by: All except Vectorize, Couchbase`$not`Logical NOT{ price: { $not: { $lt: 100 } } }Supported by: Astra, Qdrant, Upstash, PgVector, libSQL, MongoDB`$nor`Logical NOR{ $nor: \[{ status: "deleted" }, { archived: true }] }Supported by: Qdrant, Upstash, PgVector, libSQL, MongoDB
41
+ `$and`Logical AND{ $and: \[{ price: { $gt: 100 } }, { stock: { $gt: 0 } }] }Supported by: All except Vectorize, Couchbase`$or`Logical OR{ $or: \[{ status: "active" }, { priority: "high" }] }Supported by: All except Vectorize, Couchbase`$not`Logical NOT{ price: { $not: { $lt: 100 } } }Supported by: Astra, Qdrant, Upstash, PgVector, libSQL, MongoDB, OracleDB`$nor`Logical NOR{ $nor: \[{ status: "deleted" }, { archived: true }] }Supported by: Qdrant, Upstash, PgVector, libSQL, MongoDB, OracleDB
42
42
 
43
43
  ### Element Operators
44
44
 
@@ -46,7 +46,7 @@ const results = await store.query({
46
46
 
47
47
  ### Custom Operators
48
48
 
49
- `$contains`Text contains substring{ description: { $contains: "sale" } }Supported by: Upstash, libSQL, PgVector`$regex`Regular expression match{ name: { $regex: "^test" } }Supported by: Qdrant, PgVector, Upstash, MongoDB`$size`Array length check{ tags: { $size: { $gt: 2 } } }Supported by: Astra, libSQL, PgVector, MongoDB`$geo`Geospatial query{ location: { $geo: { type: "radius", ... } } }Supported by: Qdrant`$datetime`Datetime range query{ created: { $datetime: { range: { gt: "2024-01-01" } } } }Supported by: Qdrant`$hasId`Vector ID existence check{ $hasId: \["id1", "id2"] }Supported by: Qdrant`$hasVector`Vector existence check{ $hasVector: true }Supported by: Qdrant
49
+ `$contains`Text contains substring{ description: { $contains: "sale" } }Supported by: Upstash, libSQL, PgVector, OracleDB`$regex`Regular expression match{ name: { $regex: "^test" } }Supported by: Qdrant, PgVector, Upstash, MongoDB, OracleDB`$size`Array length check{ tags: { $size: 3 } }Supported by: Astra, libSQL, PgVector, MongoDB, OracleDB`$geo`Geospatial query{ location: { $geo: { type: "radius", ... } } }Supported by: Qdrant`$datetime`Datetime range query{ created: { $datetime: { range: { gt: "2024-01-01" } } } }Supported by: Qdrant`$hasId`Vector ID existence check{ $hasId: \["id1", "id2"] }Supported by: Qdrant`$hasVector`Vector existence check{ $hasVector: true }Supported by: Qdrant
50
50
 
51
51
  ## Common rules and restrictions
52
52
 
@@ -124,6 +124,14 @@ const results = await store.query({
124
124
  - Empty arrays in conditions are handled gracefully
125
125
  - Metadata is stored in a JSONB column for efficient querying
126
126
 
127
+ ### OracleDB
128
+
129
+ - Metadata is stored as Oracle JSON alongside each `VECTOR` row
130
+ - Scalar comparisons use `JSON_VALUE`, while array, existence, and element-match checks use `JSON_EXISTS`
131
+ - `$regex` uses Oracle `REGEXP_LIKE`; string `$contains` uses case-insensitive `LIKE`
132
+ - Nested fields are supported with dot notation and are converted to quoted Oracle JSON paths
133
+ - User-provided metadata values are bound as parameters instead of interpolated into SQL
134
+
127
135
  ### PgVector
128
136
 
129
137
  - Full support for PostgreSQL's native JSON querying capabilities
@@ -211,6 +219,7 @@ const results = await store.query({
211
219
  - [Cloudflare Vectorize](https://mastra.ai/reference/vectors/vectorize)
212
220
  - [libSQL](https://mastra.ai/reference/vectors/libsql)
213
221
  - [MongoDB](https://mastra.ai/reference/vectors/mongodb)
222
+ - [OracleDB](https://mastra.ai/reference/vectors/oracledb)
214
223
  - [PgStore](https://mastra.ai/reference/vectors/pg)
215
224
  - [Pinecone](https://mastra.ai/reference/vectors/pinecone)
216
225
  - [Qdrant](https://mastra.ai/reference/vectors/qdrant)
@@ -63,11 +63,11 @@ This example shows the essentials. Initialize a document and create chunks, then
63
63
 
64
64
  ## Document processing
65
65
 
66
- The basic building block of RAG is document processing. Documents can be chunked using strategies (recursive, sliding window, etc.) and enriched with metadata. See the [chunking and embedding doc](https://mastra.ai/guides/rag/chunking-and-embedding).
66
+ The basic building block of RAG is document processing. Documents can be chunked using strategies (recursive, sliding window, etc.) and enriched with metadata. See the [chunking and embedding doc](https://mastra.ai/reference/rag/chunking-and-embedding).
67
67
 
68
68
  ## Vector storage
69
69
 
70
- Mastra supports multiple vector stores for embedding persistence and similarity search, including pgvector, Pinecone, Qdrant, and MongoDB. See the [vector database doc](https://mastra.ai/guides/rag/vector-databases).
70
+ Mastra supports multiple vector stores for embedding persistence and similarity search, including pgvector, OracleDB, Pinecone, Qdrant, and MongoDB. See the [vector database doc](https://mastra.ai/reference/rag/vector-databases).
71
71
 
72
72
  ## More resources
73
73