@mastra/pg 1.7.2 → 1.8.0-alpha.0

package/CHANGELOG.md

@@ -1,5 +1,67 @@
 # @mastra/pg
 
+## 1.8.0-alpha.0
+
+### Minor Changes
+
+- Added `metadataIndexes` option to `createIndex()` for PgVector. This allows creating btree indexes on specific metadata fields in vector tables, significantly improving query performance when filtering by those fields. This is especially impactful for Memory's `memory_messages` table, which filters by `thread_id` and `resource_id` — previously causing sequential scans under load. ([#14034](https://github.com/mastra-ai/mastra/pull/14034))
+
+  **Usage example:**
+
+  ```ts
+  await pgVector.createIndex({
+    indexName: 'my_vectors',
+    dimension: 1536,
+    metadataIndexes: ['thread_id', 'resource_id'],
+  });
+  ```
+
+  Fixes #12109
+
+- Add support for pgvector's `bit` and `sparsevec` vector storage types ([#12815](https://github.com/mastra-ai/mastra/pull/12815))
+
+  You can now store binary and sparse vectors in `@mastra/pg`:
+
+  ```ts
+  // Binary vectors for fast similarity search
+  await db.createIndex({
+    indexName: 'my_binary_index',
+    dimension: 128,
+    metric: 'hamming', // or 'jaccard'
+    vectorType: 'bit',
+  });
+
+  // Sparse vectors for BM25/TF-IDF representations
+  await db.createIndex({
+    indexName: 'my_sparse_index',
+    dimension: 500,
+    metric: 'cosine',
+    vectorType: 'sparsevec',
+  });
+  ```
+
+  What's new:
+  - `vectorType: 'bit'` for binary vectors with `'hamming'` and `'jaccard'` distance metrics
+  - `vectorType: 'sparsevec'` for sparse vectors (cosine, euclidean, dotproduct)
+  - Automatic metric normalization: `bit` defaults to `'hamming'` when no metric is specified
+  - `includeVector` round-trips work correctly for all vector types
+  - Requires pgvector >= 0.7.0
+
+- Added `requestContext` column to the spans table. Request context data from tracing is now persisted alongside other span data. ([#14020](https://github.com/mastra-ai/mastra/pull/14020))
+
+- Added `requestContext` and `requestContextSchema` column support to dataset storage. Dataset items now persist request context alongside input and ground truth data. ([#13938](https://github.com/mastra-ai/mastra/pull/13938))
+
+### Patch Changes
+
+- Added resilient column handling to insert and update operations. Unknown columns in records are now silently dropped instead of causing SQL errors, ensuring forward compatibility when newer domain packages add fields that haven't been migrated yet. ([#14021](https://github.com/mastra-ai/mastra/pull/14021))
+
+  For example, calling `db.insert({ tableName, record: { id: '1', title: 'Hello', futureField: 'value' } })` will silently ignore `futureField` if it doesn't exist in the database table, rather than throwing. The same applies to `update` — unknown fields in the data payload are dropped before building the SQL statement.
+
+- Fixed slow semantic recall in the Postgres storage adapter for large threads. Recall now completes in under 500ms even for threads with 7,000+ messages, down from ~30 seconds. (Fixes #11702) ([#14022](https://github.com/mastra-ai/mastra/pull/14022))
+
+- Updated dependencies [[`4f71b43`](https://github.com/mastra-ai/mastra/commit/4f71b436a4a6b8839842d8da47b57b84509af56c), [`a070277`](https://github.com/mastra-ai/mastra/commit/a07027766ce195ba74d0783116d894cbab25d44c), [`b628b91`](https://github.com/mastra-ai/mastra/commit/b628b9128b372c0f54214d902b07279f03443900), [`332c014`](https://github.com/mastra-ai/mastra/commit/332c014e076b81edf7fe45b58205882726415e90), [`6b63153`](https://github.com/mastra-ai/mastra/commit/6b63153878ea841c0f4ce632ba66bb33e57e9c1b), [`4246e34`](https://github.com/mastra-ai/mastra/commit/4246e34cec9c26636d0965942268e6d07c346671), [`b8837ee`](https://github.com/mastra-ai/mastra/commit/b8837ee77e2e84197609762bfabd8b3da326d30c), [`5d950f7`](https://github.com/mastra-ai/mastra/commit/5d950f7bf426a215a1808f0abef7de5c8336ba1c), [`28c85b1`](https://github.com/mastra-ai/mastra/commit/28c85b184fc32b40f7f160483c982da6d388ecbd), [`e9a08fb`](https://github.com/mastra-ai/mastra/commit/e9a08fbef1ada7e50e961e2f54f55e8c10b4a45c), [`631ffd8`](https://github.com/mastra-ai/mastra/commit/631ffd82fed108648b448b28e6a90e38c5f53bf5), [`aae2295`](https://github.com/mastra-ai/mastra/commit/aae2295838a2d329ad6640829e87934790ffe5b8), [`aa61f29`](https://github.com/mastra-ai/mastra/commit/aa61f29ff8095ce46a4ae16e46c4d8c79b2b685b), [`7ff3714`](https://github.com/mastra-ai/mastra/commit/7ff37148515439bb3be009a60e02c3e363299760), [`41d79a1`](https://github.com/mastra-ai/mastra/commit/41d79a14bd8cb6de1e2565fd0a04786bae2f211b), [`e673376`](https://github.com/mastra-ai/mastra/commit/e6733763ad1321aa7e5ae15096b9c2104f93b1f3), [`b2204c9`](https://github.com/mastra-ai/mastra/commit/b2204c98a42848bbfb6f0440f005dc2b6354f1cd), [`a1bf1e3`](https://github.com/mastra-ai/mastra/commit/a1bf1e385ed4c0ef6f11b56c5887442970d127f2), [`b6f647a`](https://github.com/mastra-ai/mastra/commit/b6f647ae2388e091f366581595feb957e37d5b40), [`0c57b8b`](https://github.com/mastra-ai/mastra/commit/0c57b8b0a69a97b5a4ae3f79be6c610f29f3cf7b), [`b081f27`](https://github.com/mastra-ai/mastra/commit/b081f272cf411716e1d6bd72ceac4bcee2657b19), [`0c09eac`](https://github.com/mastra-ai/mastra/commit/0c09eacb1926f64cfdc9ae5c6d63385cf8c9f72c), [`6b9b93d`](https://github.com/mastra-ai/mastra/commit/6b9b93d6f459d1ba6e36f163abf62a085ddb3d64), [`31b6067`](https://github.com/mastra-ai/mastra/commit/31b6067d0cc3ab10e1b29c36147f3b5266bc714a), [`797ac42`](https://github.com/mastra-ai/mastra/commit/797ac4276de231ad2d694d9aeca75980f6cd0419), [`0bc289e`](https://github.com/mastra-ai/mastra/commit/0bc289e2d476bf46c5b91c21969e8d0c6864691c), [`9b75a06`](https://github.com/mastra-ai/mastra/commit/9b75a06e53ebb0b950ba7c1e83a0142047185f46), [`4c3a1b1`](https://github.com/mastra-ai/mastra/commit/4c3a1b122ea083e003d71092f30f3b31680b01c0), [`85cc3b3`](https://github.com/mastra-ai/mastra/commit/85cc3b3b6f32ae4b083c26498f50d5b250ba944b), [`97ea28c`](https://github.com/mastra-ai/mastra/commit/97ea28c746e9e4147d56047bbb1c4a92417a3fec), [`d567299`](https://github.com/mastra-ai/mastra/commit/d567299cf81e02bd9d5221d4bc05967d6c224161), [`716ffe6`](https://github.com/mastra-ai/mastra/commit/716ffe68bed81f7c2690bc8581b9e140f7bf1c3d), [`8296332`](https://github.com/mastra-ai/mastra/commit/8296332de21c16e3dfc3d0b2d615720a6dc88f2f), [`4df2116`](https://github.com/mastra-ai/mastra/commit/4df211619dd922c047d396ca41cd7027c8c4c8e7), [`2219c1a`](https://github.com/mastra-ai/mastra/commit/2219c1acbd21da116da877f0036ffb985a9dd5a3), [`17c4145`](https://github.com/mastra-ai/mastra/commit/17c4145166099354545582335b5252bdfdfd908b)]:
+  - @mastra/core@1.11.0-alpha.0
+
 ## 1.7.2
 
 ### Patch Changes

package/dist/docs/SKILL.md

@@ -3,7 +3,7 @@ name: mastra-pg
 description: Documentation for @mastra/pg. Use when working with @mastra/pg APIs, configuration, or implementation.
 metadata:
   package: "@mastra/pg"
-  version: "1.7.2"
+  version: "1.8.0-alpha.0"
 ---
 
 ## When to use

package/dist/docs/assets/SOURCE_MAP.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.7.2",
+  "version": "1.8.0-alpha.0",
   "package": "@mastra/pg",
   "exports": {},
   "modules": {}

package/dist/docs/references/docs-memory-semantic-recall.md

@@ -35,7 +35,7 @@ const agent = new Agent({
 
 ## Using the recall() Method
 
-While `listMessages` retrieves messages by thread ID with basic pagination, [`recall()`](https://mastra.ai/reference/memory/recall) adds support for **semantic search**. When you need to find messages by meaning rather than just recency, use `recall()` with a `vectorSearchString`:
+While `listMessages` retrieves messages by thread ID with basic pagination, [`recall()`](https://mastra.ai/reference/memory/recall) adds support for **semantic search**. When you need to find messages by meaning rather than recency, use `recall()` with a `vectorSearchString`:
 
 ```typescript
 const memory = await agent.getMemory()
@@ -264,7 +264,7 @@ For detailed information about index configuration options and performance tunin
 
 ## Disabling
 
-There is a performance impact to using semantic recall. New messages are converted into embeddings and used to query a vector database before new messages are sent to the LLM.
+Semantic recall has a performance impact. New messages are converted into embeddings and used to query a vector database before new messages are sent to the LLM.
 
 Semantic recall is enabled by default but can be disabled when not needed:
 

package/dist/docs/references/docs-memory-storage.md

@@ -170,7 +170,7 @@ export const agent = new Agent({
 })
 ```
 
-Title generation runs asynchronously after the agent responds and does not affect response time.
+Title generation runs asynchronously after the agent responds and doesn't affect response time.
 
 To optimize cost or behavior, provide a smaller [`model`](https://mastra.ai/models) and custom `instructions`:
 

package/dist/docs/references/docs-memory-working-memory.md

@@ -170,11 +170,11 @@ const memory = new Memory({
 
 ## Designing Effective Templates
 
-A well-structured template keeps the information easy for the agent to parse and update. Treat the template as a short form that you want the assistant to keep up to date.
+A well-structured template keeps the information straightforward for the agent to parse and update. Treat the template as a short form that you want the assistant to keep up to date.
 
-- **Short, focused labels.** Avoid paragraphs or very long headings. Keep labels brief (for example `## Personal Info` or `- Name:`) so updates are easy to read and less likely to be truncated.
+- **Short, focused labels.** Avoid paragraphs or very long headings. Keep labels brief (for example `## Personal Info` or `- Name:`) so updates are readable and less likely to be truncated.
 - **Use consistent casing.** Inconsistent capitalization (`Timezone:` vs `timezone:`) can cause messy updates. Stick to Title Case or lower case for headings and bullet labels.
-- **Keep placeholder text simple.** Use hints such as `[e.g., Formal]` or `[Date]` to help the LLM fill in the correct spots.
+- **Keep placeholder text minimal.** Use hints such as `[e.g., Formal]` or `[Date]` to help the LLM fill in the correct spots.
 - **Abbreviate very long values.** If you only need a short form, include guidance like `- Name: [First name or nickname]` or `- Address (short):` rather than the full legal text.
 - **Mention update rules in `instructions`.** You can instruct how and when to fill or clear parts of the template directly in the agent's `instructions` field.
 
@@ -263,13 +263,13 @@ Schema-based working memory uses **merge semantics**, meaning the agent only nee
 
 - **Object fields are deep merged:** Only provided fields are updated; others remain unchanged
 - **Set a field to `null` to delete it:** This explicitly removes the field from memory
-- **Arrays are replaced entirely:** When an array field is provided, it replaces the existing array (arrays are not merged element-by-element)
+- **Arrays are replaced entirely:** When an array field is provided, it replaces the existing array (arrays aren't merged element-by-element)
 
 ## Choosing Between Template and Schema
 
 - Use a **template** (Markdown) if you want the agent to maintain memory as a free-form text block, such as a user profile or scratchpad. Templates use **replace semantics** — the agent must provide the complete memory content on each update.
 - Use a **schema** if you need structured, type-safe data that can be validated and programmatically accessed as JSON. Schemas use **merge semantics** — the agent only provides fields to update, and existing fields are preserved.
-- Only one mode can be active at a time: setting both `template` and `schema` is not supported.
+- Only one mode can be active at a time: setting both `template` and `schema` isn't supported.
 
 ## Example: Multi-step Retention
 
@@ -301,7 +301,7 @@ Below is a simplified view of how the `User Profile` template updates across a s
 
 The agent can now refer to `Sam` or `Berlin` in later responses without requesting the information again because it has been stored in working memory.
 
-If your agent is not properly updating working memory when you expect it to, you can add system instructions on _how_ and _when_ to use this template in your agent's `instructions` setting.
+If your agent isn't properly updating working memory when you expect it to, you can add system instructions on _how_ and _when_ to use this template in your agent's `instructions` setting.
 
 ## Setting Initial Working Memory
 

package/dist/docs/references/docs-rag-vector-databases.md

@@ -359,7 +359,7 @@ The dimension size must match the output dimension of your chosen embedding mode
 - Cohere embed-multilingual-v3: 1024 dimensions
 - Google gemini-embedding-001: 768 dimensions (or custom)
 
-> **Warning:** Index dimensions cannot be changed after creation. To use a different model, delete and recreate the index with the new dimension size.
+> **Warning:** Index dimensions can't be changed after creation. To use a different model, delete and recreate the index with the new dimension size.
 
 ### Naming Rules for Databases
 
@@ -583,7 +583,7 @@ Key metadata considerations:
 
 ## Deleting Vectors
 
-When building RAG applications, you often need to clean up stale vectors when documents are deleted or updated. Mastra provides the `deleteVectors` method that supports deleting vectors by metadata filters, making it easy to remove all embeddings associated with a specific document.
+When building RAG applications, you often need to clean up stale vectors when documents are deleted or updated. Mastra provides the `deleteVectors` method that supports deleting vectors by metadata filters, making it straightforward to remove all embeddings associated with a specific document.
 
 ### Delete by Metadata Filter
 

package/dist/docs/references/reference-processors-message-history-processor.md

@@ -66,7 +66,7 @@ export const agent = new Agent({
 
 1. Retrieves `threadId` from the request context
 2. Fetches historical messages from storage (ordered by creation date, descending)
-3. Filters out system messages (they should not be stored in the database)
+3. Filters out system messages (they shouldn't be stored in the database)
 4. Merges historical messages with incoming messages, avoiding duplicates by ID
 5. Adds historical messages with `source: 'memory'` tag
 

package/dist/docs/references/reference-rag-metadata-filters.md

@@ -48,7 +48,7 @@ const results = await store.query({
 
 ## Common Rules and Restrictions
 
-1. Field names cannot:
+1. Field names can't:
 
    - Contain dots (.) unless referring to nested fields
    - Start with $ or contain null characters
@@ -63,11 +63,11 @@ const results = await store.query({
 3. Logical operators:
 
    - Must contain valid conditions
-   - Cannot be empty
+   - Can't be empty
    - Must be properly nested
    - Can only be used at top level or nested within other logical operators
-   - Cannot be used at field level or nested inside a field
-   - Cannot be used inside an operator
+   - Can't be used at field level or nested inside a field
+   - Can't be used inside an operator
    - Valid: `{ "$and": [{ "field": { "$gt": 100 } }] }`
    - Valid: `{ "$or": [{ "$and": [{ "field": { "$gt": 100 } }] }] }`
    - Invalid: `{ "field": { "$and": [{ "$gt": 100 }] } }`
@@ -76,7 +76,7 @@ const results = await store.query({
 4. $not operator:
 
    - Must be an object
-   - Cannot be empty
+   - Can't be empty
    - Can be used at field level or top level
    - Valid: `{ "$not": { "field": "value" } }`
    - Valid: `{ "field": { "$not": { "$eq": "value" } } }`
@@ -98,7 +98,7 @@ const results = await store.query({
 ### ChromaDB
 
 - Where filters only return results where the filtered field exists in metadata
-- Empty metadata fields are not included in filter results
+- Empty metadata fields aren't included in filter results
 - Metadata fields must be present for negative matches (e.g., $ne won't match documents missing the field)
 
 ### Cloudflare Vectorize
@@ -109,7 +109,7 @@ const results = await store.query({
 - String values are indexed up to first 64 bytes (truncated on UTF-8 boundaries)
 - Number values use float64 precision
 - Filter JSON must be under 2048 bytes
-- Field names cannot contain dots (.) or start with $
+- Field names can't contain dots (.) or start with $
 - Field names limited to 512 characters
 - Vectors must be re-upserted after creating new metadata indexes to be included in filtered results
 - Range queries may have reduced accuracy with very large datasets (\~10M+ vectors)
@@ -186,12 +186,12 @@ const results = await store.query({
 
 ### Couchbase
 
-- Currently does not have support for metadata filters. Filtering must be done client-side after retrieving results or by using the Couchbase SDK's Search capabilities directly for more complex queries.
+- Currently doesn't have support for metadata filters. Filtering must be done client-side after retrieving results or by using the Couchbase SDK's Search capabilities directly for more complex queries.
 
 ### Amazon S3 Vectors
 
-- Equality values must be primitives (string/number/boolean). `null`/`undefined`, arrays, objects, and Date are not allowed for equality. Range operators accept numbers or Date (Dates are normalized to epoch ms).
-- `$in`/`$nin` require **non-empty arrays of primitives**; Date elements are allowed and normalized to epoch ms. **Array equality** is not supported.
+- Equality values must be primitives (string/number/boolean). `null`/`undefined`, arrays, objects, and Date aren't allowed for equality. Range operators accept numbers or Date (Dates are normalized to epoch ms).
+- `$in`/`$nin` require **non-empty arrays of primitives**; Date elements are allowed and normalized to epoch ms. **Array equality** isn't supported.
 - Implicit AND is canonicalized (`{a:1,b:2}` → `{$and:[{a:1},{b:2}]}`). Logical operators must contain field conditions, use non-empty arrays, and appear only at the root or within other logical operators (not inside field values).
 - Keys listed in `nonFilterableMetadataKeys` at index creation are stored but not filterable; this setting is immutable.
 - $exists requires a boolean value.

package/dist/docs/references/reference-storage-dynamodb.md

@@ -2,7 +2,7 @@
 
 The DynamoDB storage implementation provides a scalable and performant NoSQL database solution for Mastra, leveraging a single-table design pattern with [ElectroDB](https://electrodb.dev/).
 
-> **Observability Not Supported:** DynamoDB storage **does not support the observability domain**. Traces from the `DefaultExporter` cannot be persisted to DynamoDB, and Mastra Studio's observability features won't work with DynamoDB as your only storage provider. To enable observability, use [composite storage](https://mastra.ai/reference/storage/composite) to route observability data to a supported provider like ClickHouse or PostgreSQL.
+> **Observability Not Supported:** DynamoDB storage **doesn't support the observability domain**. Traces from the `DefaultExporter` can't be persisted to DynamoDB, and Mastra Studio's observability features won't work with DynamoDB as your only storage provider. To enable observability, use [composite storage](https://mastra.ai/reference/storage/composite) to route observability data to a supported provider like ClickHouse or PostgreSQL.
 
 > **Item Size Limit:** DynamoDB enforces a **400 KB maximum item size**. This limit can be exceeded when storing messages with base64-encoded attachments such as images. See [Handling large attachments](https://mastra.ai/docs/memory/storage) for workarounds including uploading attachments to external storage.
 
@@ -211,7 +211,7 @@ aws dynamodb update-time-to-live \
 1. Go to the DynamoDB console
 2. Select your table
 3. Go to "Additional settings" tab
-4. Under "Time to Live (TTL)", click "Manage TTL"
+4. Under "Time to Live (TTL)", select "Manage TTL"
 5. Enable TTL and specify the attribute name (default: `ttl`)
 
 > **Note:** DynamoDB deletes expired items within 48 hours after expiration. Items remain queryable until actually deleted.

package/dist/docs/references/reference-storage-postgresql.md

@@ -177,7 +177,7 @@ const memoryStore = await storage.getStore('memory')
 const thread = await memoryStore?.getThreadById({ threadId: '...' })
 ```
 
-> **Warning:** If `init()` is not called, tables won't be created and storage operations will fail silently or throw errors.
+> **Warning:** If `init()` isn't called, tables won't be created and storage operations will fail silently or throw errors.
 
 ### Using an Existing Pool
 
@@ -201,7 +201,7 @@ const storage = new PostgresStore({
 
 **Pool lifecycle behavior:**
 
-- When you **provide a pool**: Mastra uses your pool but does **not** close it when `store.close()` is called. You manage the pool lifecycle.
+- When you **provide a pool**: Mastra uses your pool but **doesn't** close it when `store.close()` is called. You manage the pool lifecycle.
 - When Mastra **creates a pool**: Mastra owns the pool and will close it when `store.close()` is called.
 
 ### Direct Database and Pool Access

package/dist/docs/references/reference-vectors-pg.md

@@ -82,6 +82,8 @@ const vectorStore = new PgVector({
 
 **buildIndex** (`boolean`): Whether to build the index (Default: `true`)
 
+**metadataIndexes** (`string[]`): Array of metadata field names to create btree indexes on. Improves query performance when filtering by these metadata fields.
+
 #### IndexConfig
 
 **type** (`'flat' | 'hnsw' | 'ivfflat'`): Index type (Default: `ivfflat`)