@mastra/pg 1.22.0-alpha.3 → 1.22.0

package/CHANGELOG.md

@@ -1,5 +1,121 @@
 # @mastra/pg
 
+## 1.22.0
+
+### Minor Changes
+
+- **In-progress traces now appear in Studio with `PostgresStoreVNext`** ([#22137](https://github.com/mastra-ai/mastra/pull/22137))
+
+  `PostgresStoreVNext` previously persisted a span only after it finished, so a long agent run stayed invisible until it completed. It now uses the `event-sourced` tracing strategy: one row is written when a span starts and another when it ends, and reads collapse those rows back into a single span. Traces show up in Studio while the run is executing, and filtering by `running` status works.
+
+  This also fixes duplicate traces from durable runs. A run that suspends and resumes opens a second root span on the same trace, which used to render as two separate entries; the trace list now shows the current root only.
+
+  Writes stay append-only, so throughput is unchanged. The span table gains an `isPending` column, added automatically on `init()` — no manual migration needed. Closes #22054.
+
+- Added native application collection counts so totals no longer load matching rows. ([#22021](https://github.com/mastra-ai/mastra/pull/22021))
+
+  **Before**
+
+  ```ts
+  const total = (await storage.ops.findMany('jobs', { status: 'failed' })).length;
+  ```
+
+  **After**
+
+  ```ts
+  const total = await storage.ops.count?.('jobs', { status: 'failed' });
+  ```
+
+- Added namespace isolation to PgVector operations so applications can safely reuse vector indexes across tenants. Existing vectors remain available in the default namespace. ([#22149](https://github.com/mastra-ai/mastra/pull/22149))
+
+  ```ts
+  await pgVector.upsert({
+    indexName: 'documents',
+    vectors,
+    ids,
+    namespace: 'tenant-123',
+  });
+
+  const results = await pgVector.query({
+    indexName: 'documents',
+    queryVector,
+    namespace: 'tenant-123',
+  });
+  ```
+
+- Added support for filtering scores by metadata key-value pairs in listScores. ([#22047](https://github.com/mastra-ai/mastra/pull/22047))
+
+  ```typescript
+  const result = await storage.listScores({
+    filters: { metadata: { env: 'prod' } },
+  });
+  ```
+
+  Each top-level metadata key is matched with exact equality against the stored value. Nested objects and arrays compare structurally (key order doesn't matter) with no partial/subset matching, and an empty metadata filter is a no-op.
+
+### Patch Changes
+
+- Workflow snapshot upserts no longer overwrite a previously stored `resourceId` with NULL when a run is re-persisted without one (for example during resume). ([#22105](https://github.com/mastra-ai/mastra/pull/22105))
+
+- Stop observability filter discovery from re-scanning all history on every refresh ([#22136](https://github.com/mastra-ai/mastra/pull/22136))
+
+  Discovery queries that build Studio's Traces, Logs, and Metrics filter suggestions scanned every span, metric, and log event each time the cache went stale, which grew unbounded with retained data. Refreshes are now bounded to the last 30 days by default (configurable via `observability.discovery.lookbackSeconds`, `0` restores the previous unbounded behaviour), so the planner can prune partitions instead of reading all of them. Only one process refreshes a given cache entry at a time, so running several server instances no longer multiplies the work, and Studio holds discovery results for five minutes instead of refetching on every page mount.
+
+- Fix PostgreSQL observability writes failing on NUL characters and unpaired Unicode surrogates ([#21728](https://github.com/mastra-ai/mastra/pull/21728))
+
+  Span serialization truncated strings by UTF-16 code unit, so a cut inside an emoji left a lone surrogate that PostgreSQL rejected on the jsonb cast (`22P02`). NUL characters were rejected as well (`22P05`). Because observability events are inserted as a single multi-row statement, one malformed field discarded the entire batch.
+
+  Truncation now preserves complete surrogate pairs, and the v-next PostgreSQL observability encoder sanitizes NUL and unpaired surrogates before the jsonb cast, using the same sanitizer that workflow snapshots already rely on. Valid Unicode, including complete emoji, is preserved.
+
+- Experiment results now include isolated metadata snapshots from the dataset items that ran. ([#22005](https://github.com/mastra-ai/mastra/pull/22005))
+
+- Fix `PgFactoryStorage` reads of `json` columns holding a JSON value that isn't an object. ([#22258](https://github.com/mastra-ai/mastra/pull/22258))
+
+  node-pg parses JSONB through its own type parsers, so the value reaching row deserialization is already a JS value. Deserialization parsed it a second time when it was a string, which threw and failed the whole read. Objects and arrays came back as JS objects and never hit that branch, so the defect stayed hidden until a caller stored a string.
+
+  This surfaced through Factory credential encryption, which stores secrets as an opaque envelope string: saving or reading any credential on Postgres threw `Unexpected token ... is not valid JSON`, affecting model provider credentials, integrations, and custom providers. libsql was unaffected, since it stores `json` columns as text where parsing on read is correct.
+
+- Fixed `PgVector` scanning every vector table on startup. Constructing a `PgVector` warms an index cache in the background, and that warmup asked for full index statistics, which include `SELECT COUNT(*)` per table. On a large index that is a full table scan per index, per process start, and the warmup never used the count it paid for. `query()`, `upsert()`, `updateVector()` and the "has this index changed?" check in `createIndex()` paid for the same count. ([#22180](https://github.com/mastra-ai/mastra/pull/22180))
+
+  These paths now read only the index metadata they use (dimension, metric, index type, vector type, index configuration), all of which comes from the Postgres catalog at a cost that does not grow with the size of the table.
+
+  `describeIndex()` is unchanged and still returns an exact `count`:
+
+  ```ts
+  const stats = await pgVector.describeIndex({ indexName: 'embeddings' });
+  console.log(stats.count); // exact row count, as before
+  ```
+
+  Concurrent callers on a cold cache also no longer duplicate the lookup: the first call is shared with everyone waiting on it, and a failed lookup is not cached.
+
+  Fixes [#21952](https://github.com/mastra-ai/mastra/issues/21952).
+
+- Enforce atomic conditional background task state updates so cancellation cannot be overwritten during dispatch. Background task storage is no longer exposed by Cloudflare KV or ClickHouse, which cannot provide the required compare-and-set semantics. ([#22228](https://github.com/mastra-ai/mastra/pull/22228))
+
+- Updated dependencies [[`79f04a7`](https://github.com/mastra-ai/mastra/commit/79f04a7f6c6829da541139f638f2f1d267916e08), [`65edab1`](https://github.com/mastra-ai/mastra/commit/65edab1c233d17b8f163bad12fca410d0e6f16b1), [`1e47b75`](https://github.com/mastra-ai/mastra/commit/1e47b7520cab4cfaa8daed52f17e2e6d14ff7539), [`ab20a38`](https://github.com/mastra-ai/mastra/commit/ab20a38d0275f8d85e0f3833bd87ef487bcc609f), [`fd4d5fe`](https://github.com/mastra-ai/mastra/commit/fd4d5fe4f943699b85db5e74404f190d5a6b8c2a), [`ae8790c`](https://github.com/mastra-ai/mastra/commit/ae8790c4bfaa088d2ab279d1dcc06f326b9fd109), [`2c85f42`](https://github.com/mastra-ai/mastra/commit/2c85f428e04ccd63ea31a7ec80b5b327afdad555), [`11bbeb9`](https://github.com/mastra-ai/mastra/commit/11bbeb9b108ef2264e05acefc6dafb9cbb342921), [`48ef1f1`](https://github.com/mastra-ai/mastra/commit/48ef1f1d24eedafbb07f64e659a81b52b67b8bf6), [`aa3a85d`](https://github.com/mastra-ai/mastra/commit/aa3a85daf094c683bb97efdf4b6a696d2e474af5), [`d29d06f`](https://github.com/mastra-ai/mastra/commit/d29d06fe00bbd35b4571150ea04c59d2ed783c71), [`e6516df`](https://github.com/mastra-ai/mastra/commit/e6516dfcdae4f4ac0e7971d84359a81385ee602f), [`1a485f3`](https://github.com/mastra-ai/mastra/commit/1a485f3538f5ec64d58bd8b5e1e99de0c695c87b), [`0d37487`](https://github.com/mastra-ai/mastra/commit/0d37487d9f349388a3f1cef6a536cf9dcc4b6273), [`8661d7d`](https://github.com/mastra-ai/mastra/commit/8661d7d7179f0a024456aabdd8679bcecd09ac28), [`dbbfeb8`](https://github.com/mastra-ai/mastra/commit/dbbfeb85ec949dc9ebc0755e1ad262e4f5eba8db), [`575e343`](https://github.com/mastra-ai/mastra/commit/575e343900451021d96110916497d334af7bc252), [`0b2a3d1`](https://github.com/mastra-ai/mastra/commit/0b2a3d1783875c5b97b7b36ab3d03d7360e0dde7), [`6bb5d71`](https://github.com/mastra-ai/mastra/commit/6bb5d7193fe9166b219f0fccae17db7a5ae86e65), [`3cc9d00`](https://github.com/mastra-ai/mastra/commit/3cc9d00b2b4333e0377a5e9df5eff92c17ce7630), [`cacb839`](https://github.com/mastra-ai/mastra/commit/cacb8392d9e74189b56d857290b0615f98a2683d), [`57de7d6`](https://github.com/mastra-ai/mastra/commit/57de7d644ba7146edb4e9e6111ec4fa98c3a59e9), [`c8e4cea`](https://github.com/mastra-ai/mastra/commit/c8e4ceac9a390d78c8327dff3cdb2861dd71957f), [`ed01e9a`](https://github.com/mastra-ai/mastra/commit/ed01e9a807514a904374bf687a7b8f18750f6f78), [`b47b26e`](https://github.com/mastra-ai/mastra/commit/b47b26e6fe95cb8a3482be2c5e52de157fe59d0b), [`0d37487`](https://github.com/mastra-ai/mastra/commit/0d37487d9f349388a3f1cef6a536cf9dcc4b6273), [`733a537`](https://github.com/mastra-ai/mastra/commit/733a537489a858b5880b2e98809334fba895a221), [`e8e299c`](https://github.com/mastra-ai/mastra/commit/e8e299cc6abdfc39947e2fec25803493015d3882), [`edfc548`](https://github.com/mastra-ai/mastra/commit/edfc548886bc7bae17b681f8b6b41a47eb32bcd2), [`b05f486`](https://github.com/mastra-ai/mastra/commit/b05f48612984d5fe2447ea2d6cdd5c604d285b97), [`a8a4871`](https://github.com/mastra-ai/mastra/commit/a8a4871215f51da95c47129602157ce5372f634a), [`eb9ecaa`](https://github.com/mastra-ai/mastra/commit/eb9ecaa89c36e889749e3b825cfc507ce7f7980b), [`4ff3ee2`](https://github.com/mastra-ai/mastra/commit/4ff3ee2bff7ed07528b4817f8f49639031c72a4d), [`9207dfa`](https://github.com/mastra-ai/mastra/commit/9207dfab8062e5fc68b751684797ff86fe0b4e70), [`5165cdc`](https://github.com/mastra-ai/mastra/commit/5165cdcdcf50e144bb8113278535196cc9b07065), [`e737014`](https://github.com/mastra-ai/mastra/commit/e737014e0fc7035759762bb5b48baef1d6c0f6a7), [`6bb5d71`](https://github.com/mastra-ai/mastra/commit/6bb5d7193fe9166b219f0fccae17db7a5ae86e65), [`f591643`](https://github.com/mastra-ai/mastra/commit/f591643becdf0be9bddce6ba1748e64bc30d77f1), [`63796ba`](https://github.com/mastra-ai/mastra/commit/63796ba0fda60253be17535e68f6bbbf1e6ffa09), [`b1ad324`](https://github.com/mastra-ai/mastra/commit/b1ad324d657f3544b0701332aef7eb10e9a36258), [`61c566d`](https://github.com/mastra-ai/mastra/commit/61c566dd2f2cde2b23ed8f139924e530d4202214), [`c24754c`](https://github.com/mastra-ai/mastra/commit/c24754c1fb6fe144e5051e536e98c8a18b0214ac), [`12c61d2`](https://github.com/mastra-ai/mastra/commit/12c61d280c8cb208bc3c8dbcbe5dcc60cf9d1cd0), [`c46eb09`](https://github.com/mastra-ai/mastra/commit/c46eb09ce4987509af57a0ac582c61241a6dd2f1), [`9ee8120`](https://github.com/mastra-ai/mastra/commit/9ee8120ce17f76b9f617489e05a283353742690a), [`d975e92`](https://github.com/mastra-ai/mastra/commit/d975e924d4936f46c386bd3dee39c671720289f6), [`45dd6ee`](https://github.com/mastra-ai/mastra/commit/45dd6ee089bd7df0d0c98a10098e483fd388e04a), [`4e9a228`](https://github.com/mastra-ai/mastra/commit/4e9a2283d5fd6ed1b70a2751eb3dc2cbf82ada20), [`d6ce34a`](https://github.com/mastra-ai/mastra/commit/d6ce34aeceb06ddf3d595a1eed5cc74f481a46a1), [`f95f468`](https://github.com/mastra-ai/mastra/commit/f95f468cf1e7c2b924a13826494f98b8f2ccd581), [`30ed33e`](https://github.com/mastra-ai/mastra/commit/30ed33ee14084a26019aba15fceadda6d6ddefaf), [`04a815f`](https://github.com/mastra-ai/mastra/commit/04a815fc8971d29e97fcdcc5008a1eb472fc00ff), [`1cfa878`](https://github.com/mastra-ai/mastra/commit/1cfa8784d8da0dfaa0317e5048bc48b6084a5ea5), [`9a12ef3`](https://github.com/mastra-ai/mastra/commit/9a12ef3fccf3f4186db0f294f4ee1f02cf4d8db2), [`32d3583`](https://github.com/mastra-ai/mastra/commit/32d358332cb8ac2306b83b73cf3536e74dbd435e), [`7960688`](https://github.com/mastra-ai/mastra/commit/7960688828e04eaf3106e34f7758fa580257eef6), [`91ad69d`](https://github.com/mastra-ai/mastra/commit/91ad69d64994c89199b0c55399e64ed91c61df2f), [`8dc408d`](https://github.com/mastra-ai/mastra/commit/8dc408d34438f9e13297f792c11a5cfd6cf952e1), [`c92def1`](https://github.com/mastra-ai/mastra/commit/c92def10a13c822972c96f0a4ca6ffc1f4258aed), [`63041eb`](https://github.com/mastra-ai/mastra/commit/63041eb4c50b520a0a80e03d4cd6ea99f67715a0), [`c118318`](https://github.com/mastra-ai/mastra/commit/c1183181c9804303db4b511c2e2648f8b714712b), [`c5eaec5`](https://github.com/mastra-ai/mastra/commit/c5eaec5a860d80d0e3805e67db0414b87ac8cbed), [`fc07c64`](https://github.com/mastra-ai/mastra/commit/fc07c6465043e08e99193a6751a01c56ffc2e7a1), [`cced745`](https://github.com/mastra-ai/mastra/commit/cced745a056ec2225c5bc702e32d848847aa8b65), [`542dee2`](https://github.com/mastra-ai/mastra/commit/542dee254167f974ff8cbbbfc0ce10f9a2616a7b), [`3c19dce`](https://github.com/mastra-ai/mastra/commit/3c19dcef8e73062a80627a4927eae3ec11145afd), [`aca2869`](https://github.com/mastra-ai/mastra/commit/aca2869b2031982f3c4a2f52525c9be7cf123ef8), [`a58483c`](https://github.com/mastra-ai/mastra/commit/a58483cff1a9d41fce7c931843f48cb0ac450f64), [`a58483c`](https://github.com/mastra-ai/mastra/commit/a58483cff1a9d41fce7c931843f48cb0ac450f64), [`e6f8450`](https://github.com/mastra-ai/mastra/commit/e6f845074d478527026b18d85031b23353e1d0a4), [`895e9df`](https://github.com/mastra-ai/mastra/commit/895e9dfc17d6f34299eca64e317ded9e5f5e5ef8), [`e66b2ba`](https://github.com/mastra-ai/mastra/commit/e66b2ba100db63eaeab6e21e1ea34b113f2ec781), [`3e8727e`](https://github.com/mastra-ai/mastra/commit/3e8727e11ec1a5d733acedb5c872896394be18c1)]:
+  - @mastra/core@1.62.0
+
+## 1.22.0-alpha.4
+
+### Patch Changes
+
+- Stop observability filter discovery from re-scanning all history on every refresh ([#22136](https://github.com/mastra-ai/mastra/pull/22136))
+
+  Discovery queries that build Studio's Traces, Logs, and Metrics filter suggestions scanned every span, metric, and log event each time the cache went stale, which grew unbounded with retained data. Refreshes are now bounded to the last 30 days by default (configurable via `observability.discovery.lookbackSeconds`, `0` restores the previous unbounded behaviour), so the planner can prune partitions instead of reading all of them. Only one process refreshes a given cache entry at a time, so running several server instances no longer multiplies the work, and Studio holds discovery results for five minutes instead of refetching on every page mount.
+
+- Experiment results now include isolated metadata snapshots from the dataset items that ran. ([#22005](https://github.com/mastra-ai/mastra/pull/22005))
+
+- Fix `PgFactoryStorage` reads of `json` columns holding a JSON value that isn't an object. ([#22258](https://github.com/mastra-ai/mastra/pull/22258))
+
+  node-pg parses JSONB through its own type parsers, so the value reaching row deserialization is already a JS value. Deserialization parsed it a second time when it was a string, which threw and failed the whole read. Objects and arrays came back as JS objects and never hit that branch, so the defect stayed hidden until a caller stored a string.
+
+  This surfaced through Factory credential encryption, which stores secrets as an opaque envelope string: saving or reading any credential on Postgres threw `Unexpected token ... is not valid JSON`, affecting model provider credentials, integrations, and custom providers. libsql was unaffected, since it stores `json` columns as text where parsing on read is correct.
+
+- Enforce atomic conditional background task state updates so cancellation cannot be overwritten during dispatch. Background task storage is no longer exposed by Cloudflare KV or ClickHouse, which cannot provide the required compare-and-set semantics. ([#22228](https://github.com/mastra-ai/mastra/pull/22228))
+
+- Updated dependencies [[`aa3a85d`](https://github.com/mastra-ai/mastra/commit/aa3a85daf094c683bb97efdf4b6a696d2e474af5), [`d29d06f`](https://github.com/mastra-ai/mastra/commit/d29d06fe00bbd35b4571150ea04c59d2ed783c71), [`e6516df`](https://github.com/mastra-ai/mastra/commit/e6516dfcdae4f4ac0e7971d84359a81385ee602f), [`0b2a3d1`](https://github.com/mastra-ai/mastra/commit/0b2a3d1783875c5b97b7b36ab3d03d7360e0dde7), [`6bb5d71`](https://github.com/mastra-ai/mastra/commit/6bb5d7193fe9166b219f0fccae17db7a5ae86e65), [`57de7d6`](https://github.com/mastra-ai/mastra/commit/57de7d644ba7146edb4e9e6111ec4fa98c3a59e9), [`e8e299c`](https://github.com/mastra-ai/mastra/commit/e8e299cc6abdfc39947e2fec25803493015d3882), [`edfc548`](https://github.com/mastra-ai/mastra/commit/edfc548886bc7bae17b681f8b6b41a47eb32bcd2), [`a8a4871`](https://github.com/mastra-ai/mastra/commit/a8a4871215f51da95c47129602157ce5372f634a), [`5165cdc`](https://github.com/mastra-ai/mastra/commit/5165cdcdcf50e144bb8113278535196cc9b07065), [`6bb5d71`](https://github.com/mastra-ai/mastra/commit/6bb5d7193fe9166b219f0fccae17db7a5ae86e65), [`9ee8120`](https://github.com/mastra-ai/mastra/commit/9ee8120ce17f76b9f617489e05a283353742690a), [`d975e92`](https://github.com/mastra-ai/mastra/commit/d975e924d4936f46c386bd3dee39c671720289f6), [`1cfa878`](https://github.com/mastra-ai/mastra/commit/1cfa8784d8da0dfaa0317e5048bc48b6084a5ea5), [`c118318`](https://github.com/mastra-ai/mastra/commit/c1183181c9804303db4b511c2e2648f8b714712b), [`fc07c64`](https://github.com/mastra-ai/mastra/commit/fc07c6465043e08e99193a6751a01c56ffc2e7a1), [`542dee2`](https://github.com/mastra-ai/mastra/commit/542dee254167f974ff8cbbbfc0ce10f9a2616a7b), [`a58483c`](https://github.com/mastra-ai/mastra/commit/a58483cff1a9d41fce7c931843f48cb0ac450f64), [`a58483c`](https://github.com/mastra-ai/mastra/commit/a58483cff1a9d41fce7c931843f48cb0ac450f64), [`895e9df`](https://github.com/mastra-ai/mastra/commit/895e9dfc17d6f34299eca64e317ded9e5f5e5ef8)]:
+  - @mastra/core@1.62.0-alpha.8
+
 ## 1.22.0-alpha.3
 
 ### Minor Changes

package/dist/docs/SKILL.md

@@ -3,7 +3,7 @@ name: mastra-pg
 description: Documentation for @mastra/pg. Use when working with @mastra/pg APIs, configuration, or implementation.
 metadata:
   package: "@mastra/pg"
-  version: "1.22.0-alpha.3"
+  version: "1.22.0"
 ---
 
 ## When to use

package/dist/docs/assets/SOURCE_MAP.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.22.0-alpha.3",
+  "version": "1.22.0",
   "package": "@mastra/pg",
   "exports": {},
   "modules": {}

package/dist/docs/references/docs-memory-semantic-recall.md

@@ -350,6 +350,25 @@ const agent = new Agent({
 })
 ```
 
+FastEmbed also exposes the multilingual E5 model for non-English content. E5 is asymmetric, so it's exposed as two models: `multilingualE5LargePassage` for text you index and `multilingualE5LargeQuery` for search text.
+
+```ts
+import { Memory } from '@mastra/memory'
+import { Agent } from '@mastra/core/agent'
+import { fastembed } from '@mastra/fastembed'
+
+const agent = new Agent({
+  id: 'agent',
+  memory: new Memory({
+    embedder: fastembed.multilingualE5LargePassage,
+  }),
+})
+```
+
+Memory uses a single embedder for both storing and recalling messages, so pick one of the two models and use it consistently. Use the paired `multilingualE5LargeQuery` model only where you control both sides of the pipeline, such as a RAG workflow that indexes with the passage model and searches with the query model.
+
+Multilingual E5 produces 1024-dimensional vectors. Your vector index must be created with matching dimensions, and E5 vectors can't be mixed with vectors from another embedder in the same index.
+
 ## PostgreSQL index optimization
 
 When using PostgreSQL as your vector store, you can optimize semantic recall performance by configuring the vector index. This is particularly important for large-scale deployments with thousands of messages.

package/dist/docs/references/docs-storage.md

@@ -197,6 +197,7 @@ Each provider page includes installation instructions, configuration parameters,
 - [Convex](https://mastra.ai/integrations/databases/convex)
 - [DuckDB](https://mastra.ai/integrations/databases/duckdb)
 - [DynamoDB](https://mastra.ai/integrations/databases/dynamodb)
+- [Elasticsearch](https://mastra.ai/integrations/databases/elasticsearch)
 - [Google Cloud Spanner](https://mastra.ai/integrations/databases/spanner)
 - [LanceDB](https://mastra.ai/integrations/databases/lancedb)
 - [libSQL](https://mastra.ai/integrations/databases/libsql)

package/dist/docs/references/integrations-databases-postgresql.md

@@ -148,6 +148,30 @@ PostgreSQL supports observability and can handle low trace volumes. Throughput c
 
 `PostgresStoreVNext` uses the `event-sourced` [tracing strategy](https://mastra.ai/docs/observability/integrations/exporters/mastra-storage) instead. It writes one row when a span starts and another when it ends, never updating a row in place, and collapses those rows when a trace is read. Writes stay append-only, and traces appear in Studio while the run is still executing.
 
+#### Filter suggestions
+
+Studio's Traces, Logs, and Metrics pages offer filter values (tags, service names, environments, entity names, metric names and labels) discovered from your observability data. Those values are cached in the database and recomputed in the background when the cache goes stale.
+
+To keep the refresh cheap on large datasets, it only scans events from the last 30 days. Values that appear exclusively in older events won't be suggested, though filtering by them still works. Use `discovery` to change the window or how often it refreshes:
+
+```typescript
+import { PostgresStoreVNext } from '@mastra/pg'
+
+const storage = new PostgresStoreVNext({
+  id: 'pg-storage',
+  connectionString: process.env.DATABASE_URL,
+  observability: {
+    connectionString: process.env.OBSERVABILITY_DATABASE_URL,
+    discovery: {
+      lookbackSeconds: 7 * 24 * 60 * 60, // scan the last 7 days; 0 scans all history
+      ttlSeconds: 15 * 60, // refresh at most every 15 minutes
+    },
+  },
+})
+```
+
+Lower `lookbackSeconds` if refreshes are slow, and raise `ttlSeconds` if they run more often than your filter values change. Only one process refreshes a given cache entry at a time, so adding server instances doesn't multiply the work.
+
 ### Initialization
 
 When you pass storage to the Mastra class, `init()` is called automatically before any storage operation:

package/dist/index.cjs

@@ -5045,7 +5045,7 @@ var BackgroundTasksPG = class BackgroundTasksPG extends _mastra_core_storage.Bac
 			}
 		});
 	}
-	async updateTask(taskId, update) {
+	async updateTask(taskId, update, options) {
 		const setClauses = [];
 		const params = [];
 		let paramIdx = 1;
@@ -5087,10 +5087,15 @@ var BackgroundTasksPG = class BackgroundTasksPG extends _mastra_core_storage.Bac
 			const val = update.completedAt?.toISOString() ?? null;
 			params.push(val, val);
 		}
-		if (setClauses.length === 0) return;
+		if (setClauses.length === 0) return false;
 		const table = getTableName$4(getSchemaName$4(this.#schema));
 		params.push(taskId);
-		await this.#db.client.none(`UPDATE ${table} SET ${setClauses.join(", ")} WHERE "id" = $${paramIdx}`, params);
+		let where = `"id" = $${paramIdx++}`;
+		if (options?.expectedStatus) {
+			where += ` AND "status" = $${paramIdx}`;
+			params.push(options.expectedStatus);
+		}
+		return ((await this.#db.client.query(`UPDATE ${table} SET ${setClauses.join(", ")} WHERE ${where}`, params)).rowCount ?? 0) > 0;
 	}
 	async getTask(taskId) {
 		const table = getTableName$4(getSchemaName$4(this.#schema));
@@ -6735,6 +6740,7 @@ var ExperimentsPG = class ExperimentsPG extends _mastra_core_storage.Experiments
 				"tags",
 				"comment",
 				"toolMockReport",
+				"metadata",
 				"organizationId",
 				"projectId",
 				"attempt"
@@ -6931,6 +6937,7 @@ var ExperimentsPG = class ExperimentsPG extends _mastra_core_storage.Experiments
 			input: (0, _mastra_core_storage.safelyParseJSON)(row.input),
 			output: row.output ? (0, _mastra_core_storage.safelyParseJSON)(row.output) : null,
 			groundTruth: row.groundTruth ? (0, _mastra_core_storage.safelyParseJSON)(row.groundTruth) : null,
+			metadata: row.metadata ? (0, _mastra_core_storage.safelyParseJSON)(row.metadata) : null,
 			error: row.error ? (0, _mastra_core_storage.safelyParseJSON)(row.error) : null,
 			startedAt: (0, _mastra_core_storage.ensureDate)(row.startedAtZ || row.startedAt),
 			completedAt: (0, _mastra_core_storage.ensureDate)(row.completedAtZ || row.completedAt),
@@ -7239,6 +7246,7 @@ var ExperimentsPG = class ExperimentsPG extends _mastra_core_storage.Experiments
 					input: input.input,
 					output: input.output ?? null,
 					groundTruth: input.groundTruth ?? null,
+					metadata: input.metadata ?? null,
 					error: input.error ?? null,
 					startedAt: input.startedAt.toISOString(),
 					completedAt: input.completedAt.toISOString(),
@@ -7261,6 +7269,7 @@ var ExperimentsPG = class ExperimentsPG extends _mastra_core_storage.Experiments
 				input: input.input,
 				output: input.output ?? null,
 				groundTruth: input.groundTruth ?? null,
+				metadata: input.metadata ?? null,
 				error: input.error ?? null,
 				startedAt: input.startedAt,
 				completedAt: input.completedAt,
@@ -7297,9 +7306,9 @@ var ExperimentsPG = class ExperimentsPG extends _mastra_core_storage.Experiments
 					const id = crypto.randomUUID();
 					return t.one(`INSERT INTO ${tableName} (
               "id", "experimentId", "itemId", "itemDatasetVersion", "organizationId", "projectId",
-              "input", "output", "groundTruth", "error", "startedAt", "completedAt",
+              "input", "output", "groundTruth", "metadata", "error", "startedAt", "completedAt",
               "retryCount", "attempt", "traceId", "status", "tags", "toolMockReport", "createdAt"
-            ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19)
+            ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20)
             RETURNING *`, [
 						id,
 						input.experimentId,
@@ -7310,6 +7319,7 @@ var ExperimentsPG = class ExperimentsPG extends _mastra_core_storage.Experiments
 						JSON.stringify(input.input),
 						input.output != null ? JSON.stringify(input.output) : null,
 						input.groundTruth != null ? JSON.stringify(input.groundTruth) : null,
+						input.metadata != null ? JSON.stringify(input.metadata) : null,
 						input.error != null ? JSON.stringify(input.error) : null,
 						input.startedAt.toISOString(),
 						input.completedAt.toISOString(),
@@ -7324,9 +7334,9 @@ var ExperimentsPG = class ExperimentsPG extends _mastra_core_storage.Experiments
 				}
 				return t.one(`UPDATE ${tableName} SET
             "itemDatasetVersion" = $2, "organizationId" = $3, "projectId" = $4,
-            "input" = $5, "output" = $6, "groundTruth" = $7, "error" = $8,
-            "startedAt" = $9, "completedAt" = $10, "retryCount" = $11, "attempt" = $12,
-            "traceId" = $13, "status" = $14, "tags" = $15, "toolMockReport" = $16
+            "input" = $5, "output" = $6, "groundTruth" = $7, "metadata" = $8, "error" = $9,
+            "startedAt" = $10, "completedAt" = $11, "retryCount" = $12, "attempt" = $13,
+            "traceId" = $14, "status" = $15, "tags" = $16, "toolMockReport" = $17
           WHERE "id" = $1 RETURNING *`, [
 					existing.id,
 					input.itemDatasetVersion ?? null,
@@ -7335,6 +7345,7 @@ var ExperimentsPG = class ExperimentsPG extends _mastra_core_storage.Experiments
 					JSON.stringify(input.input),
 					input.output != null ? JSON.stringify(input.output) : null,
 					input.groundTruth != null ? JSON.stringify(input.groundTruth) : null,
+					input.metadata != null ? JSON.stringify(input.metadata) : null,
 					input.error != null ? JSON.stringify(input.error) : null,
 					input.startedAt.toISOString(),
 					input.completedAt.toISOString(),
@@ -14441,6 +14452,7 @@ function allIndexDDL(schema) {
 //#endregion
 //#region src/storage/domains/observability/v-next/discovery.ts
 const DEFAULT_TTL_SECONDS = 300;
+const DEFAULT_LOOKBACK_SECONDS = 720 * 60 * 60;
 /** All signal tables that contain a column. Used by cross-signal discovery. */
 const SIGNAL_TABLES_WITH_CONTEXT = [
 	TABLE_SPAN_EVENTS,
@@ -14505,22 +14517,71 @@ function startOrJoinRefresh(dedupeKey, cacheKey, refresh, upsert, logger) {
 async function readWithRefresh(client, schema, cacheKey, refresh, ttlSeconds, logger) {
 	const table = qualifiedTable(schema, TABLE_DISCOVERY);
 	const row = await client.oneOrNone(`SELECT "values", "refreshedAt" FROM ${table} WHERE "cacheKey" = $1`, [cacheKey]);
-	const refreshedAtMs = row ? new Date(row.refreshedAt).getTime() : 0;
-	if (!(!row || Date.now() - refreshedAtMs > ttlSeconds * 1e3)) return row.values;
-	const refreshing = startOrJoinRefresh(`${schema}:${cacheKey}`, cacheKey, refresh, (values) => upsertCache(client, schema, cacheKey, values), logger);
+	const dedupeKey = `${schema}:${cacheKey}`;
+	const startRefresh = () => startOrJoinRefresh(dedupeKey, cacheKey, refresh, (values) => upsertCache(client, schema, cacheKey, values), logger);
 	if (ttlSeconds <= 0) try {
-		return await refreshing;
+		return await startRefresh();
 	} catch {
 		return row?.values ?? [];
 	}
+	const refreshedAtMs = row ? new Date(row.refreshedAt).getTime() : 0;
+	if (!(!row || Date.now() - refreshedAtMs > ttlSeconds * 1e3)) return row.values;
 	if (!row) try {
-		return await refreshing;
+		return await startRefresh();
 	} catch {
 		return [];
 	}
-	refreshing.catch(() => {});
+	const previousRefreshedAt = row.refreshedAt;
+	claimRefresh(client, schema, cacheKey, ttlSeconds).then((claimedAt) => {
+		if (!claimedAt) return;
+		return startRefresh().then(() => {}, () => releaseClaim(client, schema, cacheKey, previousRefreshedAt, claimedAt));
+	}).catch(() => {});
 	return row.values;
 }
+/**
+* Try to become the process that refreshes `cacheKey`.
+*
+* The conditional UPDATE re-checks staleness in the database and stamps
+* `refreshedAt` in the same statement, so exactly one caller can transition a
+* given cache row out of the stale window — losers see zero rows updated and
+* skip their refresh. This is the cross-process counterpart to
+* `inFlightRefreshes`: N frontends going stale together run one refresh
+* instead of N.
+*
+* It reuses the cache table rather than an advisory lock because `DbClient`
+* is pool-backed, so a transaction-scoped lock would mean threading explicit
+* transaction handling through the discovery read path.
+*
+* The stamp is provisional in both directions: a completed refresh overwrites
+* it via `upsertCache`, and a failed one restores it via `releaseClaim`.
+*
+* Returns the exact provisional stamp written by the claim (or `null` when
+* the claim was lost) so `releaseClaim` can restore the row only while it
+* still holds this claim's stamp.
+*/
+async function claimRefresh(client, schema, cacheKey, ttlSeconds) {
+	const table = qualifiedTable(schema, TABLE_DISCOVERY);
+	return (await client.oneOrNone(`UPDATE ${table} SET "refreshedAt" = NOW()
+      WHERE "cacheKey" = $1 AND "refreshedAt" <= NOW() - ($2 || ' seconds')::interval
+     RETURNING "refreshedAt"::text AS "refreshedAt"`, [cacheKey, Math.floor(ttlSeconds)]))?.refreshedAt ?? null;
+}
+/**
+* Hand back a claim whose refresh failed by restoring the timestamp the
+* claiming reader observed. Guarded on the row still holding this claim's
+* exact provisional stamp so a refresh that completed in the meantime
+* (another process, or a later claim) isn't dragged backwards into staleness.
+*/
+async function releaseClaim(client, schema, cacheKey, previousRefreshedAt, claimedAt) {
+	const table = qualifiedTable(schema, TABLE_DISCOVERY);
+	try {
+		await client.query(`UPDATE ${table} SET "refreshedAt" = $2
+        WHERE "cacheKey" = $1 AND "refreshedAt" = $3::timestamptz`, [
+			cacheKey,
+			previousRefreshedAt,
+			claimedAt
+		]);
+	} catch {}
+}
 async function upsertCache(client, schema, cacheKey, values) {
 	const table = qualifiedTable(schema, TABLE_DISCOVERY);
 	await client.query(`INSERT INTO ${table} ("cacheKey", "refreshedAt", "values")
@@ -14529,61 +14590,95 @@ async function upsertCache(client, schema, cacheKey, values) {
        "refreshedAt" = EXCLUDED."refreshedAt",
        "values" = EXCLUDED."values"`, [cacheKey, JSON.stringify(values)]);
 }
-async function distinctAcrossTables(client, schema, column, tables, filterSql = "", filterParams = []) {
+/**
+* Time predicate that bounds a discovery refresh to recent events.
+*
+* Each signal table is range-partitioned on its own time column
+* (`SIGNAL_TIME_COLUMN`: `endedAt` for spans, `timestamp` for everything
+* else), so the predicate has to be built per table rather than shared across
+* a UNION. Bounding on that exact column is what lets the planner prune
+* partitions/chunks instead of scanning all history.
+*
+* The interval is interpolated rather than bound to a parameter on purpose:
+* the UNION branches in `distinctAcrossTables` and `getTags` all reference the
+* same positional placeholders, so adding a per-branch parameter would break
+* `$N` numbering. Safety comes from coercing to a finite integer here — the
+* value originates from `DiscoveryConfig`, never from user input.
+*/
+function lookbackPredicate(table, lookbackSeconds) {
+	if (!Number.isFinite(lookbackSeconds) || lookbackSeconds <= 0) return "";
+	const seconds = Math.floor(lookbackSeconds);
+	const timeColumn = SIGNAL_TIME_COLUMN[table];
+	if (!timeColumn) return "";
+	return `AND "${timeColumn}" >= NOW() - INTERVAL '${seconds} seconds'`;
+}
+async function distinctAcrossTables(client, schema, column, tables, lookbackSeconds, filterSql = "", filterParams = []) {
 	const safeColumn = (0, _mastra_core_utils.parseSqlIdentifier)(column, "column name");
-	const unions = tables.map((t) => `SELECT DISTINCT "${safeColumn}" AS v FROM ${qualifiedTable(schema, t)} WHERE "${safeColumn}" IS NOT NULL AND "${safeColumn}" <> '' ${filterSql}`).join(" UNION ");
+	const unions = tables.map((t) => `SELECT DISTINCT "${safeColumn}" AS v FROM ${qualifiedTable(schema, t)} WHERE "${safeColumn}" IS NOT NULL AND "${safeColumn}" <> '' ${filterSql} ${lookbackPredicate(t, lookbackSeconds)}`).join(" UNION ");
 	return (await client.manyOrNone(`SELECT v FROM (${unions}) sub ORDER BY v`, filterParams)).map((r) => r.v);
 }
 async function getEntityTypes(client, schema, _args, config) {
-	return { entityTypes: await readWithRefresh(client, schema, "entity_types", () => distinctAcrossTables(client, schema, "entityType", ENTITY_DISCOVERY_TABLES), config.ttlSeconds ?? DEFAULT_TTL_SECONDS, config.logger) };
+	const ttl = config.ttlSeconds ?? DEFAULT_TTL_SECONDS;
+	const lookback = config.lookbackSeconds ?? DEFAULT_LOOKBACK_SECONDS;
+	return { entityTypes: await readWithRefresh(client, schema, "entity_types", () => distinctAcrossTables(client, schema, "entityType", ENTITY_DISCOVERY_TABLES, lookback), ttl, config.logger) };
 }
 async function getEntityNames(client, schema, args, config) {
 	const ttl = config.ttlSeconds ?? DEFAULT_TTL_SECONDS;
+	const lookback = config.lookbackSeconds ?? DEFAULT_LOOKBACK_SECONDS;
 	const cacheKey = args.entityType ? `entity_names:${args.entityType}` : "entity_names";
 	const filterSql = args.entityType ? `AND "entityType" = $1` : "";
 	const filterParams = args.entityType ? [args.entityType] : [];
-	return { names: await readWithRefresh(client, schema, cacheKey, () => distinctAcrossTables(client, schema, "entityName", ENTITY_DISCOVERY_TABLES, filterSql, filterParams), ttl, config.logger) };
+	return { names: await readWithRefresh(client, schema, cacheKey, () => distinctAcrossTables(client, schema, "entityName", ENTITY_DISCOVERY_TABLES, lookback, filterSql, filterParams), ttl, config.logger) };
 }
 async function getServiceNames(client, schema, _args, config) {
-	return { serviceNames: await readWithRefresh(client, schema, "service_names", () => distinctAcrossTables(client, schema, "serviceName", SIGNAL_TABLES_WITH_CONTEXT), config.ttlSeconds ?? DEFAULT_TTL_SECONDS, config.logger) };
+	const ttl = config.ttlSeconds ?? DEFAULT_TTL_SECONDS;
+	const lookback = config.lookbackSeconds ?? DEFAULT_LOOKBACK_SECONDS;
+	return { serviceNames: await readWithRefresh(client, schema, "service_names", () => distinctAcrossTables(client, schema, "serviceName", SIGNAL_TABLES_WITH_CONTEXT, lookback), ttl, config.logger) };
 }
 async function getEnvironments(client, schema, _args, config) {
-	return { environments: await readWithRefresh(client, schema, "environments", () => distinctAcrossTables(client, schema, "environment", SIGNAL_TABLES_WITH_CONTEXT), config.ttlSeconds ?? DEFAULT_TTL_SECONDS, config.logger) };
+	const ttl = config.ttlSeconds ?? DEFAULT_TTL_SECONDS;
+	const lookback = config.lookbackSeconds ?? DEFAULT_LOOKBACK_SECONDS;
+	return { environments: await readWithRefresh(client, schema, "environments", () => distinctAcrossTables(client, schema, "environment", SIGNAL_TABLES_WITH_CONTEXT, lookback), ttl, config.logger) };
 }
 async function getTags(client, schema, args, config) {
 	const ttl = config.ttlSeconds ?? DEFAULT_TTL_SECONDS;
+	const lookback = config.lookbackSeconds ?? DEFAULT_LOOKBACK_SECONDS;
 	const cacheKey = args.entityType ? `tags:${args.entityType}` : "tags";
 	const refresh = async () => {
 		const filter = args.entityType ? `AND "entityType" = $1` : "";
 		const params = args.entityType ? [args.entityType] : [];
-		const unions = ENTITY_DISCOVERY_TABLES.map((t) => `SELECT DISTINCT UNNEST("tags") AS v FROM ${qualifiedTable(schema, t)} WHERE array_length("tags", 1) > 0 ${filter}`).join(" UNION ");
+		const unions = ENTITY_DISCOVERY_TABLES.map((t) => `SELECT DISTINCT UNNEST("tags") AS v FROM ${qualifiedTable(schema, t)} WHERE array_length("tags", 1) > 0 ${filter} ${lookbackPredicate(t, lookback)}`).join(" UNION ");
 		return (await client.manyOrNone(`SELECT v FROM (${unions}) sub WHERE v IS NOT NULL AND v <> '' ORDER BY v`, params)).map((r) => r.v);
 	};
 	return { tags: await readWithRefresh(client, schema, cacheKey, refresh, ttl, config.logger) };
 }
 async function getMetricNames(client, schema, args, config) {
+	const ttl = config.ttlSeconds ?? DEFAULT_TTL_SECONDS;
+	const lookback = config.lookbackSeconds ?? DEFAULT_LOOKBACK_SECONDS;
 	let filtered = await readWithRefresh(client, schema, "metric_names", async () => {
 		return (await client.manyOrNone(`SELECT DISTINCT "name" AS v FROM ${qualifiedTable(schema, TABLE_METRIC_EVENTS)}
-         WHERE "name" IS NOT NULL AND "name" <> '' ORDER BY "name"`)).map((r) => r.v);
-	}, config.ttlSeconds ?? DEFAULT_TTL_SECONDS, config.logger);
+         WHERE "name" IS NOT NULL AND "name" <> '' ${lookbackPredicate(TABLE_METRIC_EVENTS, lookback)} ORDER BY "name"`)).map((r) => r.v);
+	}, ttl, config.logger);
 	if (args.prefix) filtered = filtered.filter((v) => v.startsWith(args.prefix));
 	if (args.limit) filtered = filtered.slice(0, args.limit);
 	return { names: filtered };
 }
 async function getMetricLabelKeys(client, schema, args, config) {
 	const ttl = config.ttlSeconds ?? DEFAULT_TTL_SECONDS;
+	const lookback = config.lookbackSeconds ?? DEFAULT_LOOKBACK_SECONDS;
 	return { keys: await readWithRefresh(client, schema, `metric_label_keys:${args.metricName}`, async () => {
 		return (await client.manyOrNone(`SELECT DISTINCT k AS v
          FROM ${qualifiedTable(schema, TABLE_METRIC_EVENTS)}, jsonb_object_keys("labels") k
-         WHERE "name" = $1 ORDER BY k`, [args.metricName])).map((r) => r.v);
+         WHERE "name" = $1 ${lookbackPredicate(TABLE_METRIC_EVENTS, lookback)} ORDER BY k`, [args.metricName])).map((r) => r.v);
 	}, ttl, config.logger) };
 }
 async function getMetricLabelValues(client, schema, args, config) {
 	const ttl = config.ttlSeconds ?? DEFAULT_TTL_SECONDS;
+	const lookback = config.lookbackSeconds ?? DEFAULT_LOOKBACK_SECONDS;
 	let filtered = await readWithRefresh(client, schema, `metric_label_values:${args.metricName}:${args.labelKey}`, async () => {
 		return (await client.manyOrNone(`SELECT DISTINCT "labels" ->> $2 AS v
          FROM ${qualifiedTable(schema, TABLE_METRIC_EVENTS)}
-         WHERE "name" = $1 AND "labels" ? $2
+         WHERE "name" = $1 AND "labels" ? $2 ${lookbackPredicate(TABLE_METRIC_EVENTS, lookback)}
          ORDER BY v`, [args.metricName, args.labelKey])).map((r) => r.v).filter((v) => v != null && v !== "");
 	}, ttl, config.logger);
 	if (args.prefix) filtered = filtered.filter((v) => v.startsWith(args.prefix));
@@ -21931,7 +22026,7 @@ var PgFactoryStorageOps = class {
 					row[name] = Boolean(value);
 					break;
 				case "json":
-					row[name] = typeof value === "string" ? JSON.parse(value) : value;
+					row[name] = value;
 					break;
 				case "bigint":
 				case "integer":