@mastra/client-js 1.46.1-alpha.0 → 1.47.0-alpha.10

package/dist/docs/references/reference-observability-tracing-trace-query.md

@@ -4,9 +4,9 @@
 
 # Advanced trace queries
 
-Use `POST /api/observability/traces/query` to find completed logical traces that match trace fields and related span or score records. The endpoint returns a fixed lightweight trace projection, or distinct thread IDs when you group by `threadId`.
+Use `queryTraces()` or `POST /api/observability/traces/query` to find completed logical traces that match trace fields and related span, score, or feedback records. Use `queryTraceThreads()` or `POST /api/observability/threads/query` to find thread identities that qualify across multiple eligible traces.
 
-The endpoint uses the same authentication as other observability routes and requires the `observability:read` permission. The configured observability store must support advanced trace queries.
+Both endpoints use the same authentication as other observability routes and require the `observability:read` permission. The configured observability store must support the requested trace-query or thread-query operation.
 
 ## Query traces with the client SDK
 
@@ -37,7 +37,74 @@ const result = await mastraClient.queryTraces({
 
 A `some` clause matches when one related record satisfies its complete nested predicate. In this example, `scorerId`, `scorerVersion`, and `score` must match on the same score record. A `none` clause uses anti-existence semantics: it matches when no related record satisfies its complete nested predicate. A trace with no related scores therefore matches `scores.none`.
 
-Span clauses examine the current root span and current child spans. The root `timeRange` applies only to the selected current root's `startedAt`. Related spans and scores can participate even when their own timestamps are outside that range. Related records correlate only through a matching non-null `traceId`.
+Span clauses examine the current root span and current child spans. The root `timeRange` applies only to the selected current root's `startedAt`. Related spans, scores, and feedback can participate even when their own timestamps are outside that range. Related records correlate only through a matching non-null `traceId`.
+
+## Query threads across traces
+
+Pass an eligible trace selection and an optional thread predicate to `queryTraceThreads()`:
+
+```typescript
+const result = await mastraClient.queryTraceThreads({
+  traces: {
+    timeRange: {
+      from: '2026-08-01T00:00:00.000Z',
+      to: '2026-08-08T00:00:00.000Z',
+    },
+    where: {
+      op: 'eq',
+      left: { path: 'environment' },
+      right: { literal: 'production' },
+    },
+  },
+  where: {
+    op: 'and',
+    args: [
+      {
+        traces: {
+          some: {
+            scores: {
+              some: {
+                op: 'lt',
+                left: { path: 'score' },
+                right: { literal: 0.6 },
+              },
+            },
+          },
+        },
+      },
+      {
+        traces: {
+          some: {
+            feedback: {
+              some: {
+                op: 'eq',
+                left: { path: 'feedbackType' },
+                right: { literal: 'clinician-correction' },
+              },
+            },
+          },
+        },
+      },
+    ],
+  },
+  page: { limit: 25 },
+})
+```
+
+The server applies `traces.timeRange` and `traces.where` first. This produces the complete eligible trace population. It then derives distinct non-null `threadId` values and evaluates the top-level `where` predicate against eligible traces in each thread.
+
+One `traces.some` clause requires one eligible trace to satisfy its complete nested trace predicate. Separate `traces.some` clauses are independent, so different traces in the same thread can satisfy them. Nested `spans.some`, `scores.some`, and `feedback.some` clauses still require one related record to satisfy every condition inside that clause.
+
+`traces.none: P` uses anti-existence semantics: it matches only when no eligible trace in the thread satisfies `P`. This differs from `traces.some: { feedback: { none: P } }`, which requires at least one eligible trace with no matching feedback record.
+
+The response contains identities only:
+
+```json
+{
+  "threads": [{ "threadId": "thread-123" }],
+  "page": { "next": null }
+}
+```
 
 ## Send an HTTP request
 
@@ -65,34 +132,143 @@ curl --request POST \
   }'
 ```
 
+## Discover fields and values
+
+Use trace-query discovery to build field and value autocomplete without scanning trace payloads or copying the query grammar into your client. The configured observability store must support `trace-query-discovery`.
+
+Call `getTraceQueryFields()` for one predicate scope:
+
+```typescript
+const timeRange = {
+  from: '2026-08-01T00:00:00.000Z',
+  to: '2026-08-08T00:00:00.000Z',
+}
+
+const fields = await mastraClient.getTraceQueryFields({
+  timeRange,
+  predicateScope: 'trace',
+  search: 'region',
+  limit: 25,
+})
+```
+
+The response separates canonical fields from observed metadata fields:
+
+```json
+{
+  "canonicalFields": [],
+  "observedFields": [
+    {
+      "path": "metadata.region",
+      "valueKind": "string",
+      "operators": ["eq", "ne", "in", "notIn", "exists", "notExists"],
+      "valueSuggestions": true,
+      "occurrences": 125
+    }
+  ],
+  "observedFieldsTruncated": false
+}
+```
+
+Canonical fields come from the same registry used to validate trace queries. They don't consume `limit`. The canonical order follows that registry. `observedFields` contains top-level string `metadata.<key>` paths found on qualifying current root spans. These fields are ordered by `occurrences` descending, then by path. Nested metadata, keys containing `.`, empty keys, non-string values, and oversized paths are omitted.
+
+`predicateScope` selects the local predicate grammar. It isn't an authorization scope:
+
+| `predicateScope` | Example returned path              | Query usage                               |
+| ---------------- | ---------------------------------- | ----------------------------------------- |
+| `trace`          | `environment` or `metadata.region` | Root `where` predicate                    |
+| `spans`          | `model`                            | Inside `spans.some` or `spans.none`       |
+| `scores`         | `scorerId`                         | Inside `scores.some` or `scores.none`     |
+| `feedback`       | `feedbackType`                     | Inside `feedback.some` or `feedback.none` |
+
+A global field picker can request all four scopes in parallel and qualify display labels locally. Every returned path is directly accepted by `queryTraces()` in its matching scope. `metadata` by itself isn't a field. A user interface can offer manual metadata-key entry separately.
+
+After a user selects one field with `valueSuggestions: true`, call `getTraceQueryValues()`:
+
+```typescript
+const abortController = new AbortController()
+const suggestions = await mastraClient.getTraceQueryValues(
+  {
+    timeRange,
+    predicateScope: 'trace',
+    path: 'metadata.region',
+    search: 'west',
+    limit: 25,
+  },
+  { signal: abortController.signal },
+)
+```
+
+```json
+{
+  "values": [
+    { "value": "eu-west-1", "count": 82 },
+    { "value": "us-west-2", "count": 43 }
+  ],
+  "valuesTruncated": false
+}
+```
+
+Discovery returns string values only. Missing values, empty strings, and strings larger than 4,096 UTF-8 bytes are omitted. Values are ordered by `count` descending, then by value. Suggestions are advisory: `queryTraces()` continues to accept valid manual literals that discovery doesn't return.
+
+Value suggestions are available for these canonical fields:
+
+| Scope    | Fields                                                                        |
+| -------- | ----------------------------------------------------------------------------- |
+| Trace    | `entityName`, `entityType`, `environment`, `status`                           |
+| Spans    | `name`, `spanType`, `model`, `provider`, `status`, `entityType`, `entityName` |
+| Scores   | `scorerId`, `scorerVersion`, `scoreSource`                                    |
+| Feedback | `feedbackType`, `feedbackSource`                                              |
+
+Each executable top-level string `metadata.<key>` field also supports value suggestions in the `trace` scope. Identifiers, timestamps, durations, numeric score or feedback values, comments, errors, and version IDs don't. The fields response still includes canonical fields with `valueSuggestions: false`. The values endpoint rejects those paths with `422 TRACE_QUERY_INVALID`.
+
+Both discovery requests require a half-open time range of at most 31 days. They use the same current-record semantics as `queryTraces()`: qualifying completed roots satisfy `from <= startedAt < to`, and related values come only from current records joined to those roots by `traceId`. Discovery doesn't accept or apply a draft `where` predicate.
+
+`search` defaults to the empty string, is trimmed, and matches case-insensitive literal substrings. Characters such as `%` and `_` have no wildcard meaning. An empty search returns the most frequent results. `limit` defaults to 25 and has a maximum of 100. Both endpoints fetch one extra result to set their truncation flag, but don't provide a cursor. When a flag is `true`, refine `search`. It doesn't imply that a next page is available. No matches return an empty array and `false` truncation.
+
+If discovery exceeds its execution timeout, the route returns `504 TRACE_QUERY_EXECUTION_TIMEOUT`. If the backend exceeds a memory or resource budget, it returns `503 TRACE_QUERY_RESOURCE_LIMIT`. Neither failure returns partial suggestions or sets a truncation flag. Truncation only means a complete ranking was cut to the requested response limit.
+
+ClickHouse discovery requests default to a 5-second timeout and a 256 MiB per-query memory limit. Configure these with `observability.traceQuery.discovery.timeoutMs` and `observability.traceQuery.discovery.memoryLimitBytes` on `ClickhouseStoreVNext`. Discovery falls back to `observability.traceQuery.timeoutMs` when its timeout isn't configured, without changing ordinary `queryTraces()` execution limits.
+
+Both routes require `observability:read` and use the configured observability storage boundary. Don't send organization or project authorization fields in the request. Treat discovered paths, values, and counts as observability data.
+
 ## Request fields
 
-| Field       | Required | Description                                                                                                                                         |
-| ----------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `timeRange` | Yes      | Trace start-time boundary. `from` is inclusive, `to` is exclusive, and the range can span at most 31 days. Both values must be ISO timestamps.      |
-| `where`     | No       | Recursive trace predicate. Supports scalar conditions and `spans.some`, `spans.none`, `scores.some`, and `scores.none`.                             |
-| `group`     | No       | Set to `{ by: ['threadId'] }` to return distinct non-null thread IDs.                                                                               |
-| `orderBy`   | No       | One item ordering ungrouped results by `startedAt` or `endedAt`, in `asc` or `desc` order. Defaults to `startedAt desc`. Not accepted with `group`. |
-| `page`      | No       | `{ limit, after }`. `limit` defaults to 100 and has a maximum of 1000. Pass the opaque `page.next` value as `after`.                                |
+### Trace queries
+
+| Field       | Required | Description                                                                                                                                                                     |
+| ----------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `timeRange` | Yes      | Trace start-time boundary. `from` is inclusive and `to` is exclusive. Both values must be ISO timestamps, `from` must be earlier than `to`, and the range can't exceed 31 days. |
+| `where`     | No       | Recursive trace predicate. Supports scalar conditions and `spans`, `scores`, and `feedback` `some` or `none` clauses.                                                           |
+| `orderBy`   | No       | One item ordering results by `startedAt` or `endedAt`, in `asc` or `desc` order. Defaults to `startedAt desc`.                                                                  |
+| `page`      | No       | `{ limit, after }`. `limit` defaults to 100 and has a maximum of 1000. Pass the opaque `page.next` value as `after`.                                                            |
+
+### Thread queries
 
-Unknown fields are rejected. The request can't select a projection, declare joins, or control authorization. Request bodies are limited to 256 KiB.
+| Field    | Required | Description                                                                                                                                         |
+| -------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `traces` | Yes      | Eligible trace selection containing `timeRange` and an optional trace `where` predicate.                                                            |
+| `where`  | No       | Recursive thread predicate composed with boolean operators and `traces.some` or `traces.none`. Each quantifier contains a complete trace predicate. |
+| `page`   | No       | `{ limit, after }`. Thread identities always use fixed ordinal `threadId` ascending order. Pass the opaque `page.next` value as `after`.            |
+
+Unknown fields are rejected. Requests can't select a projection, declare joins, request counts or measures, or control authorization. Request bodies are limited to 256 KiB.
 
 ## Query limits
 
 The planner rejects a query before storage execution when it exceeds any of these limits:
 
-| Input                                     | Maximum           |
-| ----------------------------------------- | ----------------- |
-| Root `timeRange`                          | 31 days           |
-| Predicate nesting depth                   | 12 levels         |
-| Predicate nodes                           | 100               |
-| Related `spans` and `scores` clauses      | 8 total           |
-| Values in one `in` or `notIn` set         | 100               |
-| Comparison literals and membership values | 1,000 total       |
-| String literal                            | 4,096 UTF-8 bytes |
-| Raw predicate path                        | 128 UTF-8 bytes   |
-| `page.limit`                              | 1,000             |
-| HTTP request body                         | 256 KiB           |
+| Input                                                       | Maximum           |
+| ----------------------------------------------------------- | ----------------- |
+| Trace selection `timeRange`                                 | 31 days           |
+| Predicate nesting depth                                     | 12 levels         |
+| Predicate nodes                                             | 100               |
+| Related `traces`, `spans`, `scores`, and `feedback` clauses | 8 total           |
+| Values in one `in` or `notIn` set                           | 100               |
+| Comparison literals and membership values                   | 1,000 total       |
+| String literal                                              | 4,096 UTF-8 bytes |
+| Raw predicate path                                          | 128 UTF-8 bytes   |
+| `page.limit`                                                | 1,000             |
+| HTTP request body                                           | 256 KiB           |
 
 Each comparison literal counts as one literal. Each member of an `in` or `notIn` set also counts as one literal, even when the set is within its per-set limit.
 
@@ -271,7 +447,7 @@ const messageTrace = {
 
 The key must name one top-level property. Empty keys and nested paths are rejected. Metadata keys aren't trimmed, so leading and trailing whitespace remains part of the exact key identity. When metadata duplicates a canonical trace field, such as `resourceId`, `threadId`, or `environment`, prefer the canonical field because it uses the dedicated storage column. The `metadata.<key>` form remains available when you specifically need the value from the metadata object. Metadata and canonical values may differ.
 
-Metadata fields aren't available for grouping or field discovery.
+Metadata fields aren't available for grouping. Trace-query discovery returns executable top-level string metadata fields observed in the selected time range.
 
 ### Filter by feedback
 
@@ -336,43 +512,40 @@ An ungrouped query returns only lightweight completed traces:
 }
 ```
 
-A grouped query returns distinct non-null thread IDs in ascending order:
+A thread query returns distinct non-null thread IDs in ordinal ascending order:
 
-```typescript
-const result = await mastraClient.queryTraces({
-  timeRange: {
-    from: '2026-08-01T00:00:00.000Z',
-    to: '2026-08-08T00:00:00.000Z',
-  },
-  group: { by: ['threadId'] },
-})
-// { groups: [{ threadId: 'thread-123' }], page: { next: null } }
+```json
+{
+  "threads": [{ "threadId": "thread-123" }],
+  "page": { "next": null }
+}
 ```
 
 Related evidence isn't embedded in either response. Use the trace-detail and branch APIs to load spans after selecting a result.
 
 ## Pagination and errors
 
-Ordering is deterministic. Ungrouped ordering appends `traceId` ascending as a tie-breaker. Grouped queries always order by `threadId` ascending.
+Ordering is deterministic. Trace ordering appends `traceId` ascending as a tie-breaker. Thread queries always use raw ordinal `threadId` ascending order. Callers can't override it.
 
-Cursors are bound to the accepted normalized query shape and ordering. Reusing a cursor after changing the time range, predicates, grouping, or ordering returns `409`. A malformed cursor returns `400`.
+Cursors are bound to the operation, accepted normalized query, authorization state, and ordering. Reusing a cursor after changing the trace selection, predicates, or ordering returns `409`. Trace and thread cursors aren't interchangeable. A malformed cursor returns `400`.
 
 Cursor pagination is deterministic, but it isn't a database snapshot. Traces or replacement signals written between page requests can change later pages.
 
-PostgreSQL and ClickHouse stop an advanced trace query after 15 seconds by default and return `504` when the database timeout is exceeded. Set `traceQueryTimeoutMs` in the store's vNext observability configuration to an integer from 1 through 300,000 milliseconds to change the timeout. DuckDB doesn't currently provide query-scoped timeout or cancellation through its driver wrapper, so this `504` guarantee doesn't apply to DuckDB.
+PostgreSQL and ClickHouse stop advanced trace and thread queries after 15 seconds by default and return `504` when the database timeout is exceeded. Set `traceQueryTimeoutMs` in the store's vNext observability configuration to an integer from 1 through 300,000 milliseconds to change the timeout. DuckDB doesn't currently provide query-scoped timeout or cancellation through its driver wrapper, so this `504` guarantee doesn't apply to DuckDB.
 
-| Status | Meaning                                                                                                                               |
-| ------ | ------------------------------------------------------------------------------------------------------------------------------------- |
-| `400`  | Malformed JSON or malformed cursor.                                                                                                   |
-| `409`  | The cursor doesn't match the query.                                                                                                   |
-| `413`  | The request body exceeds 256 KiB.                                                                                                     |
-| `422`  | The JSON is well formed, but the request is structurally or semantically invalid. The response includes stable issue codes and paths. |
-| `501`  | The configured observability store doesn't support advanced trace queries.                                                            |
-| `504`  | A PostgreSQL or ClickHouse query exceeded its configured database execution timeout.                                                  |
+| Status | Meaning                                                                                                                                                                                                                                                  |
+| ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `400`  | Malformed JSON or malformed cursor.                                                                                                                                                                                                                      |
+| `409`  | The cursor doesn't match the query.                                                                                                                                                                                                                      |
+| `413`  | The request body exceeds 256 KiB.                                                                                                                                                                                                                        |
+| `422`  | The JSON is well formed, but the request is structurally or semantically invalid. The response includes stable issue codes and paths. Discovery validation uses `TRACE_QUERY_INVALID`.                                                                   |
+| `501`  | The installed Core or configured observability store doesn't support the requested operation. Discovery returns `TRACE_QUERY_DISCOVERY_UNSUPPORTED` unless Core provides the discovery contract and the store implements both field and value discovery. |
+| `503`  | Discovery exceeded a backend memory or resource budget and returned `TRACE_QUERY_RESOURCE_LIMIT`.                                                                                                                                                        |
+| `504`  | A PostgreSQL or ClickHouse query exceeded its configured database execution timeout. Discovery returns `TRACE_QUERY_EXECUTION_TIMEOUT`.                                                                                                                  |
 
 ## Limitations
 
-The endpoint returns completed traces only. It doesn't support running traces, custom projections, embedded evidence, summaries, aggregations, grouping by fields other than `threadId`, or conditions over an entire group.
+Both operations consider completed traces only. They don't support running traces, custom projections, embedded evidence, summaries, aggregations, counts, measures, or custom grouping.
 
 ## Related