@mastra/duckdb 1.2.0 → 1.3.0-alpha.1

package/CHANGELOG.md

@@ -1,5 +1,61 @@
 # @mastra/duckdb
 
+## 1.3.0-alpha.1
+
+### Patch Changes
+
+- Improved performance of `listTraces` and `listBranches` on DuckDB. The Traces and Branches lists in the observability UI now load noticeably faster, especially on large span tables, because filtering and pagination happen up front and the store only assembles full span data for the rows on the page being viewed. ([#16165](https://github.com/mastra-ai/mastra/pull/16165))
+
+  No API or behavior changes — return shapes and filter semantics are unchanged, and no migration is required.
+
+- Updated dependencies [[`ca28c23`](https://github.com/mastra-ai/mastra/commit/ca28c232a2f18801a6cf20fe053479237b4d4fb0)]:
+  - @mastra/core@1.32.0-alpha.3
+
+## 1.3.0-alpha.0
+
+### Minor Changes
+
+- - **Added** `listBranches` and `getSpans` implementations. ([#16154](https://github.com/mastra-ai/mastra/pull/16154))
+  - Historical span data is queryable immediately; no migration required.
+
+- Added `count_distinct` aggregation and server-side TopK to the metrics storage API so dashboards built on high-cardinality fields (like `threadId` or `resourceId`) stay fast and bounded. ([#16137](https://github.com/mastra-ai/mastra/pull/16137))
+
+  **New aggregation**
+
+  `getMetricAggregate`, `getMetricBreakdown`, and `getMetricTimeSeries` accept `aggregation: 'count_distinct'` with a `distinctColumn`. Backends pick the most efficient native implementation — `uniq` on ClickHouse, `approx_count_distinct` on DuckDB.
+
+  `distinctColumn` is restricted to a low/medium-cardinality categorical allowlist (`entityType`, `entityName`, `parentEntityType`, `parentEntityName`, `rootEntityType`, `rootEntityName`, `name`, `provider`, `model`, `environment`, `executionSource`, `serviceName`). ID columns are not allowed — distinct counts over near-unique values converge to the row count and are rarely useful.
+
+  ```ts
+  await store.getMetricAggregate({
+    name: ['mastra_llm_tokens_total'],
+    aggregation: 'count_distinct',
+    distinctColumn: 'model',
+    filters: { timestamp: { start, end } },
+  });
+  ```
+
+  **Server-side TopK**
+
+  `getMetricBreakdown` accepts `limit` and `orderDirection`, so breakdowns never return the full cardinality of a column from the database. Ordering is always by the aggregated `value`; `orderDirection` flips between top-N (`DESC`, default) and bottom-N (`ASC`).
+
+  ```ts
+  await store.getMetricBreakdown({
+    name: ['mastra_agent_duration_ms'],
+    aggregation: 'sum',
+    groupBy: ['threadId'],
+    limit: 20,
+    orderDirection: 'DESC',
+  });
+  ```
+
+### Patch Changes
+
+- Added direct score lookup support to observability storage so score records can be fetched by `scoreId` without scanning paginated score lists, including DuckDB and ClickHouse vNext observability stores. ([#16162](https://github.com/mastra-ai/mastra/pull/16162))
+
+- Updated dependencies [[`86c0298`](https://github.com/mastra-ai/mastra/commit/86c0298e647306423c842f9d5ac827bd616bd13d), [`7fce309`](https://github.com/mastra-ai/mastra/commit/7fce30912b14170bfc41f0ac736cca0f39fe0cd4), [`7997c2e`](https://github.com/mastra-ai/mastra/commit/7997c2e55ddd121562a4098cd8d2b89c68433bf1), [`e97ccb9`](https://github.com/mastra-ai/mastra/commit/e97ccb900f8b7a390ce82c9f8eb8d6eb2c5e3777), [`c5daf48`](https://github.com/mastra-ai/mastra/commit/c5daf48556e98c46ae06caf00f92c249912007e9), [`cd96779`](https://github.com/mastra-ai/mastra/commit/cd9677937f113b2856dc8b9f3d4bdabcee58bb2e)]:
+  - @mastra/core@1.32.0-alpha.2
+
 ## 1.2.0
 
 ### Minor Changes

package/dist/docs/SKILL.md

@@ -3,7 +3,7 @@ name: mastra-duckdb
 description: Documentation for @mastra/duckdb. Use when working with @mastra/duckdb APIs, configuration, or implementation.
 metadata:
   package: "@mastra/duckdb"
-  version: "1.2.0"
+  version: "1.3.0-alpha.1"
 ---
 
 ## When to use
@@ -16,6 +16,7 @@ Read the individual reference documents for detailed explanations and code examp
 
 ### Reference
 
+- [Reference: DuckDB storage](references/reference-storage-duckdb.md) - Documentation for the DuckDB storage implementation in Mastra, an embedded analytical backend for local observability development.
 - [Reference: DuckDB vector store](references/reference-vectors-duckdb.md) - Documentation for the DuckDBVector class in Mastra, which provides embedded high-performance vector search using DuckDB with HNSW indexing.
 
 

package/dist/docs/assets/SOURCE_MAP.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.2.0",
+  "version": "1.3.0-alpha.1",
   "package": "@mastra/duckdb",
   "exports": {
     "DuckDBConnection": {

package/dist/docs/references/reference-storage-duckdb.md

@@ -0,0 +1,150 @@
+# DuckDB storage
+
+[DuckDB](https://duckdb.org/) is an embedded, in-process analytical database. The `@mastra/duckdb` package provides an OLAP-backed observability store for local development, suitable for traces, logs, metrics, scores, and feedback without running an external service.
+
+For vector search, see the [DuckDB vector store reference](https://mastra.ai/reference/vectors/duckdb), which is a separate API in the same package.
+
+## When to use DuckDB
+
+Local development of observability features. DuckDB is embedded and file-based, so it does not require a server and starts instantly. It supports the same observability signals as ClickHouse, which makes it useful for testing dashboards and trace exploration before deploying to a production backend.
+
+DuckDB currently implements only the `observability` domain. Pair it with another storage adapter (such as [LibSQL](https://mastra.ai/reference/storage/libsql)) for `memory` and `workflows` in a [composite storage](https://mastra.ai/reference/storage/composite) setup.
+
+> **Warning:** DuckDB is for development and not recommended for production. It runs in-process, persists to a single local file, and does not work on platforms with ephemeral filesystems (such as Railway, Fly.io, Render, Heroku, or serverless containers). For production observability, use [ClickHouse](https://mastra.ai/reference/storage/clickhouse).
+
+## Installation
+
+**npm**:
+
+```bash
+npm install @mastra/duckdb@latest
+```
+
+**pnpm**:
+
+```bash
+pnpm add @mastra/duckdb@latest
+```
+
+**Yarn**:
+
+```bash
+yarn add @mastra/duckdb@latest
+```
+
+**Bun**:
+
+```bash
+bun add @mastra/duckdb@latest
+```
+
+## Usage
+
+### As the observability domain in a composite store
+
+This is the standard local-development setup. LibSQL handles the other domains, and DuckDB handles observability.
+
+```typescript
+import { Mastra } from '@mastra/core'
+import { MastraCompositeStore } from '@mastra/core/storage'
+import { LibSQLStore } from '@mastra/libsql'
+import { DuckDBStore } from '@mastra/duckdb'
+import { Observability, DefaultExporter } from '@mastra/observability'
+
+export const mastra = new Mastra({
+  storage: new MastraCompositeStore({
+    id: 'composite-storage',
+    default: new LibSQLStore({
+      id: 'mastra-storage',
+      url: 'file:./mastra.db',
+    }),
+    domains: {
+      observability: new DuckDBStore().observability,
+    },
+  }),
+  observability: new Observability({
+    configs: {
+      default: {
+        serviceName: 'mastra',
+        exporters: [new DefaultExporter()],
+      },
+    },
+  }),
+})
+```
+
+The `.observability` accessor returns the observability domain store directly. The equivalent generic form uses `getStore()`, which works for any composite-style storage adapter:
+
+```typescript
+const observability = await new DuckDBStore().getStore('observability')
+```
+
+### Standalone
+
+When you need only observability storage outside the `Mastra` composite, instantiate `DuckDBStore` directly and access the observability domain:
+
+```typescript
+import { DuckDBStore } from '@mastra/duckdb'
+
+const duckdb = new DuckDBStore({ path: './traces.duckdb' })
+const observability = duckdb.observability
+
+await observability.init()
+```
+
+### In-memory database
+
+Pass `:memory:` to use an ephemeral DuckDB instance. Data is lost when the process exits, which is appropriate for unit tests and short-lived scripts.
+
+```typescript
+const duckdb = new DuckDBStore({ path: ':memory:' })
+```
+
+## Configuration
+
+### `DuckDBStore` options
+
+**id** (`string`): Unique identifier for this storage instance. (Default: `'duckdb'`)
+
+**path** (`string`): Path to the DuckDB database file. Use \`:memory:\` for an ephemeral in-memory database. (Default: `'mastra.duckdb'`)
+
+### Lower-level types
+
+`@mastra/duckdb` also exports `DuckDBConnection` for sharing a single underlying database across multiple Mastra storage instances, and the corresponding `DuckDBStorageConfig` type. Most applications will not need these directly.
+
+## Supported domains
+
+DuckDB currently implements one storage domain:
+
+| Domain          | Supported |
+| --------------- | --------- |
+| `observability` | Yes       |
+| `memory`        | No        |
+| `workflows`     | No        |
+| `scores`        | No        |
+| `agents`        | No        |
+
+For a full storage solution, compose `DuckDBStore` with an adapter that covers the missing domains (most commonly [LibSQL](https://mastra.ai/reference/storage/libsql) for local development).
+
+## Initialization
+
+When passed to `Mastra` through `MastraCompositeStore`, the observability domain initializes itself on first use. To run initialization explicitly outside of `Mastra`, call `init()` on the observability store:
+
+```typescript
+import { DuckDBStore } from '@mastra/duckdb'
+
+const duckdb = new DuckDBStore({ path: './traces.duckdb' })
+await duckdb.observability.init()
+```
+
+## Observability strategy
+
+DuckDB supports the `event-sourced` strategy used by `DefaultExporter`, which buffers spans in memory and writes completed events in batches. This is appropriate for development-scale traffic. For high-volume production workloads, see [`DefaultExporter` storage provider support](https://mastra.ai/docs/observability/tracing/exporters/default).
+
+## Related
+
+- [Storage overview](https://mastra.ai/reference/storage/overview)
+- [Composite storage](https://mastra.ai/reference/storage/composite)
+- [ClickHouse storage](https://mastra.ai/reference/storage/clickhouse): Production observability backend
+- [DuckDB vector store](https://mastra.ai/reference/vectors/duckdb): Vector search using the same package
+- [Observability overview](https://mastra.ai/docs/observability/overview)

package/dist/index.cjs

@@ -583,7 +583,7 @@ var ObservabilityStorageDuckDB = class extends storage.ObservabilityStorage {
       return null;
     }
     if (!this.loadPromise) {
-      this.loadPromise = import('./observability-AILZGFQT.cjs').then(({ ObservabilityStorageDuckDB: ObservabilityStorageDuckDB2 }) => {
+      this.loadPromise = import('./observability-2PJ44OVC.cjs').then(({ ObservabilityStorageDuckDB: ObservabilityStorageDuckDB2 }) => {
         const delegate = new ObservabilityStorageDuckDB2({ db: this.db });
         this.delegate = delegate;
         return delegate;
@@ -640,6 +640,10 @@ var ObservabilityStorageDuckDB = class extends storage.ObservabilityStorage {
     const delegate = await this.requireDelegate();
     return delegate.getSpan(...args);
   }
+  async getSpans(...args) {
+    const delegate = await this.requireDelegate();
+    return delegate.getSpans(...args);
+  }
   async getRootSpan(...args) {
     const delegate = await this.requireDelegate();
     return delegate.getRootSpan(...args);
@@ -656,6 +660,10 @@ var ObservabilityStorageDuckDB = class extends storage.ObservabilityStorage {
     const delegate = await this.requireDelegate();
     return delegate.listTraces(...args);
   }
+  async listBranches(...args) {
+    const delegate = await this.requireDelegate();
+    return delegate.listBranches(...args);
+  }
   async batchCreateSpans(...args) {
     const delegate = await this.requireDelegate();
     return delegate.batchCreateSpans(...args);
@@ -744,6 +752,10 @@ var ObservabilityStorageDuckDB = class extends storage.ObservabilityStorage {
     const delegate = await this.requireDelegate();
     return delegate.listScores(...args);
   }
+  async getScoreById(...args) {
+    const delegate = await this.requireDelegate();
+    return delegate.getScoreById(...args);
+  }
   async getScoreAggregate(...args) {
     const delegate = await this.requireDelegate();
     return delegate.getScoreAggregate(...args);