@mastra/clickhouse 1.9.1 → 1.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +70 -0
- package/README.md +27 -0
- package/dist/docs/SKILL.md +1 -1
- package/dist/docs/assets/SOURCE_MAP.json +1 -1
- package/dist/docs/references/reference-storage-clickhouse.md +52 -7
- package/dist/docs/references/reference-storage-composite.md +34 -2
- package/dist/index.cjs +243 -38
- package/dist/index.cjs.map +1 -1
- package/dist/index.js +244 -39
- package/dist/index.js.map +1 -1
- package/dist/storage/db/index.d.ts +7 -1
- package/dist/storage/db/index.d.ts.map +1 -1
- package/dist/storage/db/replication.d.ts +52 -0
- package/dist/storage/db/replication.d.ts.map +1 -0
- package/dist/storage/db/utils.d.ts +2 -0
- package/dist/storage/db/utils.d.ts.map +1 -1
- package/dist/storage/domains/observability/v-next/index.d.ts.map +1 -1
- package/dist/storage/index.d.ts +8 -0
- package/dist/storage/index.d.ts.map +1 -1
- package/package.json +8 -7
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,75 @@
|
|
|
1
1
|
# @mastra/clickhouse
|
|
2
2
|
|
|
3
|
+
## 1.10.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- Added opt-in replicated ClickHouse table support for multi-replica clusters. ([#17684](https://github.com/mastra-ai/mastra/pull/17684))
|
|
8
|
+
|
|
9
|
+
Configure `replication` to have Mastra create replicated MergeTree tables and add `ON CLUSTER` to Mastra-owned DDL when a cluster name is provided:
|
|
10
|
+
|
|
11
|
+
```ts
|
|
12
|
+
new ClickhouseStoreVNext({
|
|
13
|
+
id: 'clickhouse-storage',
|
|
14
|
+
url: process.env.CLICKHOUSE_URL!,
|
|
15
|
+
username: process.env.CLICKHOUSE_USERNAME!,
|
|
16
|
+
password: process.env.CLICKHOUSE_PASSWORD!,
|
|
17
|
+
replication: {
|
|
18
|
+
cluster: 'company_cluster',
|
|
19
|
+
},
|
|
20
|
+
});
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
Notes:
|
|
24
|
+
- If existing Mastra tables use local `MergeTree` or `ReplacingMergeTree` engines, initialization fails. Migrate existing local tables to `Replicated*` engines before enabling `replication`.
|
|
25
|
+
- vNext observability signal-table migrations (`migrateSpans()`) are blocked while `replication` is configured. Migrate legacy signal tables before turning on replication.
|
|
26
|
+
- The default `zookeeperPath` is `/clickhouse/tables/{shard}/{database}/{table}`. Override it if your cluster uses a different convention.
|
|
27
|
+
|
|
28
|
+
### Patch Changes
|
|
29
|
+
|
|
30
|
+
- dependencies updates: ([#17148](https://github.com/mastra-ai/mastra/pull/17148))
|
|
31
|
+
- Updated dependency [`@clickhouse/client@^1.20.0` ↗︎](https://www.npmjs.com/package/@clickhouse/client/v/1.20.0) (from `^1.18.2`, in `dependencies`)
|
|
32
|
+
|
|
33
|
+
- Fixed DateTime64 comparison error with ClickHouse 26.5 by removing invalid empty string comparison on nullable DateTime64 columns in span deduplication query ([#17149](https://github.com/mastra-ai/mastra/pull/17149))
|
|
34
|
+
|
|
35
|
+
- Updated dependencies [[`d468acb`](https://github.com/mastra-ai/mastra/commit/d468acb07aec1bb19a2cb0ada8042b05b46746b2), [`575f815`](https://github.com/mastra-ai/mastra/commit/575f815c5c3567b71c0b83cbb7fa98c8253a9d9c), [`34839c1`](https://github.com/mastra-ai/mastra/commit/34839c1910b6964bf59ed0cee58844efebbb684e), [`053735a`](https://github.com/mastra-ai/mastra/commit/053735a75c2c18e23ce34d9468007efa4a45f4c4), [`306909a`](https://github.com/mastra-ai/mastra/commit/306909a693de77d709b38706e2673c9547d24a28), [`5191af8`](https://github.com/mastra-ai/mastra/commit/5191af80c799eea25357c545fc05d91b3883531d), [`43bd3d4`](https://github.com/mastra-ai/mastra/commit/43bd3d421987463fdf35386a45199c49499ed069), [`e6fa79e`](https://github.com/mastra-ai/mastra/commit/e6fa79ec72a2ddffdd25e85270398951e9d552a4), [`904bcdf`](https://github.com/mastra-ai/mastra/commit/904bcdf7b8004aa7be823f9f70ca63580e47e470), [`7f5ee1d`](https://github.com/mastra-ai/mastra/commit/7f5ee1dca46daee8d2817f2ebe49e6335da81956), [`1e9aab5`](https://github.com/mastra-ai/mastra/commit/1e9aab50ff11e6e88fde4d7cbf512c44a9fe8d61), [`2bccba4`](https://github.com/mastra-ai/mastra/commit/2bccba4c03cadc815c2d54cbf4dd43a922140a8d), [`bf8eb6d`](https://github.com/mastra-ai/mastra/commit/bf8eb6d0ec213a403eb9265a594ad283c44ab3dc), [`e9be4e7`](https://github.com/mastra-ai/mastra/commit/e9be4e747ec3d8b65548bff92f9377db06105376), [`493a328`](https://github.com/mastra-ai/mastra/commit/493a328f4346a1deeb9f1e2e44c8f2a3a4d7591b), [`d53cfc2`](https://github.com/mastra-ai/mastra/commit/d53cfc2c7f8d78343a4aa84ec4e129ba25f3325e), [`65799d4`](https://github.com/mastra-ai/mastra/commit/65799d4d549e5ebb9c848fbe3f51ac090f64becf), [`c268c89`](https://github.com/mastra-ai/mastra/commit/c268c89f4c63a93ee474d3cffdf3ea60bf00d4f2), [`34839c1`](https://github.com/mastra-ai/mastra/commit/34839c1910b6964bf59ed0cee58844efebbb684e), [`014e00f`](https://github.com/mastra-ai/mastra/commit/014e00f2b3a597a016b72f9901c6ab27d491f822), [`029a414`](https://github.com/mastra-ai/mastra/commit/029a4141719793bd3e898a39eb5a0466a55f5f3a), [`d468acb`](https://github.com/mastra-ai/mastra/commit/d468acb07aec1bb19a2cb0ada8042b05b46746b2), [`b147b29`](https://github.com/mastra-ai/mastra/commit/b147b2907f0cd1aa812efe6d6e3f58d22e66fc88), [`d371ac1`](https://github.com/mastra-ai/mastra/commit/d371ac1d9820afaaf7cfdbc380a475946a994d8f), [`2bccba4`](https://github.com/mastra-ai/mastra/commit/2bccba4c03cadc815c2d54cbf4dd43a922140a8d), [`0c72f03`](https://github.com/mastra-ai/mastra/commit/0c72f032abb13254df5a7856d64be2f207b8006d), [`cf182b7`](https://github.com/mastra-ai/mastra/commit/cf182b7fb495767946d9840ef29f19cfa906f31f), [`3b45ea9`](https://github.com/mastra-ai/mastra/commit/3b45ea95015557a6cb9d70dc5252af54ab1b78ac), [`a049c2a`](https://github.com/mastra-ai/mastra/commit/a049c2a9dfb41d0ee2e7a28874a88cd64fd5669f), [`f084be1`](https://github.com/mastra-ai/mastra/commit/f084be1fcbe33ad7480913e44d6130c421c0976f), [`b147b29`](https://github.com/mastra-ai/mastra/commit/b147b2907f0cd1aa812efe6d6e3f58d22e66fc88), [`2a96528`](https://github.com/mastra-ai/mastra/commit/2a9652848dfa3c5a2426f952e9d93554c26fd90f), [`f2ab060`](https://github.com/mastra-ai/mastra/commit/f2ab060162bea81505fda553e2cee29c1979fd04), [`5d302c8`](https://github.com/mastra-ai/mastra/commit/5d302c8eda1a6ac74eab5e442c4f64db6cc97a06), [`34839c1`](https://github.com/mastra-ai/mastra/commit/34839c1910b6964bf59ed0cee58844efebbb684e), [`a952852`](https://github.com/mastra-ai/mastra/commit/a952852c971a21fb646cd907c75fcf4443cdc963), [`2656d9c`](https://github.com/mastra-ai/mastra/commit/2656d9c2976d4f3354253bfbbbf9b88a1b2bbf34), [`63e3fe1`](https://github.com/mastra-ai/mastra/commit/63e3fe13cc1ea96f91d7c68aea92f400faf9e4da), [`1d4ce8d`](https://github.com/mastra-ai/mastra/commit/1d4ce8daaa54511f325c1b609d31b8e54009d677), [`8c68372`](https://github.com/mastra-ai/mastra/commit/8c68372e85fe0b066ec12c58bd29ffb93e54c552)]:
|
|
36
|
+
- @mastra/core@1.42.0
|
|
37
|
+
|
|
38
|
+
## 1.10.0-alpha.0
|
|
39
|
+
|
|
40
|
+
### Minor Changes
|
|
41
|
+
|
|
42
|
+
- Added opt-in replicated ClickHouse table support for multi-replica clusters. ([#17684](https://github.com/mastra-ai/mastra/pull/17684))
|
|
43
|
+
|
|
44
|
+
Configure `replication` to have Mastra create replicated MergeTree tables and add `ON CLUSTER` to Mastra-owned DDL when a cluster name is provided:
|
|
45
|
+
|
|
46
|
+
```ts
|
|
47
|
+
new ClickhouseStoreVNext({
|
|
48
|
+
id: 'clickhouse-storage',
|
|
49
|
+
url: process.env.CLICKHOUSE_URL!,
|
|
50
|
+
username: process.env.CLICKHOUSE_USERNAME!,
|
|
51
|
+
password: process.env.CLICKHOUSE_PASSWORD!,
|
|
52
|
+
replication: {
|
|
53
|
+
cluster: 'company_cluster',
|
|
54
|
+
},
|
|
55
|
+
});
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Notes:
|
|
59
|
+
- If existing Mastra tables use local `MergeTree` or `ReplacingMergeTree` engines, initialization fails. Migrate existing local tables to `Replicated*` engines before enabling `replication`.
|
|
60
|
+
- vNext observability signal-table migrations (`migrateSpans()`) are blocked while `replication` is configured. Migrate legacy signal tables before turning on replication.
|
|
61
|
+
- The default `zookeeperPath` is `/clickhouse/tables/{shard}/{database}/{table}`. Override it if your cluster uses a different convention.
|
|
62
|
+
|
|
63
|
+
### Patch Changes
|
|
64
|
+
|
|
65
|
+
- dependencies updates: ([#17148](https://github.com/mastra-ai/mastra/pull/17148))
|
|
66
|
+
- Updated dependency [`@clickhouse/client@^1.20.0` ↗︎](https://www.npmjs.com/package/@clickhouse/client/v/1.20.0) (from `^1.18.2`, in `dependencies`)
|
|
67
|
+
|
|
68
|
+
- Fixed DateTime64 comparison error with ClickHouse 26.5 by removing invalid empty string comparison on nullable DateTime64 columns in span deduplication query ([#17149](https://github.com/mastra-ai/mastra/pull/17149))
|
|
69
|
+
|
|
70
|
+
- Updated dependencies [[`575f815`](https://github.com/mastra-ai/mastra/commit/575f815c5c3567b71c0b83cbb7fa98c8253a9d9c), [`306909a`](https://github.com/mastra-ai/mastra/commit/306909a693de77d709b38706e2673c9547d24a28), [`5191af8`](https://github.com/mastra-ai/mastra/commit/5191af80c799eea25357c545fc05d91b3883531d), [`43bd3d4`](https://github.com/mastra-ai/mastra/commit/43bd3d421987463fdf35386a45199c49499ed069), [`e6fa79e`](https://github.com/mastra-ai/mastra/commit/e6fa79ec72a2ddffdd25e85270398951e9d552a4), [`904bcdf`](https://github.com/mastra-ai/mastra/commit/904bcdf7b8004aa7be823f9f70ca63580e47e470), [`7f5ee1d`](https://github.com/mastra-ai/mastra/commit/7f5ee1dca46daee8d2817f2ebe49e6335da81956), [`1e9aab5`](https://github.com/mastra-ai/mastra/commit/1e9aab50ff11e6e88fde4d7cbf512c44a9fe8d61), [`bf8eb6d`](https://github.com/mastra-ai/mastra/commit/bf8eb6d0ec213a403eb9265a594ad283c44ab3dc), [`493a328`](https://github.com/mastra-ai/mastra/commit/493a328f4346a1deeb9f1e2e44c8f2a3a4d7591b), [`029a414`](https://github.com/mastra-ai/mastra/commit/029a4141719793bd3e898a39eb5a0466a55f5f3a), [`b147b29`](https://github.com/mastra-ai/mastra/commit/b147b2907f0cd1aa812efe6d6e3f58d22e66fc88), [`d371ac1`](https://github.com/mastra-ai/mastra/commit/d371ac1d9820afaaf7cfdbc380a475946a994d8f), [`cf182b7`](https://github.com/mastra-ai/mastra/commit/cf182b7fb495767946d9840ef29f19cfa906f31f), [`a049c2a`](https://github.com/mastra-ai/mastra/commit/a049c2a9dfb41d0ee2e7a28874a88cd64fd5669f), [`b147b29`](https://github.com/mastra-ai/mastra/commit/b147b2907f0cd1aa812efe6d6e3f58d22e66fc88), [`2a96528`](https://github.com/mastra-ai/mastra/commit/2a9652848dfa3c5a2426f952e9d93554c26fd90f), [`2656d9c`](https://github.com/mastra-ai/mastra/commit/2656d9c2976d4f3354253bfbbbf9b88a1b2bbf34), [`63e3fe1`](https://github.com/mastra-ai/mastra/commit/63e3fe13cc1ea96f91d7c68aea92f400faf9e4da), [`1d4ce8d`](https://github.com/mastra-ai/mastra/commit/1d4ce8daaa54511f325c1b609d31b8e54009d677), [`8c68372`](https://github.com/mastra-ai/mastra/commit/8c68372e85fe0b066ec12c58bd29ffb93e54c552)]:
|
|
71
|
+
- @mastra/core@1.42.0-alpha.4
|
|
72
|
+
|
|
3
73
|
## 1.9.1
|
|
4
74
|
|
|
5
75
|
### Patch Changes
|
package/README.md
CHANGED
|
@@ -68,9 +68,35 @@ type ClickhouseConfig = {
|
|
|
68
68
|
url: string; // Clickhouse HTTP interface URL
|
|
69
69
|
username: string; // Database username
|
|
70
70
|
password: string; // Database password
|
|
71
|
+
replication?: {
|
|
72
|
+
cluster?: string; // Adds ON CLUSTER to Mastra-owned DDL when set
|
|
73
|
+
zookeeperPath?: string; // Defaults to '/clickhouse/tables/{shard}/{database}/{table}'
|
|
74
|
+
replicaName?: string; // Defaults to '{replica}'
|
|
75
|
+
};
|
|
71
76
|
};
|
|
72
77
|
```
|
|
73
78
|
|
|
79
|
+
### Replicated ClickHouse clusters
|
|
80
|
+
|
|
81
|
+
Set `replication` when Mastra writes to a multi-replica ClickHouse cluster through a load balancer. Mastra will create its tables with replicated MergeTree engines and add `ON CLUSTER` to Mastra-owned DDL when `cluster` is provided.
|
|
82
|
+
|
|
83
|
+
```typescript
|
|
84
|
+
const store = new ClickhouseStore({
|
|
85
|
+
url: 'http://clickhouse-lb:8123',
|
|
86
|
+
username: 'default',
|
|
87
|
+
password: 'password',
|
|
88
|
+
replication: {
|
|
89
|
+
cluster: 'company_cluster',
|
|
90
|
+
},
|
|
91
|
+
});
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
The default `zookeeperPath` is `/clickhouse/tables/{shard}/{database}/{table}`. If your cluster's existing tables use a different layout (for example `/clickhouse/tables/{shard}/{table}` without the `{database}` segment), set `zookeeperPath` explicitly to match. Mastra does not infer your cluster's convention from Keeper.
|
|
95
|
+
|
|
96
|
+
Manual maintenance such as `optimizeTable()` and `materializeTtl()` runs on every replica when `cluster` is set. These operations can be expensive on a large cluster. Prefer running them outside peak hours.
|
|
97
|
+
|
|
98
|
+
If Mastra finds an existing local `MergeTree` or `ReplacingMergeTree` table while replication is enabled, initialization fails instead of silently mixing local and replicated tables. Migrate existing local tables manually before enabling this option.
|
|
99
|
+
|
|
74
100
|
## Features
|
|
75
101
|
|
|
76
102
|
### Storage Features
|
|
@@ -89,6 +115,7 @@ The store uses different table engines for different types of data:
|
|
|
89
115
|
|
|
90
116
|
- `MergeTree()`: Used for messages, traces, and evals
|
|
91
117
|
- `ReplacingMergeTree()`: Used for threads and workflow snapshots
|
|
118
|
+
- `ReplicatedMergeTree(...)` / `ReplicatedReplacingMergeTree(...)`: Used instead when `replication` is enabled
|
|
92
119
|
|
|
93
120
|
## Storage Methods
|
|
94
121
|
|
package/dist/docs/SKILL.md
CHANGED
|
@@ -8,7 +8,7 @@ ClickHouse is most commonly used as the dedicated observability backend in a [co
|
|
|
8
8
|
|
|
9
9
|
Production observability for traces, logs, metrics, scores, and feedback.
|
|
10
10
|
|
|
11
|
-
For local development, use a composite store that combines [LibSQL](https://mastra.ai/reference/storage/libsql) (for memory and workflows) and `@mastra/duckdb` (for observability). Neither alone covers a development setup: LibSQL
|
|
11
|
+
For local development, use a composite store that combines [LibSQL](https://mastra.ai/reference/storage/libsql) (for memory and workflows) and `@mastra/duckdb` (for observability). Neither alone covers a development setup: LibSQL doesn't implement the observability domain, and DuckDB doesn't implement the other domains. See the [observability overview](https://mastra.ai/docs/observability/overview) for an example.
|
|
12
12
|
|
|
13
13
|
## Installation
|
|
14
14
|
|
|
@@ -44,7 +44,7 @@ You will also need a running ClickHouse server. See [Hosting options](#hosting-o
|
|
|
44
44
|
|
|
45
45
|
`ObservabilityStorageClickhouseVNext` is the current observability domain implementation. It uses an insert-only schema backed by `ReplacingMergeTree` and is optimized for the volume produced by traces, logs, metrics, scores, and feedback.
|
|
46
46
|
|
|
47
|
-
Compose it with another storage adapter so observability writes
|
|
47
|
+
Compose it with another storage adapter so observability writes don't contend with your application data:
|
|
48
48
|
|
|
49
49
|
```typescript
|
|
50
50
|
import { Mastra } from '@mastra/core'
|
|
@@ -81,11 +81,11 @@ export const mastra = new Mastra({
|
|
|
81
81
|
})
|
|
82
82
|
```
|
|
83
83
|
|
|
84
|
-
`MastraStorageExporter` automatically selects the `insert-only` strategy when ClickHouse is the observability backend, which gives the highest write throughput. See [tracing strategies](https://mastra.ai/docs/observability/
|
|
84
|
+
`MastraStorageExporter` automatically selects the `insert-only` strategy when ClickHouse is the observability backend, which gives the highest write throughput. See [tracing strategies](https://mastra.ai/docs/observability/integrations/exporters/mastra-storage) for details.
|
|
85
85
|
|
|
86
86
|
### Observability with the legacy domain
|
|
87
87
|
|
|
88
|
-
`ObservabilityStorageClickhouse` is the original observability adapter and remains supported for projects that
|
|
88
|
+
`ObservabilityStorageClickhouse` is the original observability adapter and remains supported for projects that haven't migrated to the vNext schema. The configuration shape is the same as the vNext class.
|
|
89
89
|
|
|
90
90
|
```typescript
|
|
91
91
|
import { ObservabilityStorageClickhouse } from '@mastra/clickhouse'
|
|
@@ -182,10 +182,55 @@ The same `client` form is accepted by `ObservabilityStorageClickhouse` and `Obse
|
|
|
182
182
|
|
|
183
183
|
**ttl** (`object`): Per-table TTL configuration applied at table creation time. Accepts row-level and column-level TTLs in interval units from \`NANOSECOND\` through \`YEAR\`.
|
|
184
184
|
|
|
185
|
+
**replication** (`{ cluster?: string; zookeeperPath?: string; replicaName?: string }`): Opt-in replicated table configuration for multi-replica ClickHouse clusters. When set, Mastra creates replicated MergeTree tables. When \`cluster\` is set, Mastra also adds \`ON CLUSTER\` to Mastra-owned data definition language (DDL).
|
|
186
|
+
|
|
185
187
|
**disableInit** (`boolean`): When \`true\`, the store does not run table creation or migrations on first use. Call \`storage.init()\` explicitly from your deployment scripts. (Default: `false`)
|
|
186
188
|
|
|
187
189
|
`ClickhouseStore` also accepts every option from `ClickHouseClientConfigOptions` (such as `database`, `request_timeout`, `compression`, `keep_alive`, and `max_open_connections`).
|
|
188
190
|
|
|
191
|
+
### Replicated clusters
|
|
192
|
+
|
|
193
|
+
Use `replication` when Mastra writes to a multi-replica ClickHouse cluster through a load balancer.
|
|
194
|
+
|
|
195
|
+
```typescript
|
|
196
|
+
const storage = new ClickhouseStoreVNext({
|
|
197
|
+
id: 'clickhouse-storage',
|
|
198
|
+
url: process.env.CLICKHOUSE_URL!,
|
|
199
|
+
username: process.env.CLICKHOUSE_USERNAME!,
|
|
200
|
+
password: process.env.CLICKHOUSE_PASSWORD!,
|
|
201
|
+
replication: {
|
|
202
|
+
cluster: 'company_cluster',
|
|
203
|
+
},
|
|
204
|
+
})
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
When `replication` is set, Mastra rewrites its `MergeTree` and `ReplacingMergeTree` table engines to `ReplicatedMergeTree` and `ReplicatedReplacingMergeTree`. The default engine arguments are:
|
|
208
|
+
|
|
209
|
+
- `zookeeperPath`: `'/clickhouse/tables/{shard}/{database}/{table}'`
|
|
210
|
+
- `replicaName`: `'{replica}'`
|
|
211
|
+
|
|
212
|
+
The defaults match the most common self-managed convention. If your cluster's existing tables use a different layout (for example `/clickhouse/tables/{shard}/{table}` without the `{database}` segment), set `zookeeperPath` explicitly to match. Mastra does not read your cluster's convention from Keeper, so a mismatched default writes Mastra's metadata to a separate branch from the rest of the cluster.
|
|
213
|
+
|
|
214
|
+
```typescript
|
|
215
|
+
new ClickhouseStoreVNext({
|
|
216
|
+
url: process.env.CLICKHOUSE_URL!,
|
|
217
|
+
username: process.env.CLICKHOUSE_USERNAME!,
|
|
218
|
+
password: process.env.CLICKHOUSE_PASSWORD!,
|
|
219
|
+
replication: {
|
|
220
|
+
cluster: 'company_cluster',
|
|
221
|
+
zookeeperPath: '/clickhouse/tables/{shard}/{table}',
|
|
222
|
+
},
|
|
223
|
+
})
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
Set `cluster` to add `ON CLUSTER` to Mastra-owned DDL such as table creation, materialized view creation, column migrations, TTL changes, and table drops.
|
|
227
|
+
|
|
228
|
+
Manual maintenance such as `optimizeTable()` and `materializeTtl()` runs on every replica when `cluster` is set. These operations can be expensive on a large cluster. Prefer running them outside peak hours, and let routine merges happen on the background merge queue rather than triggering them on every restart.
|
|
229
|
+
|
|
230
|
+
If existing Mastra tables use local `MergeTree` or `ReplacingMergeTree` engines, initialization fails while `replication` is enabled. Mastra refuses to silently convert local tables because copy-and-swap is unsafe across replicas. To migrate, recreate the affected tables as `Replicated*` before enabling replication. The typical sequence is: rename the local table, run `CREATE TABLE ... ENGINE = ReplicatedMergeTree(...) ON CLUSTER ...`, run `INSERT INTO ... SELECT * FROM <renamed_local>`, then drop the renamed local table.
|
|
231
|
+
|
|
232
|
+
Do not set `replication` on ClickHouse Cloud. Cloud rewrites `MergeTree` to `SharedMergeTree` server-side, and explicit `ReplicatedMergeTree` engines produce incorrect DDL. `replication` is only for self-managed multi-replica clusters.
|
|
233
|
+
|
|
189
234
|
### Observability domain options
|
|
190
235
|
|
|
191
236
|
`ObservabilityStorageClickhouse` and `ObservabilityStorageClickhouseVNext` accept the same connection options as `ClickhouseStore` (`url`, `username`, `password`, or a pre-configured `client`).
|
|
@@ -261,7 +306,7 @@ Two ways to provision the database:
|
|
|
261
306
|
|
|
262
307
|
The same approach applies to other hosts with ephemeral filesystems. For application data that should also live off-host, pair this setup with a managed PostgreSQL or LibSQL/Turso instance for the `default` storage.
|
|
263
308
|
|
|
264
|
-
> **Warning:**
|
|
309
|
+
> **Warning:** Don't point an embedded backend like DuckDB at a path inside an ephemeral container filesystem. Data written there is lost when the container restarts, and on some platforms the path is read-only.
|
|
265
310
|
|
|
266
311
|
## Initialization
|
|
267
312
|
|
|
@@ -290,11 +335,11 @@ ClickHouse is the recommended backend for production observability:
|
|
|
290
335
|
- **Insert-only strategy**: `MastraStorageExporter` writes completed spans in batches without per-span updates, which is the highest-throughput strategy available.
|
|
291
336
|
- **Columnar compression**: Span attributes and log payloads compress well compared to the same data in row-oriented databases.
|
|
292
337
|
|
|
293
|
-
For the full strategy matrix and production guidance, see the [`MastraStorageExporter` reference](https://mastra.ai/docs/observability/
|
|
338
|
+
For the full strategy matrix and production guidance, see the [`MastraStorageExporter` reference](https://mastra.ai/docs/observability/integrations/exporters/mastra-storage).
|
|
294
339
|
|
|
295
340
|
## Related
|
|
296
341
|
|
|
297
342
|
- [Storage overview](https://mastra.ai/reference/storage/overview)
|
|
298
343
|
- [Composite storage](https://mastra.ai/reference/storage/composite)
|
|
299
|
-
- [`MastraStorageExporter`](https://mastra.ai/docs/observability/
|
|
344
|
+
- [`MastraStorageExporter`](https://mastra.ai/docs/observability/integrations/exporters/mastra-storage)
|
|
300
345
|
- [Observability overview](https://mastra.ai/docs/observability/overview)
|
|
@@ -240,6 +240,38 @@ const storage = new MastraCompositeStore({
|
|
|
240
240
|
})
|
|
241
241
|
```
|
|
242
242
|
|
|
243
|
-
> **Note:** `ObservabilityStorageClickhouseVNext` is the current observability domain implementation. The legacy `ObservabilityStorageClickhouse` class is also exported and remains supported for projects that
|
|
243
|
+
> **Note:** `ObservabilityStorageClickhouseVNext` is the current observability domain implementation. The legacy `ObservabilityStorageClickhouse` class is also exported and remains supported for projects that haven't migrated. See the [ClickHouse storage reference](https://mastra.ai/reference/storage/clickhouse) for details.
|
|
244
244
|
|
|
245
|
-
|
|
245
|
+
### Replicated ClickHouse for multi-replica clusters
|
|
246
|
+
|
|
247
|
+
For self-managed ClickHouse clusters with multiple replicas, set `replication` so Mastra emits `ReplicatedMergeTree` engines and applies `ON CLUSTER` to its DDL:
|
|
248
|
+
|
|
249
|
+
```typescript
|
|
250
|
+
import { MastraCompositeStore } from '@mastra/core/storage'
|
|
251
|
+
import { MemoryPG, WorkflowsPG, ScoresPG } from '@mastra/pg'
|
|
252
|
+
import { ObservabilityStorageClickhouseVNext } from '@mastra/clickhouse'
|
|
253
|
+
|
|
254
|
+
const storage = new MastraCompositeStore({
|
|
255
|
+
id: 'composite',
|
|
256
|
+
domains: {
|
|
257
|
+
memory: new MemoryPG({ connectionString: process.env.DATABASE_URL }),
|
|
258
|
+
workflows: new WorkflowsPG({ connectionString: process.env.DATABASE_URL }),
|
|
259
|
+
scores: new ScoresPG({ connectionString: process.env.DATABASE_URL }),
|
|
260
|
+
observability: new ObservabilityStorageClickhouseVNext({
|
|
261
|
+
url: process.env.CLICKHOUSE_URL,
|
|
262
|
+
username: process.env.CLICKHOUSE_USERNAME,
|
|
263
|
+
password: process.env.CLICKHOUSE_PASSWORD,
|
|
264
|
+
replication: {
|
|
265
|
+
cluster: 'production_cluster',
|
|
266
|
+
// Optional (defaults shown):
|
|
267
|
+
// zookeeperPath: '/clickhouse/tables/{shard}/{database}/{table}',
|
|
268
|
+
// replicaName: '{replica}',
|
|
269
|
+
},
|
|
270
|
+
}),
|
|
271
|
+
},
|
|
272
|
+
})
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
Do not set `replication` on ClickHouse Cloud. Cloud rewrites `MergeTree` to `SharedMergeTree` server-side. See the [ClickHouse storage reference](https://mastra.ai/reference/storage/clickhouse) for the full config shape and operator notes.
|
|
276
|
+
|
|
277
|
+
> **Info:** This approach is also required when using storage providers that don't support observability (like Convex, DynamoDB, or Cloudflare). See the [MastraStorageExporter documentation](https://mastra.ai/docs/observability/integrations/exporters/mastra-storage) for the full list of supported providers.
|