@mastra/libsql 1.23.0 → 1.23.1-alpha.1

package/dist/docs/SKILL.md

@@ -3,7 +3,7 @@ name: mastra-libsql
 description: Documentation for @mastra/libsql. Use when working with @mastra/libsql APIs, configuration, or implementation.
 metadata:
   package: "@mastra/libsql"
-  version: "1.23.0"
+  version: "1.23.1-alpha.1"
 ---
 
 ## When to use

package/dist/docs/assets/SOURCE_MAP.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.23.0",
+  "version": "1.23.1-alpha.1",
   "package": "@mastra/libsql",
   "exports": {},
   "modules": {}

package/dist/docs/references/docs-memory-message-history.md

@@ -124,6 +124,27 @@ You can use this history in two ways:
 
 > **Note:** `lastMessages` counts every stored message, including tool calls, tool results, and [signals](https://mastra.ai/docs/harness/signals) of any kind, so a single turn can add several messages to the count. The window also slides forward on every request: once a thread grows past the limit, the oldest message leaves context on each turn, which changes the start of the prompt and invalidates the provider prompt cache. For long-running conversations, use [Observational Memory](https://mastra.ai/docs/memory/observational-memory), which keeps the prompt prefix stable.
 
+### Limit history by tokens
+
+Message count is a poor proxy for context size: a tool result can be a few tokens or thousands. Use `messageHistory` to keep recent history within a token budget instead:
+
+```typescript
+export const agent = new Agent({
+  id: 'test-agent',
+  memory: new Memory({
+    options: {
+      messageHistory: { maxTokens: 8_000, atMaxRemoveTokens: 2_000 },
+    },
+  }),
+})
+```
+
+Mastra counts the complete prompt against `maxTokens`, including remembered history, system instructions, context, and the current turn. When the prompt exceeds the budget, Mastra removes the oldest remembered messages until the prompt is at most `maxTokens - atMaxRemoveTokens`. `atMaxRemoveTokens` defaults to 25% of `maxTokens`. Removing history in chunks keeps the prompt prefix stable across several turns, which helps provider prompt caches stay warm.
+
+System messages, context, the current turn's input, and the agent's responses are never trimmed. Linked tool calls and results are removed together. If protected content alone exceeds `maxTokens`, Mastra removes all remembered history but keeps the protected content.
+
+Trimmed messages stay in storage. During agent runs, Mastra persists a per-thread boundary so they're excluded from later turns. Setting `messageHistory` without `lastMessages` disables the default 10-message cap. Set both to combine a count cap with a token budget. Set `maxTokens` to `0` to disable message history.
+
 > **Tip:** When memory is enabled, [Studio](https://mastra.ai/docs/studio/overview) uses message history to display past conversations in the chat sidebar.
 
 ## Thread title generation

package/dist/docs/references/reference-core-mastra-class.md

@@ -131,7 +131,7 @@ Visit the [Configuration reference](https://mastra.ai/reference/configuration) f
 
 **recovery** (`MastraRecoveryConfig`): Boot-time recovery behavior for orphaned agent and workflow runs. See Crash recovery. (Default: `{ durableAgents: 'off' }`)
 
-**recovery.durableAgents** (`'auto' | 'off'`): Set to 'auto' to automatically re-drive orphaned RUNNING durable agent runs on server boot. Recovery re-issues LLM calls and re-executes tool calls, so tools must be idempotent. See Crash recovery.
+**recovery.durableAgents** (`'auto' | 'off'`): Set to 'auto' to automatically re-drive orphaned RUNNING durable agent runs on server boot. This also controls the default snapshot-persistence policy for durable agents: running checkpoints are only written when set to 'auto' (or when an agent sets a custom shouldPersistSnapshot that includes running). Recovery re-issues LLM calls and re-executes tool calls, so tools must be idempotent. See Crash recovery.
 
 ## Methods
 

package/dist/docs/references/reference-memory-memory-class.md

@@ -41,6 +41,8 @@ export const agent = new Agent({
 
 **options.lastMessages** (`number | false`): Number of most recent messages to include in context. Set to false to disable the message history feature entirely (messages are not loaded into context or saved). Use Number.MAX\_SAFE\_INTEGER to retrieve all messages with no limit. To load messages without saving new ones, use the readOnly option. The window slides forward on every request, so once a thread exceeds the limit, each turn invalidates the provider prompt cache. For long-running conversations, use Observational Memory instead.
 
+**options.messageHistory** (`{ maxTokens: number; atMaxRemoveTokens?: number }`): Token budget for the complete prompt, including remembered history, system instructions, context, and the current turn. When the prompt exceeds maxTokens, the oldest remembered messages are dropped until the prompt is at most maxTokens - atMaxRemoveTokens (defaults to 25% of maxTokens). Protected content is never removed. Linked tool calls and results are removed together. During agent runs, trimmed history is excluded from later turns via a persisted per-thread boundary, but the messages themselves stay in storage. When set without an explicit lastMessages, the default 10-message cap is not applied. Set maxTokens to 0 to disable message history. Prefer this over lastMessages, since message count is a poor proxy for context size.
+
 **options.readOnly** (`boolean`): When true, prevents memory from saving new messages and provides working memory as read-only context (without the updateWorkingMemory tool). Useful for read-only operations like previews, internal routing agents, or sub agents that should reference but not modify memory.
 
 **options.semanticRecall** (`boolean | { topK: number; messageRange: number | { before: number; after: number }; scope?: 'thread' | 'resource' }`): Enable semantic search in message history. Can be a boolean or an object with configuration options. When enabled, requires both vector store and embedder to be configured. Default topK is 4, default messageRange is {before: 1, after: 1}.

package/dist/docs/references/reference-storage-retention.md

@@ -6,11 +6,37 @@
 
 Because storage grows without bound by default, Mastra provides an opt-in, age-based retention system. Declare per-table `maxAge` policies in the `retention` config, then call `storage.prune()` to delete rows older than their configured age. Unconfigured data is kept forever, so behavior doesn't change until you opt in.
 
-`prune()` deletes rows. It caps growth and is safe to run against large tables (batched, bounded, resumable, cancellable). It never reclaims disk: on SQLite/libSQL the freed pages are reused by future writes so the file stops growing, but handing disk back to the OS (for example a `VACUUM`) is left to the underlying database and the operator to manage.
+`prune()` deletes rows in bounded batches. Runs are resumable and cancellable, so you can limit how much work each maintenance window performs. Pruning doesn't reclaim disk space by itself. Use the database-specific maintenance guidance below when you need to return freed space to the operating system.
 
 Retention covers **growth tables** only: tables that accumulate rows unbounded as a side effect of normal operation (conversation history, telemetry, job and run records, schedule fire history, event feeds). User-authored artifacts and config (agents, skills, workspaces, prompt blocks, datasets, schedule definitions, channel installations, and so on) grow with user intent and are edited or deleted explicitly, so they're not valid retention keys.
 
-The reference implementations are [libSQL](https://mastra.ai/integrations/databases/libsql), [PostgreSQL](https://mastra.ai/integrations/databases/postgresql), and [MongoDB](https://mastra.ai/integrations/databases/mongodb). Other adapters keep rows forever until they implement retention.
+Storage adapters use the shared core retention contract for `prune()`, or a database-native mechanism when that better matches the backend.
+
+| Adapter              | Mechanism               | Retention support                                                                                                                                                                      |
+| -------------------- | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| libSQL               | `prune()`               | All supported growth domains                                                                                                                                                           |
+| PostgreSQL           | `prune()`               | All supported growth domains. V-next observability drops expired partitions or chunks                                                                                                  |
+| MongoDB              | `prune()` or native TTL | All supported growth domains. Native TTL indexes are also available                                                                                                                    |
+| DuckDB               | `prune()`               | Observability spans, metrics, logs, scores, and feedback                                                                                                                               |
+| MySQL                | `prune()`               | Observability spans                                                                                                                                                                    |
+| Microsoft SQL Server | `prune()`               | Observability spans                                                                                                                                                                    |
+| Oracle Database      | `prune()`               | Observability spans and logs                                                                                                                                                           |
+| Amazon Aurora DSQL   | `prune()`               | Observability spans                                                                                                                                                                    |
+| Google Cloud Spanner | `prune()`               | Observability spans, plus metrics when metrics storage is enabled                                                                                                                      |
+| ClickHouse           | Native TTL              | Observability spans, metrics, logs, scores, and feedback. When all five signals have finite retention, deletion-request records expire after the longest signal retention plus 30 days |
+
+## Storage-specific maintenance
+
+| Adapter           | Maintenance guidance                                                                                                                            |
+| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
+| SQLite and libSQL | Freed pages are reused by future writes, which stops the database file from growing. Reclaiming disk space requires database-level maintenance. |
+| DuckDB            | For file-backed stores, run `CHECKPOINT` after pruning to reclaim deleted rows in storage. DuckDB's `VACUUM` doesn't reclaim deleted rows.      |
+
+## Schedule pruning
+
+Run `prune()` from a scheduler or maintenance worker, not from application startup or shutdown hooks. For deployments that share a database, prefer a single active scheduler or worker for pruning.
+
+Prefer lower-traffic periods when pruning large tables. Use `maxBatches`, `maxRows`, and `pauseMs` to bound each run, and pass an `AbortSignal` when the maintenance process needs to stop promptly. These recommendations apply to adapters that expose `prune()`. ClickHouse applies its native time to live (TTL) policy within the database.
 
 ## Usage example
 
@@ -94,7 +120,8 @@ Each domain specifies its age-prunable tables and the timestamp column that anch
 > - Experiments prune as whole units: an aged experiment's result rows are deleted together with it (results cascade with their parent), so a run is never left partially deleted. Retention doesn't have a separate `results` key.
 > - For `schedules`, the growth table is the fire history (`schedule_triggers`, one row per fire): schedule definitions are config and aren't pruned.
 > - On PostgreSQL, timestamp anchors use the timezone-aware mirror columns (for example `createdAtZ`, `completedAtZ`).
-> - LibSQL and PostgreSQL support all domains above except `harness`, which PostgreSQL doesn't implement. MongoDB supports all except `threadState` and `harness`.
+> - DuckDB observability stores append-only events for all five signals. Its `spans` policy uses the event `timestamp` column rather than `startedAt`.
+> - LibSQL and PostgreSQL support all domains above except `harness`, which PostgreSQL doesn't implement. MongoDB supports all except `threadState` and `harness`. DuckDB, MySQL, Microsoft SQL Server, Oracle Database, Amazon Aurora DSQL, and Google Cloud Spanner currently support retention only in their `observability` domains, with the signal coverage shown in the support matrix.
 > - The v-next PostgreSQL observability domain stores signal events in day-partitioned tables (`spans`, `metrics`, `logs`, `scores`, `feedback`). For it, `prune()` drops whole day partitions (or TimescaleDB chunks) that are entirely older than the cutoff instead of deleting rows: effective level of detail is one day, and a partition is only dropped once its entire day is past `maxAge`. `PruneResult.deleted` reports the number of rows in the dropped partitions.
 
 ## Methods
@@ -109,7 +136,7 @@ Deletes rows older than their configured `maxAge` across every domain that has a
 
 Pass `options.retention` to replace the configured policies for that call only: for example to skip a domain (keep chat history) or prune more aggressively than the standing config. The store's configured `retention` is unchanged.
 
-Anchor-column indexes are created lazily on the first `prune()` call for each table with a policy (never at `init()`) so deployments that don't configure retention pay no extra index write or disk overhead. The first prune of an existing large table pays a one-time index build. Subsequent prunes reuse the index.
+Adapters that use anchor-column indexes create them lazily on the first `prune()` call for each table with a policy (never at `init()`) so deployments that don't configure retention pay no extra index write or disk overhead. The first prune of an existing large table pays a one-time index build. Subsequent prunes reuse the index. DuckDB uses its built-in zone maps instead of creating retention indexes.
 
 ```typescript
 const results = await storage.prune({
@@ -177,6 +204,31 @@ async function retentionTick() {
 
 You can also cancel a long-running prune with an `AbortSignal`: the loop stops between batches and returns partial results with `done: false`, so the next run resumes cleanly.
 
+## ClickHouse native TTL
+
+ClickHouse observability storage uses native table TTLs instead of `prune()`. Configure retention as days per signal. `init()` applies the TTLs to new and existing tables and skips `ALTER TABLE` statements when the configured TTL is already present.
+
+For deployments that need to update TTL configuration without running the full initialization path, call `applyRetention()` on the v-next observability store:
+
+```typescript
+import { ObservabilityStorageClickhouseVNext } from '@mastra/clickhouse'
+
+const observability = new ObservabilityStorageClickhouseVNext({
+  client,
+  retention: {
+    tracing: 30,
+    logs: 7,
+    metrics: 14,
+    scores: 90,
+    feedback: 60,
+  },
+})
+
+await observability.applyRetention()
+```
+
+Deletion requests are retained long enough to keep enforcing erasure after signal rows expire. Mastra applies a TTL to `mastra_deletion_requests` only when tracing, logs, metrics, scores, and feedback all have finite retention. The deletion-request TTL is the longest of those periods plus 30 days. For example, if score retention is the longest period at 90 days, deletion requests expire after 120 days. When any signal is unbounded, deletion requests remain unbounded because trace deletion requests cover rows across all five signals.
+
 ## MongoDB TTL indexes (alternative to prune)
 
 MongoDB offers native [TTL (Time-To-Live) indexes](https://www.mongodb.com/docs/manual/core/index-ttl/) that automatically delete expired documents without requiring manual `prune()` calls. This is a database-level feature that runs as a background thread.

package/dist/index.cjs

@@ -2873,13 +2873,6 @@ var AgentsLibSQL = class extends _mastra_core_storage.AgentsStorage {
 };
 //#endregion
 //#region src/storage/retention.ts
-const DEFAULT_BATCH_SIZE = 1e3;
-/**
-* Lazily create the single-column anchor index a prune target relies on.
-* Called from the prune path (not init) so only deployments that actually
-* configure retention pay the index's write/disk overhead. Best-effort: a
-* failure is logged and pruning proceeds (correct, just slower).
-*/
 async function ensureAnchorIndex(db, target, logger) {
 	if (!target.indexed) return;
 	try {
@@ -2892,134 +2885,32 @@ async function ensureAnchorIndex(db, target, logger) {
 		logger?.warn?.(`Failed to ensure retention index on ${target.table}(${target.column}):`, error);
 	}
 }
-async function sleep(ms, signal) {
-	if (ms <= 0 || signal?.aborted) return;
-	await new Promise((resolve) => {
-		const timer = setTimeout(() => {
-			signal?.removeEventListener("abort", onAbort);
-			resolve();
-		}, ms);
-		const onAbort = () => {
-			clearTimeout(timer);
-			resolve();
-		};
-		signal?.addEventListener("abort", onAbort, { once: true });
-	});
-}
-/**
-* Runs the bounded, batched, cancellable delete loop for a set of tables in the
-* given order (callers pass children before parents for cascade-safe pruning),
-* and returns one {@link PruneResult} per table.
-*
-* The loop:
-* - deletes in chunks of `batchSize` (default 1000), each its own statement;
-* - stops a table's loop when a batch deletes fewer rows than requested (drained),
-*   or when `maxBatches`/`maxRows` is hit, or the `signal` aborts — the latter
-*   three leave `done: false` so the caller can resume;
-* - pauses `pauseMs` between batches when set, to avoid starving live traffic.
-*
-* `prune()` only deletes rows; it never reclaims disk. Freed pages are reused
-* by future writes so the file stops growing. Handing disk back to the OS is
-* left to the underlying database and the operator to manage.
-*/
-/** Convert a policy's `maxAge` into a cutoff bound matching the anchor's storage type. */
 function cutoffFor(policy, anchorType, now = Date.now()) {
-	const cutoffMs = now - (0, _mastra_core_storage.parseDuration)(policy.maxAge);
+	const cutoffMs = (0, _mastra_core_storage.retentionCutoffMs)(policy, now);
 	return anchorType === "epoch-ms" ? cutoffMs : new Date(cutoffMs).toISOString();
 }
-/**
-* Run the bounded/cancellable batched-delete loop for a single logical target,
-* delegating the actual delete of up to `limit` rows to `deleteBatch`. Returns
-* `{ deleted, done }`; `done: false` means the loop stopped on a bound or the
-* abort signal and eligible rows may remain.
-*/
-async function runBatchedDelete({ deleteBatch, batchSize, options }) {
-	if (!Number.isSafeInteger(batchSize) || batchSize <= 0) throw new Error(`retention batchSize must be a positive integer; received ${batchSize}`);
-	let deleted = 0;
-	let batches = 0;
-	while (true) {
-		if (options?.signal?.aborted) return {
-			deleted,
-			done: false
-		};
-		if (options?.maxBatches !== void 0 && batches >= options.maxBatches) return {
-			deleted,
-			done: false
-		};
-		let limit = batchSize;
-		if (options?.maxRows !== void 0) {
-			const remaining = options.maxRows - deleted;
-			if (remaining <= 0) return {
-				deleted,
-				done: false
-			};
-			limit = Math.min(limit, remaining);
-		}
-		const affected = await deleteBatch(limit);
-		deleted += affected;
-		batches += 1;
-		if (affected < limit) return {
-			deleted,
-			done: true
-		};
-		if (options?.pauseMs) await sleep(options.pauseMs, options.signal);
-	}
-}
-async function runPrune({ db, domain, targets, options, logger }) {
-	const results = [];
-	const now = Date.now();
-	for (const target of targets) {
-		if (options?.signal?.aborted) {
-			results.push({
-				domain,
-				table: target.table,
-				deleted: 0,
-				done: false
-			});
-			continue;
-		}
-		await ensureAnchorIndex(db, target, logger);
-		const cutoff = cutoffFor(target.policy, target.anchorType, now);
-		const { deleted, done } = await runBatchedDelete({
-			deleteBatch: (limit) => db.pruneBatch({
-				tableName: target.table,
-				column: target.column,
-				cutoff,
-				limit
-			}),
-			batchSize: target.policy.batchSize ?? DEFAULT_BATCH_SIZE,
-			options
-		});
-		results.push({
-			domain,
-			table: target.table,
-			deleted,
-			done
-		});
-	}
-	return results;
+const runBatchedDelete = _mastra_core_storage.runRetentionBatches;
+function runPrune({ db, domain, targets, options, logger }) {
+	return (0, _mastra_core_storage.executeRetentionPrune)({
+		domain,
+		targets,
+		options,
+		beforeTarget: (target) => ensureAnchorIndex(db, target, logger),
+		cutoffFor: (target, now) => cutoffFor(target.policy, target.anchorType ?? "timestamp", now),
+		deleteBatch: (target, cutoff, limit) => db.pruneBatch({
+			tableName: target.table,
+			column: target.column,
+			cutoff,
+			limit
+		})
+	});
 }
-/**
-* Resolve a domain's `{ tableKey: policy }` map plus its descriptor into an
-* ordered list of {@link PruneTarget}s. `order` lists table keys children-first
-* so cascade-dependent rows are removed before their parents. Table keys not in
-* `policies` are skipped (unset = keep forever).
-*/
 function resolveTargets({ policies, descriptor, order }) {
-	const targets = [];
-	for (const key of order) {
-		const policy = policies[key];
-		const entry = descriptor[key];
-		if (!policy || !entry) continue;
-		targets.push({
-			table: entry.table,
-			column: entry.column,
-			anchorType: entry.anchorType ?? "timestamp",
-			indexed: entry.indexed ?? true,
-			policy
-		});
-	}
-	return targets;
+	return (0, _mastra_core_storage.resolveRetentionTargets)({
+		policies,
+		descriptor,
+		order
+	});
 }
 //#endregion
 //#region src/storage/domains/background-tasks/index.ts
@@ -14408,10 +14299,15 @@ var LibSQLFactoryStorageOps = class {
 		const filter = this.#buildWhere(schema, where);
 		const sql = `UPDATE "${schema.name}" SET ${columns.map((c) => `"${c}" = ?`).join(", ")} WHERE ${filter.sql}`;
 		const args = [...columns.map((column) => this.#serialize(this.#column(schema, column), set[column])), ...filter.args];
-		return (await this.#client.execute({
-			sql,
-			args
-		})).rowsAffected;
+		try {
+			return (await this.#client.execute({
+				sql,
+				args
+			})).rowsAffected;
+		} catch (error) {
+			if (isUniqueViolation(error)) throw new _mastra_core_storage.UniqueViolationError(collection, { cause: error });
+			throw error;
+		}
 	}
 	async updateMany(collection, where, set) {
 		return this.#withWriteLock(() => this.#updateMany(collection, where, set));