@mastra/libsql 1.0.0-beta.8 → 1.0.0

package/dist/docs/memory/05-memory-processors.md

@@ -0,0 +1,318 @@
+> Learn how to use memory processors in Mastra to filter, trim, and transform messages before they
+
+# Memory Processors
+
+Memory processors transform and filter messages as they pass through an agent with memory enabled. They manage context window limits, remove unnecessary content, and optimize the information sent to the language model.
+
+When memory is enabled on an agent, Mastra adds memory processors to the agent's processor pipeline. These processors retrieve message history, working memory, and semantically relevant messages, then persist new messages after the model responds.
+
+Memory processors are [processors](https://mastra.ai/docs/v1/agents/processors) that operate specifically on memory-related messages and state.
+
+## Built-in Memory Processors
+
+Mastra automatically adds these processors when memory is enabled:
+
+### MessageHistory
+
+Retrieves message history and persists new messages.
+
+**When you configure:**
+
+```typescript
+memory: new Memory({
+  lastMessages: 10,
+});
+```
+
+**Mastra internally:**
+
+1. Creates a `MessageHistory` processor with `limit: 10`
+2. Adds it to the agent's input processors (runs before the LLM)
+3. Adds it to the agent's output processors (runs after the LLM)
+
+**What it does:**
+
+- **Input**: Fetches the last 10 messages from storage and prepends them to the conversation
+- **Output**: Persists new messages to storage after the model responds
+
+**Example:**
+
+```typescript
+import { Agent } from "@mastra/core/agent";
+import { Memory } from "@mastra/memory";
+import { LibSQLStore } from "@mastra/libsql";
+import { openai } from "@ai-sdk/openai";
+
+const agent = new Agent({
+  id: "test-agent",
+  name: "Test Agent",
+  instructions: "You are a helpful assistant",
+  model: 'openai/gpt-4o',
+  memory: new Memory({
+    storage: new LibSQLStore({
+      id: "memory-store",
+      url: "file:memory.db",
+    }),
+    lastMessages: 10, // MessageHistory processor automatically added
+  }),
+});
+```
+
+### SemanticRecall
+
+Retrieves semantically relevant messages based on the current input and creates embeddings for new messages.
+
+**When you configure:**
+
+```typescript
+memory: new Memory({
+  semanticRecall: { enabled: true },
+  vector: myVectorStore,
+  embedder: myEmbedder,
+});
+```
+
+**Mastra internally:**
+
+1. Creates a `SemanticRecall` processor
+2. Adds it to the agent's input processors (runs before the LLM)
+3. Adds it to the agent's output processors (runs after the LLM)
+4. Requires both a vector store and embedder to be configured
+
+**What it does:**
+
+- **Input**: Performs vector similarity search to find relevant past messages and prepends them to the conversation
+- **Output**: Creates embeddings for new messages and stores them in the vector store for future retrieval
+
+**Example:**
+
+```typescript
+import { Agent } from "@mastra/core/agent";
+import { Memory } from "@mastra/memory";
+import { LibSQLStore } from "@mastra/libsql";
+import { PineconeVector } from "@mastra/pinecone";
+import { OpenAIEmbedder } from "@mastra/openai";
+import { openai } from "@ai-sdk/openai";
+
+const agent = new Agent({
+  name: "semantic-agent",
+  instructions: "You are a helpful assistant with semantic memory",
+  model: 'openai/gpt-4o',
+  memory: new Memory({
+    storage: new LibSQLStore({
+      id: "memory-store",
+      url: "file:memory.db",
+    }),
+    vector: new PineconeVector({
+      id: "memory-vector",
+      apiKey: process.env.PINECONE_API_KEY!,
+    }),
+    embedder: new OpenAIEmbedder({
+      model: "text-embedding-3-small",
+      apiKey: process.env.OPENAI_API_KEY!,
+    }),
+    semanticRecall: { enabled: true }, // SemanticRecall processor automatically added
+  }),
+});
+```
+
+### WorkingMemory
+
+Manages working memory state across conversations.
+
+**When you configure:**
+
+```typescript
+memory: new Memory({
+  workingMemory: { enabled: true },
+});
+```
+
+**Mastra internally:**
+
+1. Creates a `WorkingMemory` processor
+2. Adds it to the agent's input processors (runs before the LLM)
+3. Requires a storage adapter to be configured
+
+**What it does:**
+
+- **Input**: Retrieves working memory state for the current thread and prepends it to the conversation
+- **Output**: No output processing
+
+**Example:**
+
+```typescript
+import { Agent } from "@mastra/core/agent";
+import { Memory } from "@mastra/memory";
+import { LibSQLStore } from "@mastra/libsql";
+import { openai } from "@ai-sdk/openai";
+
+const agent = new Agent({
+  name: "working-memory-agent",
+  instructions: "You are an assistant with working memory",
+  model: 'openai/gpt-4o',
+  memory: new Memory({
+    storage: new LibSQLStore({
+      id: "memory-store",
+      url: "file:memory.db",
+    }),
+    workingMemory: { enabled: true }, // WorkingMemory processor automatically added
+  }),
+});
+```
+
+## Manual Control and Deduplication
+
+If you manually add a memory processor to `inputProcessors` or `outputProcessors`, Mastra will **not** automatically add it. This gives you full control over processor ordering:
+
+```typescript
+import { Agent } from "@mastra/core/agent";
+import { Memory } from "@mastra/memory";
+import { MessageHistory } from "@mastra/core/processors";
+import { TokenLimiter } from "@mastra/core/processors";
+import { LibSQLStore } from "@mastra/libsql";
+import { openai } from "@ai-sdk/openai";
+
+// Custom MessageHistory with different configuration
+const customMessageHistory = new MessageHistory({
+  storage: new LibSQLStore({ id: "memory-store", url: "file:memory.db" }),
+  lastMessages: 20,
+});
+
+const agent = new Agent({
+  name: "custom-memory-agent",
+  instructions: "You are a helpful assistant",
+  model: 'openai/gpt-4o',
+  memory: new Memory({
+    storage: new LibSQLStore({ id: "memory-store", url: "file:memory.db" }),
+    lastMessages: 10, // This would normally add MessageHistory(10)
+  }),
+  inputProcessors: [
+    customMessageHistory, // Your custom one is used instead
+    new TokenLimiter({ limit: 4000 }), // Runs after your custom MessageHistory
+  ],
+});
+```
+
+## Processor Execution Order
+
+Understanding the execution order is important when combining guardrails with memory:
+
+### Input Processors
+
+```
+[Memory Processors] → [Your inputProcessors]
+```
+
+1. **Memory processors run FIRST**: `WorkingMemory`, `MessageHistory`, `SemanticRecall`
+2. **Your input processors run AFTER**: guardrails, filters, validators
+
+This means memory loads message history before your processors can validate or filter the input.
+
+### Output Processors
+
+```
+[Your outputProcessors] → [Memory Processors]
+```
+
+1. **Your output processors run FIRST**: guardrails, filters, validators
+2. **Memory processors run AFTER**: `SemanticRecall` (embeddings), `MessageHistory` (persistence)
+
+This ordering is designed to be **safe by default**: if your output guardrail calls `abort()`, the memory processors never run and **no messages are saved**.
+
+## Guardrails and Memory
+
+The default execution order provides safe guardrail behavior:
+
+### Output guardrails (recommended)
+
+Output guardrails run **before** memory processors save messages. If a guardrail aborts:
+
+- The tripwire is triggered
+- Memory processors are skipped
+- **No messages are persisted to storage**
+
+```typescript
+import { Agent } from "@mastra/core/agent";
+import { Memory } from "@mastra/memory";
+import { openai } from "@ai-sdk/openai";
+
+// Output guardrail that blocks inappropriate content
+const contentBlocker = {
+  id: "content-blocker",
+  processOutputResult: async ({ messages, abort }) => {
+    const hasInappropriateContent = messages.some((msg) =>
+      containsBadContent(msg)
+    );
+    if (hasInappropriateContent) {
+      abort("Content blocked by guardrail");
+    }
+    return messages;
+  },
+};
+
+const agent = new Agent({
+  name: "safe-agent",
+  instructions: "You are a helpful assistant",
+  model: 'openai/gpt-4o',
+  memory: new Memory({ lastMessages: 10 }),
+  // Your guardrail runs BEFORE memory saves
+  outputProcessors: [contentBlocker],
+});
+
+// If the guardrail aborts, nothing is saved to memory
+const result = await agent.generate("Hello");
+if (result.tripwire) {
+  console.log("Blocked:", result.tripwire.reason);
+  // Memory is empty - no messages were persisted
+}
+```
+
+### Input guardrails
+
+Input guardrails run **after** memory processors load history. If a guardrail aborts:
+
+- The tripwire is triggered
+- The LLM is never called
+- Output processors (including memory persistence) are skipped
+- **No messages are persisted to storage**
+
+```typescript
+// Input guardrail that validates user input
+const inputValidator = {
+  id: "input-validator",
+  processInput: async ({ messages, abort }) => {
+    const lastUserMessage = messages.findLast((m) => m.role === "user");
+    if (isInvalidInput(lastUserMessage)) {
+      abort("Invalid input detected");
+    }
+    return messages;
+  },
+};
+
+const agent = new Agent({
+  name: "validated-agent",
+  instructions: "You are a helpful assistant",
+  model: 'openai/gpt-4o',
+  memory: new Memory({ lastMessages: 10 }),
+  // Your guardrail runs AFTER memory loads history
+  inputProcessors: [inputValidator],
+});
+```
+
+### Summary
+
+| Guardrail Type | When it runs | If it aborts |
+| -------------- | ------------ | ------------ |
+| Input | After memory loads history | LLM not called, nothing saved |
+| Output | Before memory saves | Nothing saved to storage |
+
+Both scenarios are safe - guardrails prevent inappropriate content from being persisted to memory
+
+## Related documentation
+
+- [Processors](https://mastra.ai/docs/v1/agents/processors) - General processor concepts and custom processor creation
+- [Guardrails](https://mastra.ai/docs/v1/agents/guardrails) - Security and validation processors
+- [Memory Overview](https://mastra.ai/docs/v1/memory/overview) - Memory types and configuration
+
+When creating custom processors avoid mutating the input `messages` array or its objects directly.

package/dist/docs/memory/06-reference.md

@@ -0,0 +1,133 @@
+# Memory API Reference
+
+> API reference for memory - 1 entries
+
+
+---
+
+## Reference: Memory Class
+
+> Documentation for the `Memory` class in Mastra, which provides a robust system for managing conversation history and thread-based message storage.
+
+The `Memory` class provides a robust system for managing conversation history and thread-based message storage in Mastra. It enables persistent storage of conversations, semantic search capabilities, and efficient message retrieval. You must configure a storage provider for conversation history, and if you enable semantic recall you will also need to provide a vector store and embedder.
+
+## Usage example
+
+```typescript title="src/mastra/agents/test-agent.ts"
+import { Memory } from "@mastra/memory";
+import { Agent } from "@mastra/core/agent";
+
+export const agent = new Agent({
+  name: "test-agent",
+  instructions: "You are an agent with memory.",
+  model: "openai/gpt-5.1",
+  memory: new Memory({
+    options: {
+      workingMemory: {
+        enabled: true,
+      },
+    },
+  }),
+});
+```
+
+> To enable `workingMemory` on an agent, you’ll need a storage provider configured on your main Mastra instance. See [Mastra class](../core/mastra-class) for more information.
+
+## Constructor parameters
+
+### Options parameters
+
+## Returns
+
+## Extended usage example
+
+```typescript title="src/mastra/agents/test-agent.ts"
+import { Memory } from "@mastra/memory";
+import { Agent } from "@mastra/core/agent";
+import { LibSQLStore, LibSQLVector } from "@mastra/libsql";
+
+export const agent = new Agent({
+  name: "test-agent",
+  instructions: "You are an agent with memory.",
+  model: "openai/gpt-5.1",
+  memory: new Memory({
+    storage: new LibSQLStore({
+      id: 'test-agent-storage',
+      url: "file:./working-memory.db",
+    }),
+    vector: new LibSQLVector({
+      id: 'test-agent-vector',
+      url: "file:./vector-memory.db",
+    }),
+    options: {
+      lastMessages: 10,
+      semanticRecall: {
+        topK: 3,
+        messageRange: 2,
+        scope: "resource",
+      },
+      workingMemory: {
+        enabled: true,
+      },
+      generateTitle: true,
+    },
+  }),
+});
+```
+
+## PostgreSQL with index configuration
+
+```typescript title="src/mastra/agents/pg-agent.ts"
+import { Memory } from "@mastra/memory";
+import { Agent } from "@mastra/core/agent";
+import { ModelRouterEmbeddingModel } from "@mastra/core/llm";
+import { PgStore, PgVector } from "@mastra/pg";
+
+export const agent = new Agent({
+  name: "pg-agent",
+  instructions: "You are an agent with optimized PostgreSQL memory.",
+  model: "openai/gpt-5.1",
+  memory: new Memory({
+    storage: new PgStore({
+      id: 'pg-agent-storage',
+      connectionString: process.env.DATABASE_URL,
+    }),
+    vector: new PgVector({
+      id: 'pg-agent-vector',
+      connectionString: process.env.DATABASE_URL,
+    }),
+    embedder: new ModelRouterEmbeddingModel("openai/text-embedding-3-small"),
+    options: {
+      lastMessages: 20,
+      semanticRecall: {
+        topK: 5,
+        messageRange: 3,
+        scope: "resource",
+        indexConfig: {
+          type: "hnsw", // Use HNSW for better performance
+          metric: "dotproduct", // Optimal for OpenAI embeddings
+          m: 16, // Number of bi-directional links
+          efConstruction: 64, // Construction-time candidate list size
+        },
+      },
+      workingMemory: {
+        enabled: true,
+      },
+    },
+  }),
+});
+```
+
+### Related
+
+- [Getting Started with Memory](https://mastra.ai/docs/v1/memory/overview)
+- [Semantic Recall](https://mastra.ai/docs/v1/memory/semantic-recall)
+- [Working Memory](https://mastra.ai/docs/v1/memory/working-memory)
+- [Memory Processors](https://mastra.ai/docs/v1/memory/memory-processors)
+- [createThread](https://mastra.ai/reference/v1/memory/createThread)
+- [recall](https://mastra.ai/reference/v1/memory/recall)
+- [getThreadById](https://mastra.ai/reference/v1/memory/getThreadById)
+- [listThreads](https://mastra.ai/reference/v1/memory/listThreads)
+- [deleteMessages](https://mastra.ai/reference/v1/memory/deleteMessages)
+- [cloneThread](https://mastra.ai/reference/v1/memory/cloneThread)
+- [Clone Utility Methods](https://mastra.ai/reference/v1/memory/clone-utilities)

package/dist/docs/observability/01-overview.md

@@ -0,0 +1,64 @@
+> Monitor and debug applications with Mastra
+
+# Observability Overview
+
+Mastra provides observability features for AI applications. Monitor LLM operations, trace agent decisions, and debug complex workflows with tools that understand AI-specific patterns.
+
+## Key Features
+
+### Tracing
+
+Specialized tracing for AI operations that captures:
+
+- **Model interactions**: Token usage, latency, prompts, and completions
+- **Agent execution**: Decision paths, tool calls, and memory operations
+- **Workflow steps**: Branching logic, parallel execution, and step outputs
+- **Automatic instrumentation**: Tracing with decorators
+
+## Quick Start
+
+Configure Observability in your Mastra instance:
+
+```typescript title="src/mastra/index.ts"
+import { Mastra } from "@mastra/core";
+import { PinoLogger } from "@mastra/loggers";
+import { LibSQLStore } from "@mastra/libsql";
+import {
+  Observability,
+  DefaultExporter,
+  CloudExporter,
+  SensitiveDataFilter,
+} from "@mastra/observability";
+
+export const mastra = new Mastra({
+  logger: new PinoLogger(),
+  storage: new LibSQLStore({
+    id: 'mastra-storage',
+    url: "file:./mastra.db", // Storage is required for tracing
+  }),
+  observability: new Observability({
+    configs: {
+      default: {
+        serviceName: "mastra",
+        exporters: [
+          new DefaultExporter(), // Persists traces to storage for Mastra Studio
+          new CloudExporter(), // Sends traces to Mastra Cloud (if MASTRA_CLOUD_ACCESS_TOKEN is set)
+        ],
+        spanOutputProcessors: [
+          new SensitiveDataFilter(), // Redacts sensitive data like passwords, tokens, keys
+        ],
+      },
+    },
+  }),
+});
+```
+
+With this basic setup, you will see Traces and Logs in both Studio and in Mastra Cloud.
+
+We also support various external tracing providers like MLflow, Langfuse, Braintrust, and any OpenTelemetry-compatible platform (Datadog, New Relic, SigNoz, etc.). See more about this in the [Tracing](https://mastra.ai/docs/v1/observability/tracing/overview) documentation.
+
+## What's Next?
+
+- **[Set up Tracing](https://mastra.ai/docs/v1/observability/tracing/overview)**: Configure tracing for your application
+- **[Configure Logging](https://mastra.ai/docs/v1/observability/logging)**: Add structured logging
+- **[API Reference](https://mastra.ai/reference/v1/observability/tracing/instances)**: Detailed configuration options

package/dist/docs/observability/02-default.md

@@ -0,0 +1,177 @@
+> Store traces locally for development and debugging
+
+# Default Exporter
+
+The `DefaultExporter` persists traces to your configured storage backend, making them accessible through Studio. It's automatically enabled when using the default observability configuration and requires no external services.
+
+## Configuration
+
+### Prerequisites
+
+1. **Storage Backend**: Configure a storage provider (libSQL, PostgreSQL, etc.)
+2. **Studio**: Install for viewing traces locally
+
+### Basic Setup
+
+```typescript title="src/mastra/index.ts"
+import { Mastra } from "@mastra/core";
+import { Observability, DefaultExporter } from "@mastra/observability";
+import { LibSQLStore } from "@mastra/libsql";
+
+export const mastra = new Mastra({
+  storage: new LibSQLStore({
+    id: 'mastra-storage',
+    url: "file:./mastra.db", // Required for trace persistence
+  }),
+  observability: new Observability({
+    configs: {
+      local: {
+        serviceName: "my-service",
+        exporters: [new DefaultExporter()],
+      },
+    },
+  }),
+});
+```
+
+### Recommended Configuration
+
+Include DefaultExporter in your observability configuration:
+
+```typescript
+import { Mastra } from "@mastra/core";
+import {
+  Observability,
+  DefaultExporter,
+  CloudExporter,
+  SensitiveDataFilter,
+} from "@mastra/observability";
+import { LibSQLStore } from "@mastra/libsql";
+
+export const mastra = new Mastra({
+  storage: new LibSQLStore({
+    id: 'mastra-storage',
+    url: "file:./mastra.db",
+  }),
+  observability: new Observability({
+    configs: {
+      default: {
+        serviceName: "mastra",
+        exporters: [
+          new DefaultExporter(), // Persists traces to storage for Mastra Studio
+          new CloudExporter(), // Sends traces to Mastra Cloud (requires MASTRA_CLOUD_ACCESS_TOKEN)
+        ],
+        spanOutputProcessors: [
+          new SensitiveDataFilter(),
+        ],
+      },
+    },
+  }),
+});
+```
+
+## Viewing Traces
+
+### Studio
+
+Access your traces through Studio:
+
+1. Start Studio
+2. Navigate to Observability
+3. Filter and search your local traces
+4. Inspect detailed span information
+
+## Tracing Strategies
+
+DefaultExporter automatically selects the optimal tracing strategy based on your storage provider. You can also override this selection if needed.
+
+### Available Strategies
+
+| Strategy               | Description                                               | Use Case                            |
+| ---------------------- | --------------------------------------------------------- | ----------------------------------- |
+| **realtime**           | Process each event immediately                            | Development, debugging, low traffic |
+| **batch-with-updates** | Buffer events and batch write with full lifecycle support | Low volume Production               |
+| **insert-only**        | Only process completed spans, ignore updates              | High volume Production              |
+
+### Strategy Configuration
+
+```typescript
+new DefaultExporter({
+  strategy: "auto", // Default - let storage provider decide
+  // or explicitly set:
+  // strategy: 'realtime' | 'batch-with-updates' | 'insert-only'
+
+  // Batching configuration (applies to both batch-with-updates and insert-only)
+  maxBatchSize: 1000, // Max spans per batch
+  maxBatchWaitMs: 5000, // Max wait before flushing
+  maxBufferSize: 10000, // Max spans to buffer
+});
+```
+
+## Storage Provider Support
+
+Different storage providers support different tracing strategies.
+
+If you set the strategy to `'auto'`, the `DefaultExporter` automatically selects the optimal strategy for the storage provider. If you set the strategy to a mode that the storage provider doesn't support, you will get an error message.
+
+| Storage Provider                                | Preferred Strategy | Supported Strategies                      | Notes                                 |
+| ----------------------------------------------- | ------------------ | ----------------------------------------- | ------------------------------------- |
+| **[libSQL](https://mastra.ai/reference/v1/storage/libsql)**         | batch-with-updates | realtime, batch-with-updates, insert-only | Default storage, good for development |
+| **[PostgreSQL](https://mastra.ai/reference/v1/storage/postgresql)** | batch-with-updates | batch-with-updates, insert-only           | Recommended for production            |
+
+### Strategy Benefits
+
+- **realtime**: Immediate visibility, best for debugging
+- **batch-with-updates**: 10-100x throughput improvement, full span lifecycle
+- **insert-only**: Additional 70% reduction in database operations, perfect for analytics
+
+## Batching Behavior
+
+### Flush Triggers
+
+For both batch strategies (`batch-with-updates` and `insert-only`), traces are flushed to storage when any of these conditions are met:
+
+1. **Size trigger**: Buffer reaches `maxBatchSize` spans
+2. **Time trigger**: `maxBatchWaitMs` elapsed since first event
+3. **Emergency flush**: Buffer approaches `maxBufferSize` limit
+4. **Shutdown**: Force flush all pending events
+
+### Error Handling
+
+The DefaultExporter includes robust error handling for production use:
+
+- **Retry Logic**: Exponential backoff (500ms, 1s, 2s, 4s)
+- **Transient Failures**: Automatic retry with backoff
+- **Persistent Failures**: Drop batch after 4 failed attempts
+- **Buffer Overflow**: Prevent memory issues during storage outages
+
+### Configuration Examples
+
+```typescript
+// Zero config - recommended for most users
+new DefaultExporter();
+
+// Development override
+new DefaultExporter({
+  strategy: "realtime", // Immediate visibility for debugging
+});
+
+// High-throughput production
+new DefaultExporter({
+  maxBatchSize: 2000, // Larger batches
+  maxBatchWaitMs: 10000, // Wait longer to fill batches
+  maxBufferSize: 50000, // Handle longer outages
+});
+
+// Low-latency production
+new DefaultExporter({
+  maxBatchSize: 100, // Smaller batches
+  maxBatchWaitMs: 1000, // Flush quickly
+});
+```
+
+## Related
+
+- [Tracing Overview](https://mastra.ai/docs/v1/observability/tracing/overview)
+- [CloudExporter](https://mastra.ai/docs/v1/observability/tracing/exporters/cloud)
+- [Storage Configuration](https://mastra.ai/docs/v1/memory/storage)