@mastra/libsql 1.6.1-alpha.0 → 1.6.2-alpha.0

package/dist/docs/references/docs-observability-overview.md

@@ -0,0 +1,70 @@
+# Observability Overview
+
+Mastra provides observability features for AI applications. Monitor LLM operations, trace agent decisions, and debug complex workflows with tools that understand AI-specific patterns.
+
+## Key Features
+
+### Tracing
+
+Specialized tracing for AI operations that captures:
+
+- **Model interactions**: Token usage, latency, prompts, and completions
+- **Agent execution**: Decision paths, tool calls, and memory operations
+- **Workflow steps**: Branching logic, parallel execution, and step outputs
+- **Automatic instrumentation**: Tracing with decorators
+
+## Storage Requirements
+
+The `DefaultExporter` persists traces to your configured storage backend. Not all storage providers support observability—for the full list, see [Storage Provider Support](https://mastra.ai/docs/observability/tracing/exporters/default).
+
+For production environments with high traffic, we recommend using **ClickHouse** for the observability domain via [composite storage](https://mastra.ai/reference/storage/composite). See [Production Recommendations](https://mastra.ai/docs/observability/tracing/exporters/default) for details.
+
+## Quick Start
+
+Configure Observability in your Mastra instance:
+
+```typescript
+import { Mastra } from '@mastra/core'
+import { PinoLogger } from '@mastra/loggers'
+import { LibSQLStore } from '@mastra/libsql'
+import {
+  Observability,
+  DefaultExporter,
+  CloudExporter,
+  SensitiveDataFilter,
+} from '@mastra/observability'
+
+export const mastra = new Mastra({
+  logger: new PinoLogger(),
+  storage: new LibSQLStore({
+    id: 'mastra-storage',
+    url: 'file:./mastra.db', // Storage is required for tracing
+  }),
+  observability: new Observability({
+    configs: {
+      default: {
+        serviceName: 'mastra',
+        exporters: [
+          new DefaultExporter(), // Persists traces to storage for Mastra Studio
+          new CloudExporter(), // Sends traces to Mastra Cloud (if MASTRA_CLOUD_ACCESS_TOKEN is set)
+        ],
+        spanOutputProcessors: [
+          new SensitiveDataFilter(), // Redacts sensitive data like passwords, tokens, keys
+        ],
+      },
+    },
+  }),
+})
+```
+
+> **Serverless environments:** The `file:./mastra.db` storage URL uses the local filesystem, which doesn't work in serverless environments like Vercel, AWS Lambda, or Cloudflare Workers. For serverless deployments, use external storage. See the [Vercel deployment guide](https://mastra.ai/guides/deployment/vercel) for a complete example.
+
+With this basic setup, you will see Traces and Logs in both Studio and in Mastra Cloud.
+
+We also support various external tracing providers like MLflow, Langfuse, Braintrust, and any OpenTelemetry-compatible platform (Datadog, New Relic, SigNoz, etc.). See more about this in the [Tracing](https://mastra.ai/docs/observability/tracing/overview) documentation.
+
+## What's Next?
+
+- **[Set up Tracing](https://mastra.ai/docs/observability/tracing/overview)**: Configure tracing for your application
+- **[Configure Logging](https://mastra.ai/docs/observability/logging)**: Add structured logging
+- **[API Reference](https://mastra.ai/reference/observability/tracing/instances)**: Detailed configuration options

package/dist/docs/references/docs-observability-tracing-exporters-default.md

@@ -0,0 +1,209 @@
+# Default Exporter
+
+The `DefaultExporter` persists traces to your configured storage backend, making them accessible through Studio. It's automatically enabled when using the default observability configuration and requires no external services.
+
+> **Production Observability:** Observability data can quickly overwhelm general-purpose databases in production. For high-traffic applications, we recommend using **ClickHouse** for the observability storage domain via [composite storage](https://mastra.ai/reference/storage/composite). See [Production Recommendations](#production-recommendations) for details.
+
+## Configuration
+
+### Prerequisites
+
+1. **Storage Backend**: Configure a storage provider (libSQL, PostgreSQL, etc.)
+2. **Studio**: Install for viewing traces locally
+
+### Basic Setup
+
+```typescript
+import { Mastra } from '@mastra/core'
+import { Observability, DefaultExporter } from '@mastra/observability'
+import { LibSQLStore } from '@mastra/libsql'
+
+export const mastra = new Mastra({
+  storage: new LibSQLStore({
+    id: 'mastra-storage',
+    url: 'file:./mastra.db', // Required for trace persistence
+  }),
+  observability: new Observability({
+    configs: {
+      local: {
+        serviceName: 'my-service',
+        exporters: [new DefaultExporter()],
+      },
+    },
+  }),
+})
+```
+
+### Recommended Configuration
+
+Include DefaultExporter in your observability configuration:
+
+```typescript
+import { Mastra } from '@mastra/core'
+import {
+  Observability,
+  DefaultExporter,
+  CloudExporter,
+  SensitiveDataFilter,
+} from '@mastra/observability'
+import { LibSQLStore } from '@mastra/libsql'
+
+export const mastra = new Mastra({
+  storage: new LibSQLStore({
+    id: 'mastra-storage',
+    url: 'file:./mastra.db',
+  }),
+  observability: new Observability({
+    configs: {
+      default: {
+        serviceName: 'mastra',
+        exporters: [
+          new DefaultExporter(), // Persists traces to storage for Mastra Studio
+          new CloudExporter(), // Sends traces to Mastra Cloud (requires MASTRA_CLOUD_ACCESS_TOKEN)
+        ],
+        spanOutputProcessors: [new SensitiveDataFilter()],
+      },
+    },
+  }),
+})
+```
+
+## Viewing Traces
+
+### Studio
+
+Access your traces through Studio:
+
+1. Start Studio
+2. Navigate to Observability
+3. Filter and search your local traces
+4. Inspect detailed span information
+
+## Tracing Strategies
+
+DefaultExporter automatically selects the optimal tracing strategy based on your storage provider. You can also override this selection if needed.
+
+### Available Strategies
+
+| Strategy               | Description                                               | Use Case                            |
+| ---------------------- | --------------------------------------------------------- | ----------------------------------- |
+| **realtime**           | Process each event immediately                            | Development, debugging, low traffic |
+| **batch-with-updates** | Buffer events and batch write with full lifecycle support | Low volume Production               |
+| **insert-only**        | Only process completed spans, ignore updates              | High volume Production              |
+
+### Strategy Configuration
+
+```typescript
+new DefaultExporter({
+  strategy: 'auto', // Default - let storage provider decide
+  // or explicitly set:
+  // strategy: 'realtime' | 'batch-with-updates' | 'insert-only'
+
+  // Batching configuration (applies to both batch-with-updates and insert-only)
+  maxBatchSize: 1000, // Max spans per batch
+  maxBatchWaitMs: 5000, // Max wait before flushing
+  maxBufferSize: 10000, // Max spans to buffer
+})
+```
+
+## Storage Provider Support
+
+Different storage providers support different tracing strategies. Some providers support observability for production workloads, while others are intended primarily for local development.
+
+If you set the strategy to `'auto'`, the `DefaultExporter` automatically selects the optimal strategy for the storage provider. If you set the strategy to a mode that the storage provider doesn't support, you will get an error message.
+
+### Providers with Observability Support
+
+| Storage Provider                                                 | Preferred Strategy | Supported Strategies            | Recommended Use                       |
+| ---------------------------------------------------------------- | ------------------ | ------------------------------- | ------------------------------------- |
+| **ClickHouse** (`@mastra/clickhouse`)                            | insert-only        | insert-only                     | Production (high-volume)              |
+| **[PostgreSQL](https://mastra.ai/reference/storage/postgresql)** | batch-with-updates | batch-with-updates, insert-only | Production (low volume)               |
+| **[MSSQL](https://mastra.ai/reference/storage/mssql)**           | batch-with-updates | batch-with-updates, insert-only | Production (low volume)               |
+| **[MongoDB](https://mastra.ai/reference/storage/mongodb)**       | batch-with-updates | batch-with-updates, insert-only | Production (low volume)               |
+| **[libSQL](https://mastra.ai/reference/storage/libsql)**         | batch-with-updates | batch-with-updates, insert-only | Default storage, good for development |
+
+### Providers without Observability Support
+
+The following storage providers **do not support** the observability domain. If you're using one of these providers and need observability, use [composite storage](https://mastra.ai/reference/storage/composite) to route observability data to a supported provider:
+
+- [Convex](https://mastra.ai/reference/storage/convex)
+- [DynamoDB](https://mastra.ai/reference/storage/dynamodb)
+- [Cloudflare D1](https://mastra.ai/reference/storage/cloudflare-d1)
+- [Cloudflare Durable Objects](https://mastra.ai/reference/storage/cloudflare)
+- [Upstash](https://mastra.ai/reference/storage/upstash)
+- [LanceDB](https://mastra.ai/reference/storage/lance)
+
+### Strategy Benefits
+
+- **realtime**: Immediate visibility, best for debugging
+- **batch-with-updates**: 10-100x throughput improvement, full span lifecycle
+- **insert-only**: Additional 70% reduction in database operations, perfect for analytics
+
+## Production Recommendations
+
+Observability data grows quickly in production environments. A single agent interaction can generate hundreds of spans, and high-traffic applications can produce thousands of traces per day. Most general-purpose databases aren't optimized for this write-heavy, append-only workload.
+
+### Recommended: ClickHouse for High-Volume Production
+
+[ClickHouse](https://mastra.ai/reference/storage/composite) is a columnar database designed for high-volume analytics workloads. It's the recommended choice for production observability because:
+
+- **Optimized for writes**: Handles millions of inserts per second
+- **Efficient compression**: Reduces storage costs for trace data
+- **Fast queries**: Columnar storage enables quick trace lookups and aggregations
+- **Time-series native**: Built-in support for time-based data retention and partitioning
+
+### Using Composite Storage
+
+If you're using a provider without observability support (like Convex or DynamoDB) or want to optimize performance, use [composite storage](https://mastra.ai/reference/storage/composite) to route observability data to ClickHouse while keeping other data in your primary database.
+
+## Batching Behavior
+
+### Flush Triggers
+
+For both batch strategies (`batch-with-updates` and `insert-only`), traces are flushed to storage when any of these conditions are met:
+
+1. **Size trigger**: Buffer reaches `maxBatchSize` spans
+2. **Time trigger**: `maxBatchWaitMs` elapsed since first event
+3. **Emergency flush**: Buffer approaches `maxBufferSize` limit
+4. **Shutdown**: Force flush all pending events
+
+### Error Handling
+
+The DefaultExporter includes robust error handling for production use:
+
+- **Retry Logic**: Exponential backoff (500ms, 1s, 2s, 4s)
+- **Transient Failures**: Automatic retry with backoff
+- **Persistent Failures**: Drop batch after 4 failed attempts
+- **Buffer Overflow**: Prevent memory issues during storage outages
+
+### Configuration Examples
+
+```typescript
+// Zero config - recommended for most users
+new DefaultExporter()
+
+// Development override
+new DefaultExporter({
+  strategy: 'realtime', // Immediate visibility for debugging
+})
+
+// High-throughput production
+new DefaultExporter({
+  maxBatchSize: 2000, // Larger batches
+  maxBatchWaitMs: 10000, // Wait longer to fill batches
+  maxBufferSize: 50000, // Handle longer outages
+})
+
+// Low-latency production
+new DefaultExporter({
+  maxBatchSize: 100, // Smaller batches
+  maxBatchWaitMs: 1000, // Flush quickly
+})
+```
+
+## Related
+
+- [Tracing Overview](https://mastra.ai/docs/observability/tracing/overview)
+- [CloudExporter](https://mastra.ai/docs/observability/tracing/exporters/cloud)
+- [Composite Storage](https://mastra.ai/reference/storage/composite) - Combine multiple storage providers
+- [Storage Configuration](https://mastra.ai/docs/memory/storage)