@mastra/lance 1.0.0 → 1.0.1

package/dist/docs/{rag/01-vector-databases.md → references/docs-rag-vector-databases.md}

@@ -1,14 +1,12 @@
-> Guide on vector storage options in Mastra, including embedded and dedicated vector databases for similarity search.
-
 # Storing Embeddings in A Vector Database
 
 After generating embeddings, you need to store them in a database that supports vector similarity search. Mastra provides a consistent interface for storing and querying embeddings across various vector databases.
 
 ## Supported Databases
 
-  **mongodb:**
+**MongoDB**:
 
-```ts title="vector-store.ts"
+```ts
 import { MongoDBVector } from "@mastra/mongodb";
 
 const store = new MongoDBVector({
@@ -27,15 +25,13 @@ await store.upsert({
 });
 ```
 
-<h3>Using MongoDB Atlas Vector search</h3>
-
-For detailed setup instructions and best practices, see the [official MongoDB Atlas Vector Search documentation](https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-overview/?utm_campaign=devrel&utm_source=third-party-content&utm_medium=cta&utm_content=mastra-docs).
+### Using MongoDB Atlas Vector search
 
-  
+For detailed setup instructions and best practices, see the [official MongoDB Atlas Vector Search documentation](https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-overview/?utm_campaign=devrel\&utm_source=third-party-content\&utm_medium=cta\&utm_content=mastra-docs).
 
-  **pg-vector:**
+**PgVector**:
 
-```ts title="vector-store.ts"
+```ts
 import { PgVector } from "@mastra/pg";
 
 const store = new PgVector({
@@ -55,16 +51,13 @@ await store.upsert({
 });
 ```
 
-<h3>Using PostgreSQL with pgvector</h3>
-
-PostgreSQL with the pgvector extension is a good solution for teams already using PostgreSQL who want to minimize infrastructure complexity.
-For detailed setup instructions and best practices, see the [official pgvector repository](https://github.com/pgvector/pgvector).
+### Using PostgreSQL with pgvector
 
-  
+PostgreSQL with the pgvector extension is a good solution for teams already using PostgreSQL who want to minimize infrastructure complexity. For detailed setup instructions and best practices, see the [official pgvector repository](https://github.com/pgvector/pgvector).
 
-  **pinecone:**
+**Pinecone**:
 
-```ts title="vector-store.ts"
+```ts
 import { PineconeVector } from "@mastra/pinecone";
 
 const store = new PineconeVector({
@@ -82,11 +75,9 @@ await store.upsert({
 });
 ```
 
-  
+**Qdrant**:
 
-  **qdrant:**
-
-```ts title="vector-store.ts"
+```ts
 import { QdrantVector } from "@mastra/qdrant";
 
 const store = new QdrantVector({
@@ -107,11 +98,9 @@ await store.upsert({
 });
 ```
 
-  
-
-  **chroma:**
+**Chroma**:
 
-```ts title="vector-store.ts"
+```ts
 import { ChromaVector } from "@mastra/chroma";
 
 // Running Chroma locally
@@ -137,11 +126,9 @@ await store.upsert({
 });
 ```
 
-  
-
-  **astra:**
+**Astra**:
 
-```ts title="vector-store.ts"
+```ts
 import { AstraVector } from "@mastra/astra";
 
 const store = new AstraVector({
@@ -163,11 +150,9 @@ await store.upsert({
 });
 ```
 
-  
-
-  **libsql:**
+**libSQL**:
 
-```ts title="vector-store.ts"
+```ts
 import { LibSQLVector } from "@mastra/core/vector/libsql";
 
 const store = new LibSQLVector({
@@ -188,11 +173,9 @@ await store.upsert({
 });
 ```
 
-  
+**Upstash**:
 
-  **upstash:**
-
-```ts title="vector-store.ts"
+```ts
 import { UpstashVector } from "@mastra/upstash";
 
 // In upstash they refer to the store as an index
@@ -211,11 +194,9 @@ await store.upsert({
 });
 ```
 
-  
+**Cloudflare**:
 
-  **cloudflare:**
-
-```ts title="vector-store.ts"
+```ts
 import { CloudflareVector } from "@mastra/vectorize";
 
 const store = new CloudflareVector({
@@ -234,11 +215,9 @@ await store.upsert({
 });
 ```
 
-  
-
-  **opensearch:**
+**OpenSearch**:
 
-```ts title="vector-store.ts"
+```ts
 import { OpenSearchVector } from "@mastra/opensearch";
 
 const store = new OpenSearchVector({ id: "opensearch", node: process.env.OPENSEARCH_URL });
@@ -255,14 +234,18 @@ await store.upsert({
 });
 ```
 
-  
-
-  **elasticsearch:**
+**ElasticSearch**:
 
-```ts title="vector-store.ts"
+```ts
 import { ElasticSearchVector } from "@mastra/elasticsearch";
 
-const store = new ElasticSearchVector({ id: 'elasticsearch-vector', url: process.env.ELASTICSEARCH_URL });
+const store = new ElasticSearchVector({
+  id: 'elasticsearch-vector',
+  url: process.env.ELASTICSEARCH_URL,
+  auth: {
+    apiKey : process.env.ELASTICSEARCH_API_KEY
+  }
+});
 
 await store.createIndex({
   indexName: "my-collection",
@@ -276,10 +259,13 @@ await store.upsert({
 });
 ```
 
-  
-  **couchbase:**
+### Using Elasticsearch
+
+For detailed setup instructions and best practices, see the [official Elasticsearch documentation](https://www.elastic.co/docs/solutions/search/get-started).
 
-```ts title="vector-store.ts"
+**Couchbase**:
+
+```ts
 import { CouchbaseVector } from "@mastra/couchbase";
 
 const store = new CouchbaseVector({
@@ -302,10 +288,9 @@ await store.upsert({
 });
 ```
 
-  
-  **lancedb:**
+**Lance**:
 
-```ts title="vector-store.ts"
+```ts
 import { LanceVectorStore } from "@mastra/lance";
 
 const store = await LanceVectorStore.create("/path/to/db");
@@ -323,15 +308,13 @@ await store.upsert({
 });
 ```
 
-<h3>Using LanceDB</h3>
+### Using LanceDB
 
-LanceDB is an embedded vector database built on the Lance columnar format, suitable for local development or cloud deployment.
-For detailed setup instructions and best practices, see the [official LanceDB documentation](https://lancedb.github.io/lancedb/).
+LanceDB is an embedded vector database built on the Lance columnar format, suitable for local development or cloud deployment. For detailed setup instructions and best practices, see the [official LanceDB documentation](https://lancedb.github.io/lancedb/).
 
-  
-  **s3vectors:**
+**S3 Vectors**:
 
-```ts title="vector-store.ts"
+```ts
 import { S3Vectors } from "@mastra/s3vectors";
 
 const store = new S3Vectors({
@@ -354,8 +337,6 @@ await store.upsert({
 });
 ```
 
-  
-
 ## Using Vector Storage
 
 Once initialized, all vector stores share the same interface for creating indexes, upserting embeddings, and querying.
@@ -364,7 +345,7 @@ Once initialized, all vector stores share the same interface for creating indexe
 
 Before storing embeddings, you need to create an index with the appropriate dimension size for your embedding model:
 
-```ts title="store-embeddings.ts"
+```ts
 // Create an index with dimension 1536 (for text-embedding-3-small)
 await store.createIndex({
   indexName: "myCollection",
@@ -376,150 +357,174 @@ The dimension size must match the output dimension of your chosen embedding mode
 
 - OpenAI text-embedding-3-small: 1536 dimensions (or custom, e.g., 256)
 - Cohere embed-multilingual-v3: 1024 dimensions
-- Google text-embedding-004: 768 dimensions (or custom)
+- Google gemini-embedding-001: 768 dimensions (or custom)
 
-> **Note:**
-Index dimensions cannot be changed after creation. To use a different model, delete and recreate the index with the new dimension size.
+> **Warning:** Index dimensions cannot be changed after creation. To use a different model, delete and recreate the index with the new dimension size.
 
 ### Naming Rules for Databases
 
 Each vector database enforces specific naming conventions for indexes and collections to ensure compatibility and prevent conflicts.
 
-  **mongodb:**
-
-    Collection (index) names must:
-    - Start with a letter or underscore
-    - Be up to 120 bytes long
-    - Contain only letters, numbers, underscores, or dots
-    - Cannot contain `$` or the null character
-    - Example: `my_collection.123` is valid
-    - Example: `my-index` is not valid (contains hyphen)
-    - Example: `My$Collection` is not valid (contains `$`)
-  
-  **pgVector:**
-
-    Index names must:
-    - Start with a letter or underscore
-    - Contain only letters, numbers, and underscores
-    - Example: `my_index_123` is valid
-    - Example: `my-index` is not valid (contains hyphen)
-  
-  **pinecone:**
-
-    Index names must:
-    - Use only lowercase letters, numbers, and dashes
-    - Not contain dots (used for DNS routing)
-    - Not use non-Latin characters or emojis
-    - Have a combined length (with project ID) under 52 characters
-      - Example: `my-index-123` is valid
-      - Example: `my.index` is not valid (contains dot)
-  
-  **qdrant:**
-
-    Collection names must:
-    - Be 1-255 characters long
-    - Not contain any of these special characters:
-      - `< > : " / \ | ? *`
-      - Null character (`\0`)
-      - Unit separator (`\u{1F}`)
-    - Example: `my_collection_123` is valid
-    - Example: `my/collection` is not valid (contains slash)
-  
-  **chroma:**
-
-    Collection names must:
-    - Be 3-63 characters long
-    - Start and end with a letter or number
-    - Contain only letters, numbers, underscores, or hyphens
-    - Not contain consecutive periods (..)
-    - Not be a valid IPv4 address
-    - Example: `my-collection-123` is valid
-    - Example: `my..collection` is not valid (consecutive periods)
-  
-  **astra:**
-
-    Collection names must:
-    - Not be empty
-    - Be 48 characters or less
-    - Contain only letters, numbers, and underscores
-    - Example: `my_collection_123` is valid
-    - Example: `my-collection` is not valid (contains hyphen)
-  
-  **libsql:**
-
-    Index names must:
-    - Start with a letter or underscore
-    - Contain only letters, numbers, and underscores
-    - Example: `my_index_123` is valid
-    - Example: `my-index` is not valid (contains hyphen)
-  
-  **upstash:**
-
-    Namespace names must:
-    - Be 2-100 characters long
-    - Contain only:
-      - Alphanumeric characters (a-z, A-Z, 0-9)
-      - Underscores, hyphens, dots
-    - Not start or end with special characters (_, -, .)
-    - Can be case-sensitive
-    - Example: `MyNamespace123` is valid
-    - Example: `_namespace` is not valid (starts with underscore)
-  
-  **cloudflare:**
-
-    Index names must:
-    - Start with a letter
-    - Be shorter than 32 characters
-    - Contain only lowercase ASCII letters, numbers, and dashes
-    - Use dashes instead of spaces
-    - Example: `my-index-123` is valid
-    - Example: `My_Index` is not valid (uppercase and underscore)
-  
-  **opensearch:**
-
-    Index names must:
-    - Use only lowercase letters
-    - Not begin with underscores or hyphens
-    - Not contain spaces, commas
-    - Not contain special characters (e.g. `:`, `"`, `*`, `+`, `/`, `\`, `|`, `?`, `#`, `>`, `<`)
-    - Example: `my-index-123` is valid
-    - Example: `My_Index` is not valid (contains uppercase letters)
-    - Example: `_myindex` is not valid (begins with underscore)
-  
-  **elasticsearch:**
-
-    Index names must:
-    - Use only lowercase letters
-    - Not exceed 255 bytes (counting multi-byte characters)
-    - Not begin with underscores, hyphens, or plus signs
-    - Not contain spaces, commas
-    - Not contain special characters (e.g. `:`, `"`, `*`, `+`, `/`, `\`, `|`, `?`, `#`, `>`, `<`)
-    - Not be "." or ".."
-    - Not start with "." (deprecated except for system/hidden indices)
-    - Example: `my-index-123` is valid
-    - Example: `My_Index` is not valid (contains uppercase letters)
-    - Example: `_myindex` is not valid (begins with underscore)
-    - Example: `.myindex` is not valid (begins with dot, deprecated)
-  
-  **s3vectors:**
-
-    Index names must:
-    - Be unique within the same vector bucket
-    - Be 3–63 characters long
-    - Use only lowercase letters (`a–z`), numbers (`0–9`), hyphens (`-`), and dots (`.`)
-    - Begin and end with a letter or number
-    - Example: `my-index.123` is valid
-    - Example: `my_index` is not valid (contains underscore)
-    - Example: `-myindex` is not valid (begins with hyphen)
-    - Example: `myindex-` is not valid (ends with hyphen)
-    - Example: `MyIndex` is not valid (contains uppercase letters)
-  
+**MongoDB**:
+
+Collection (index) names must:
+
+- Start with a letter or underscore
+- Be up to 120 bytes long
+- Contain only letters, numbers, underscores, or dots
+- Cannot contain `$` or the null character
+- Example: `my_collection.123` is valid
+- Example: `my-index` is not valid (contains hyphen)
+- Example: `My$Collection` is not valid (contains `$`)
+
+**PgVector**:
+
+Index names must:
+
+- Start with a letter or underscore
+- Contain only letters, numbers, and underscores
+- Example: `my_index_123` is valid
+- Example: `my-index` is not valid (contains hyphen)
+
+**Pinecone**:
+
+Index names must:
+
+- Use only lowercase letters, numbers, and dashes
+
+- Not contain dots (used for DNS routing)
+
+- Not use non-Latin characters or emojis
+
+- Have a combined length (with project ID) under 52 characters
+
+  - Example: `my-index-123` is valid
+  - Example: `my.index` is not valid (contains dot)
+
+**Qdrant**:
+
+Collection names must:
+
+- Be 1-255 characters long
+
+- Not contain any of these special characters:
+
+  - `< > : " / \ | ? *`
+  - Null character (`\0`)
+  - Unit separator (`\u{1F}`)
+
+- Example: `my_collection_123` is valid
+
+- Example: `my/collection` is not valid (contains slash)
+
+**Chroma**:
+
+Collection names must:
+
+- Be 3-63 characters long
+- Start and end with a letter or number
+- Contain only letters, numbers, underscores, or hyphens
+- Not contain consecutive periods (..)
+- Not be a valid IPv4 address
+- Example: `my-collection-123` is valid
+- Example: `my..collection` is not valid (consecutive periods)
+
+**Astra**:
+
+Collection names must:
+
+- Not be empty
+- Be 48 characters or less
+- Contain only letters, numbers, and underscores
+- Example: `my_collection_123` is valid
+- Example: `my-collection` is not valid (contains hyphen)
+
+**libSQL**:
+
+Index names must:
+
+- Start with a letter or underscore
+- Contain only letters, numbers, and underscores
+- Example: `my_index_123` is valid
+- Example: `my-index` is not valid (contains hyphen)
+
+**Upstash**:
+
+Namespace names must:
+
+- Be 2-100 characters long
+
+- Contain only:
+
+  - Alphanumeric characters (a-z, A-Z, 0-9)
+  - Underscores, hyphens, dots
+
+- Not start or end with special characters (\_, -, .)
+
+- Can be case-sensitive
+
+- Example: `MyNamespace123` is valid
+
+- Example: `_namespace` is not valid (starts with underscore)
+
+**Cloudflare**:
+
+Index names must:
+
+- Start with a letter
+- Be shorter than 32 characters
+- Contain only lowercase ASCII letters, numbers, and dashes
+- Use dashes instead of spaces
+- Example: `my-index-123` is valid
+- Example: `My_Index` is not valid (uppercase and underscore)
+
+**OpenSearch**:
+
+Index names must:
+
+- Use only lowercase letters
+- Not begin with underscores or hyphens
+- Not contain spaces, commas
+- Not contain special characters (e.g. `:`, `"`, `*`, `+`, `/`, `\`, `|`, `?`, `#`, `>`, `<`)
+- Example: `my-index-123` is valid
+- Example: `My_Index` is not valid (contains uppercase letters)
+- Example: `_myindex` is not valid (begins with underscore)
+
+**ElasticSearch**:
+
+Index names must:
+
+- Use only lowercase letters
+- Not exceed 255 bytes (counting multi-byte characters)
+- Not begin with underscores, hyphens, or plus signs
+- Not contain spaces, commas
+- Not contain special characters (e.g. `:`, `"`, `*`, `+`, `/`, `\`, `|`, `?`, `#`, `>`, `<`)
+- Not be "." or ".."
+- Not start with "." (deprecated except for system/hidden indices)
+- Example: `my-index-123` is valid
+- Example: `My_Index` is not valid (contains uppercase letters)
+- Example: `_myindex` is not valid (begins with underscore)
+- Example: `.myindex` is not valid (begins with dot, deprecated)
+
+**S3 Vectors**:
+
+Index names must:
+
+- Be unique within the same vector bucket
+- Be 3–63 characters long
+- Use only lowercase letters (`a–z`), numbers (`0–9`), hyphens (`-`), and dots (`.`)
+- Begin and end with a letter or number
+- Example: `my-index.123` is valid
+- Example: `my_index` is not valid (contains underscore)
+- Example: `-myindex` is not valid (begins with hyphen)
+- Example: `myindex-` is not valid (ends with hyphen)
+- Example: `MyIndex` is not valid (contains uppercase letters)
 
 ### Upserting Embeddings
 
 After creating an index, you can store embeddings along with their basic metadata:
 
-```ts title="store-embeddings.ts"
+```ts
 // Store embeddings with their corresponding metadata
 await store.upsert({
   indexName: "myCollection", // index name
@@ -542,8 +547,7 @@ The upsert operation:
 
 Vector stores support rich metadata (any JSON-serializable fields) for filtering and organization. Since metadata is stored with no fixed schema, use consistent field naming to avoid unexpected query results.
 
-> **Note:**
-Metadata is crucial for vector storage - without it, you'd only have numerical embeddings with no way to return the original text or filter results. Always store at least the source text as metadata.
+> **Warning:** Metadata is crucial for vector storage - without it, you'd only have numerical embeddings with no way to return the original text or filter results. Always store at least the source text as metadata.
 
 ```ts
 // Store embeddings with rich metadata for better organization and filtering
@@ -585,7 +589,7 @@ When building RAG applications, you often need to clean up stale vectors when do
 
 The most common use case is deleting all vectors for a specific document when a user deletes it:
 
-```ts title="delete-vectors.ts"
+```ts
 // Delete all vectors for a specific document
 await store.deleteVectors({
   indexName: "myCollection",
@@ -594,6 +598,7 @@ await store.deleteVectors({
 ```
 
 This is particularly useful when:
+
 - A user deletes a document and you need to remove all its chunks
 - You're re-indexing a document and want to remove old vectors first
 - You need to clean up vectors for a specific user or tenant
@@ -602,7 +607,7 @@ This is particularly useful when:
 
 You can also use complex filters to delete vectors matching multiple conditions:
 
-```ts title="delete-vectors-advanced.ts"
+```ts
 // Delete all vectors for multiple documents
 await store.deleteVectors({
   indexName: "myCollection",
@@ -627,7 +632,7 @@ await store.deleteVectors({
 
 If you have specific vector IDs to delete, you can pass them directly:
 
-```ts title="delete-by-ids.ts"
+```ts
 // Delete specific vectors by their IDs
 await store.deleteVectors({
   indexName: "myCollection",

package/dist/docs/{storage/01-reference.md → references/reference-storage-lance.md}

@@ -1,20 +1,33 @@
-# Storage API Reference
+# LanceDB Storage
 
-> API reference for storage - 1 entries
+The LanceDB storage implementation provides a high-performance storage solution using the LanceDB database system, which excels at handling both traditional data storage and vector operations.
 
+> **Observability Not Supported:** LanceDB storage **does not support the observability domain**. Traces from the `DefaultExporter` cannot be persisted to LanceDB, and Mastra Studio's observability features won't work with LanceDB as your only storage provider. To enable observability, use [composite storage](https://mastra.ai/reference/storage/composite) to route observability data to a supported provider like ClickHouse or PostgreSQL.
 
----
+## Installation
 
-## Reference: LanceDB Storage
+**npm**:
 
-> Documentation for the LanceDB storage implementation in Mastra.
+```bash
+npm install @mastra/lance@latest
+```
 
-The LanceDB storage implementation provides a high-performance storage solution using the LanceDB database system, which excels at handling both traditional data storage and vector operations.
+**pnpm**:
 
-## Installation
+```bash
+pnpm add @mastra/lance@latest
+```
+
+**Yarn**:
 
 ```bash
-npm install @mastra/lance@beta
+yarn add @mastra/lance@latest
+```
+
+**Bun**:
+
+```bash
+bun add @mastra/lance@latest
 ```
 
 ## Usage
@@ -40,6 +53,12 @@ const storage = await LanceStorage.create("my-storage", "s3://bucket/db", {
 
 ### LanceStorage.create()
 
+**name:** (`string`): Name identifier for the storage instance
+
+**uri:** (`string`): URI to connect to the LanceDB database. Can be a local path, cloud DB URL, or S3 bucket URL
+
+**options?:** (`ConnectionOptions`): Connection options for LanceDB, such as timeout settings, authentication, etc.
+
 ## Additional Notes
 
 ### Schema Management
@@ -82,23 +101,22 @@ const memoryStore = await storage.getStore('memory');
 const thread = await memoryStore?.getThreadById({ threadId: "..." });
 ```
 
-> **Note:**
-If `init()` is not called, tables won't be created and storage operations will fail silently or throw errors.
+> **Warning:** If `init()` is not called, tables won't be created and storage operations will fail silently or throw errors.
 
 ### Deployment Options
 
 LanceDB storage can be configured for different deployment scenarios:
 
 - **Local Development**: Use a local file path for development and testing
-  ```
+  ```text
   /path/to/db
   ```
 - **Cloud Deployment**: Connect to a hosted LanceDB instance
-  ```
+  ```text
   db://host:port
   ```
 - **S3 Storage**: Use Amazon S3 for scalable cloud storage
-  ```
+  ```text
   s3://bucket/db
   ```