@mastra/mcp-docs-server 0.0.0-commonjs-20250414101718

package/.docs/raw/reference/rag/libsql.mdx

@@ -0,0 +1,357 @@
+---
+title: "Default Vector Store | Vector Databases | RAG | Mastra Docs"
+description: Documentation for the LibSQLVector class in Mastra, which provides vector search using LibSQL with vector extensions.
+---
+
+# LibSQLVector Store
+
+The LibSQL storage implementation provides a SQLite-compatible vector search [LibSQL](https://github.com/tursodatabase/libsql), a fork of SQLite with vector extensions, and [Turso](https://turso.tech/) with vector extensions, offering a lightweight and efficient vector database solution.
+It's part of the `@mastra/core` package and offers efficient vector similarity search with metadata filtering.
+
+## Installation
+
+Default vector store is included in the core package:
+
+```bash copy
+npm install @mastra/core
+```
+
+## Usage
+
+```typescript copy showLineNumbers
+import { LibSQLVector } from "@mastra/core/vector/libsql";
+
+// Create a new vector store instance
+const store = new LibSQLVector({
+  connectionUrl: process.env.DATABASE_URL,
+  // Optional: for Turso cloud databases
+  authToken: process.env.DATABASE_AUTH_TOKEN,
+});
+
+// Create an index
+await store.createIndex({
+  indexName: "myCollection",
+  dimension: 1536,
+});
+
+// Add vectors with metadata
+const vectors = [[0.1, 0.2, ...], [0.3, 0.4, ...]];
+const metadata = [
+  { text: "first document", category: "A" },
+  { text: "second document", category: "B" }
+];
+await store.upsert({
+  indexName: "myCollection",
+  vectors,
+  metadata,
+});
+
+// Query similar vectors
+const queryVector = [0.1, 0.2, ...];
+const results = await store.query({
+  indexName: "myCollection",
+  queryVector,
+  topK: 10, // top K results
+  filter: { category: "A" } // optional metadata filter
+});
+```
+
+## Constructor Options
+
+<PropertiesTable
+  content={[
+    {
+      name: "connectionUrl",
+      type: "string",
+      description:
+        "LibSQL database URL. Use ':memory:' for in-memory database, 'file:dbname.db' for local file, or a LibSQL-compatible connection string like 'libsql://your-database.turso.io'.",
+    },
+    {
+      name: "authToken",
+      type: "string",
+      isOptional: true,
+      description: "Authentication token for Turso cloud databases",
+    },
+    {
+      name: "syncUrl",
+      type: "string",
+      isOptional: true,
+      description: "URL for database replication (Turso specific)",
+    },
+    {
+      name: "syncInterval",
+      type: "number",
+      isOptional: true,
+      description:
+        "Interval in milliseconds for database sync (Turso specific)",
+    },
+  ]}
+/>
+
+## Methods
+
+### createIndex()
+
+Creates a new vector collection. The index name must start with a letter or underscore and can only contain letters, numbers, and underscores. The dimension must be a positive integer.
+
+<PropertiesTable
+  content={[
+    {
+      name: "indexName",
+      type: "string",
+      description: "Name of the index to create",
+    },
+    {
+      name: "dimension",
+      type: "number",
+      description: "Vector dimension size (must match your embedding model)",
+    },
+    {
+      name: "metric",
+      type: "'cosine' | 'euclidean' | 'dotproduct'",
+      isOptional: true,
+      defaultValue: "cosine",
+      description:
+        "Distance metric for similarity search. Note: Currently only cosine similarity is supported by LibSQL.",
+    },
+  ]}
+/>
+
+### upsert()
+
+Adds or updates vectors and their metadata in the index. Uses a transaction to ensure all vectors are inserted atomically - if any insert fails, the entire operation is rolled back.
+
+<PropertiesTable
+  content={[
+    {
+      name: "indexName",
+      type: "string",
+      description: "Name of the index to insert into",
+    },
+    {
+      name: "vectors",
+      type: "number[][]",
+      description: "Array of embedding vectors",
+    },
+    {
+      name: "metadata",
+      type: "Record<string, any>[]",
+      isOptional: true,
+      description: "Metadata for each vector",
+    },
+    {
+      name: "ids",
+      type: "string[]",
+      isOptional: true,
+      description: "Optional vector IDs (auto-generated if not provided)",
+    },
+  ]}
+/>
+
+### query()
+
+Searches for similar vectors with optional metadata filtering.
+
+<PropertiesTable
+  content={[
+    {
+      name: "indexName",
+      type: "string",
+      description: "Name of the index to search in",
+    },
+    {
+      name: "queryVector",
+      type: "number[]",
+      description: "Query vector to find similar vectors for",
+    },
+    {
+      name: "topK",
+      type: "number",
+      isOptional: true,
+      defaultValue: "10",
+      description: "Number of results to return",
+    },
+    {
+      name: "filter",
+      type: "Filter",
+      isOptional: true,
+      description: "Metadata filters",
+    },
+    {
+      name: "includeVector",
+      type: "boolean",
+      isOptional: true,
+      defaultValue: "false",
+      description: "Whether to include vector data in results",
+    },
+    {
+      name: "minScore",
+      type: "number",
+      isOptional: true,
+      defaultValue: "0",
+      description: "Minimum similarity score threshold",
+    },
+  ]}
+/>
+
+### describeIndex()
+
+Gets information about an index.
+
+<PropertiesTable
+  content={[
+    {
+      name: "indexName",
+      type: "string",
+      description: "Name of the index to describe",
+    },
+  ]}
+/>
+
+Returns:
+
+```typescript copy
+interface IndexStats {
+  dimension: number;
+  count: number;
+  metric: "cosine" | "euclidean" | "dotproduct";
+}
+```
+
+### deleteIndex()
+
+Deletes an index and all its data.
+
+<PropertiesTable
+  content={[
+    {
+      name: "indexName",
+      type: "string",
+      description: "Name of the index to delete",
+    },
+  ]}
+/>
+
+### listIndexes()
+
+Lists all vector indexes in the database.
+
+Returns: `Promise<string[]>`
+
+### truncateIndex()
+
+Removes all vectors from an index while keeping the index structure.
+
+<PropertiesTable
+  content={[
+    {
+      name: "indexName",
+      type: "string",
+      description: "Name of the index to truncate",
+    },
+  ]}
+/>
+
+### updateIndexById()
+
+Updates a specific vector entry by its ID with new vector data and/or metadata.
+
+<PropertiesTable
+  content={[
+    {
+      name: "indexName",
+      type: "string",
+      description: "Name of the index containing the vector",
+    },
+    {
+      name: "id",
+      type: "string",
+      description: "ID of the vector entry to update",
+    },
+    {
+      name: "update",
+      type: "object",
+      description: "Update data containing vector and/or metadata",
+    },
+    {
+      name: "update.vector",
+      type: "number[]",
+      isOptional: true,
+      description: "New vector data to update",
+    },
+    {
+      name: "update.metadata",
+      type: "Record<string, any>",
+      isOptional: true,
+      description: "New metadata to update",
+    },
+  ]}
+/>
+
+### deleteIndexById()
+
+Deletes a specific vector entry from an index by its ID.
+
+<PropertiesTable
+  content={[
+    {
+      name: "indexName",
+      type: "string",
+      description: "Name of the index containing the vector",
+    },
+    {
+      name: "id",
+      type: "string",
+      description: "ID of the vector entry to delete",
+    },
+  ]}
+/>
+
+## Response Types
+
+Query results are returned in this format:
+
+```typescript copy
+interface QueryResult {
+  id: string;
+  score: number;
+  metadata: Record<string, any>;
+  vector?: number[]; // Only included if includeVector is true
+}
+```
+
+## Error Handling
+
+The store throws specific errors for different failure cases:
+
+```typescript copy
+try {
+  await store.query({
+    indexName: "my-collection",
+    queryVector: queryVector,
+  });
+} catch (error) {
+  // Handle specific error cases
+  if (error.message.includes("Invalid index name format")) {
+    console.error(
+      "Index name must start with a letter/underscore and contain only alphanumeric characters",
+    );
+  } else if (error.message.includes("Table not found")) {
+    console.error("The specified index does not exist");
+  } else {
+    console.error("Vector store error:", error.message);
+  }
+}
+```
+
+Common error cases include:
+
+- Invalid index name format
+- Invalid vector dimensions
+- Table/index not found
+- Database connection issues
+- Transaction failures during upsert
+
+## Related
+
+- [Metadata Filters](./metadata-filters)

package/.docs/raw/reference/rag/metadata-filters.mdx

@@ -0,0 +1,298 @@
+---
+title: "Reference: Metadata Filters | Metadata Filtering | RAG | Mastra Docs"
+description: Documentation for metadata filtering capabilities in Mastra, which allow for precise querying of vector search results across different vector stores.
+---
+
+# Metadata Filters
+
+Mastra provides a unified metadata filtering syntax across all vector stores, based on MongoDB/Sift query syntax. Each vector store translates these filters into their native format.
+
+## Basic Example
+
+```typescript
+import { PgVector } from '@mastra/pg';
+
+const store = new PgVector(connectionString);
+
+const results = await store.query({
+  indexName: "my_index",
+  queryVector: queryVector,
+  topK: 10,
+  filter: {
+    category: "electronics",  // Simple equality
+    price: { $gt: 100 },     // Numeric comparison
+    tags: { $in: ["sale", "new"] }  // Array membership
+  }
+});
+```
+
+## Supported Operators
+
+<OperatorsTable
+  title="Basic Comparison"
+  operators={[
+    {
+      name: "$eq",
+      description: "Matches values equal to specified value",
+      example: "{ age: { $eq: 25 } }",
+      supportedBy: ["All"]
+    },
+    {
+      name: "$ne",
+      description: "Matches values not equal",
+      example: "{ status: { $ne: 'inactive' } }",
+      supportedBy: ["All"]
+    },
+    {
+      name: "$gt",
+      description: "Greater than",
+      example: "{ price: { $gt: 100 } }",
+      supportedBy: ["All"]
+    },
+    {
+      name: "$gte",
+      description: "Greater than or equal",
+      example: "{ rating: { $gte: 4.5 } }",
+      supportedBy: ["All"]
+    },
+    {
+      name: "$lt",
+      description: "Less than",
+      example: "{ stock: { $lt: 20 } }",
+      supportedBy: ["All"]
+    },
+    {
+      name: "$lte",
+      description: "Less than or equal",
+      example: "{ priority: { $lte: 3 } }",
+      supportedBy: ["All"]
+    }
+  ]}
+/>
+
+<OperatorsTable
+  title="Array Operators"
+  operators={[
+    {
+      name: "$in",
+      description: "Matches any value in array",
+      example: '{ category: { $in: ["A", "B"] } }',
+      supportedBy: ["All"]
+    },
+    {
+      name: "$nin",
+      description: "Matches none of the values",
+      example: '{ status: { $nin: ["deleted", "archived"] } }',
+      supportedBy: ["All"]
+    },
+    {
+      name: "$all",
+      description: "Matches arrays containing all elements",
+      example: '{ tags: { $all: ["urgent", "high"] } }',
+      supportedBy: ["Astra", "Pinecone", "Upstash"]
+    },
+    {
+      name: "$elemMatch",
+      description: "Matches array elements meeting criteria",
+      example: '{ scores: { $elemMatch: { $gt: 80 } } }',
+      supportedBy: ["LibSQL", "PgVector"]
+    }
+  ]}
+/>
+
+<OperatorsTable
+  title="Logical Operators"
+  operators={[
+    {
+      name: "$and",
+      description: "Logical AND",
+      example: '{ $and: [{ price: { $gt: 100 } }, { stock: { $gt: 0 } }] }',
+      supportedBy: ["All except Vectorize"]
+    },
+    {
+      name: "$or",
+      description: "Logical OR",
+      example: '{ $or: [{ status: "active" }, { priority: "high" }] }',
+      supportedBy: ["All except Vectorize"]
+    },
+    {
+      name: "$not",
+      description: "Logical NOT",
+      example: '{ price: { $not: { $lt: 100 } } }',
+      supportedBy: ["Astra", "Qdrant", "Upstash", "PgVector", "LibSQL"]
+    },
+    {
+      name: "$nor",
+      description: "Logical NOR",
+      example: '{ $nor: [{ status: "deleted" }, { archived: true }] }',
+      supportedBy: ["Qdrant", "Upstash", "PgVector", "LibSQL"]
+    }
+  ]}
+/>
+
+<OperatorsTable
+  title="Element Operators"
+  operators={[
+    {
+      name: "$exists",
+      description: "Matches documents with field",
+      example: '{ rating: { $exists: true } }',
+      supportedBy: ["All except Vectorize, Chroma"]
+    }
+  ]}
+/>
+
+<OperatorsTable
+  title="Custom Operators"
+  operators={[
+    {
+      name: "$contains",
+      description: "Text contains substring",
+      example: '{ description: { $contains: "sale" } }',
+      supportedBy: ["Upstash", "LibSQL", "PgVector"]
+    },
+    {
+      name: "$regex",
+      description: "Regular expression match",
+      example: '{ name: { $regex: "^test" } }',
+      supportedBy: ["Qdrant", "PgVector", "Upstash"]
+    },
+    {
+      name: "$size",
+      description: "Array length check",
+      example: '{ tags: { $size: { $gt: 2 } } }',
+      supportedBy: ["Astra", "LibSQL", "PgVector"]
+    },
+    {
+      name: "$geo",
+      description: "Geospatial query",
+      example: '{ location: { $geo: { type: "radius", ... } } }',
+      supportedBy: ["Qdrant"]
+    },
+    {
+      name: "$datetime",
+      description: "Datetime range query",
+      example: '{ created: { $datetime: { range: { gt: "2024-01-01" } } } }',
+      supportedBy: ["Qdrant"]
+    },
+    {
+      name: "$hasId",
+      description: "Vector ID existence check",
+      example: '{ $hasId: ["id1", "id2"] }',
+      supportedBy: ["Qdrant"]
+    },
+    {
+      name: "$hasVector",
+      description: "Vector existence check",
+      example: '{ $hasVector: true }',
+      supportedBy: ["Qdrant"]
+    }
+  ]}
+/>
+
+## Common Rules and Restrictions
+
+1. Field names cannot:
+   - Contain dots (.) unless referring to nested fields
+   - Start with $ or contain null characters
+   - Be empty strings
+
+2. Values must be:
+   - Valid JSON types (string, number, boolean, object, array)
+   - Not undefined
+   - Properly typed for the operator (e.g., numbers for numeric comparisons)
+
+3. Logical operators:
+   - Must contain valid conditions
+   - Cannot be empty
+   - Must be properly nested
+   - Can only be used at top level or nested within other logical operators
+   - Cannot be used at field level or nested inside a field
+   - Cannot be used inside an operator
+   - Valid: `{ "$and": [{ "field": { "$gt": 100 } }] }`
+   - Valid: `{ "$or": [{ "$and": [{ "field": { "$gt": 100 } }] }] }`
+   - Invalid: `{ "field": { "$and": [{ "$gt": 100 }] } }`
+   - Invalid: `{ "field": { "$gt": { "$and": [{...}] } } }`
+
+4. $not operator:
+   - Must be an object
+   - Cannot be empty
+   - Can be used at field level or top level
+   - Valid: `{ "$not": { "field": "value" } }`
+   - Valid: `{ "field": { "$not": { "$eq": "value" } } }`
+
+5. Operator nesting:
+   - Logical operators must contain field conditions, not direct operators
+   - Valid: `{ "$and": [{ "field": { "$gt": 100 } }] }`
+   - Invalid: `{ "$and": [{ "$gt": 100 }] }`
+
+## Store-Specific Notes
+
+### Astra
+- Nested field queries are supported using dot notation
+- Array fields must be explicitly defined as arrays in the metadata
+- Metadata values are case-sensitive
+
+### ChromaDB
+- Where filters only return results where the filtered field exists in metadata
+- Empty metadata fields are not included in filter results
+- Metadata fields must be present for negative matches (e.g., $ne won't match documents missing the field)
+
+### Cloudflare Vectorize
+- Requires explicit metadata indexing before filtering can be used
+- Use `createMetadataIndex()` to index fields you want to filter on
+- Up to 10 metadata indexes per Vectorize index
+- String values are indexed up to first 64 bytes (truncated on UTF-8 boundaries)
+- Number values use float64 precision
+- Filter JSON must be under 2048 bytes
+- Field names cannot contain dots (.) or start with $
+- Field names limited to 512 characters
+- Vectors must be re-upserted after creating new metadata indexes to be included in filtered results
+- Range queries may have reduced accuracy with very large datasets (~10M+ vectors)
+
+### LibSQL
+- Supports nested object queries with dot notation
+- Array fields are validated to ensure they contain valid JSON arrays
+- Numeric comparisons maintain proper type handling
+- Empty arrays in conditions are handled gracefully
+- Metadata is stored in a JSONB column for efficient querying
+
+### PgVector
+- Full support for PostgreSQL's native JSON querying capabilities
+- Efficient handling of array operations using native array functions
+- Proper type handling for numbers, strings, and booleans
+- Nested field queries use PostgreSQL's JSON path syntax internally
+- Metadata is stored in a JSONB column for efficient indexing
+
+### Pinecone
+- Metadata field names are limited to 512 characters
+- Numeric values must be within the range of ±1e38
+- Arrays in metadata are limited to 64KB total size
+- Nested objects are flattened with dot notation
+- Metadata updates replace the entire metadata object
+
+### Qdrant
+- Supports advanced filtering with nested conditions
+- Payload (metadata) fields must be explicitly indexed for filtering
+- Efficient handling of geo-spatial queries
+- Special handling for null and empty values
+- Vector-specific filtering capabilities
+- Datetime values must be in RFC 3339 format
+
+### Upstash
+- 512-character limit for metadata field keys
+- Query size is limited (avoid large IN clauses)
+- No support for null/undefined values in filters
+- Translates to SQL-like syntax internally
+- Case-sensitive string comparisons
+- Metadata updates are atomic
+
+## Related
+- [Astra](./astra)
+- [Chroma](./chroma)
+- [Cloudflare Vectorize](./vectorize)
+- [LibSQL](./libsql)
+- [PgStore](./pg)
+- [Pinecone](./pinecone)
+- [Qdrant](./qdrant)
+- [Upstash](./upstash)