@mastra/rag 2.1.1 → 2.1.2

package/dist/docs/references/reference-rag-extract-params.md

@@ -0,0 +1,168 @@
+# ExtractParams
+
+ExtractParams configures metadata extraction from document chunks using LLM analysis.
+
+## Example
+
+```typescript
+import { MDocument } from '@mastra/rag'
+
+const doc = MDocument.fromText(text)
+const chunks = await doc.chunk({
+  extract: {
+    title: true, // Extract titles using default settings
+    summary: true, // Generate summaries using default settings
+    keywords: true, // Extract keywords using default settings
+  },
+})
+
+// Example output:
+// chunks[0].metadata = {
+//   documentTitle: "AI Systems Overview",
+//   sectionSummary: "Overview of artificial intelligence concepts and applications",
+//   excerptKeywords: "KEYWORDS: AI, machine learning, algorithms"
+// }
+```
+
+## Parameters
+
+The `extract` parameter accepts the following fields:
+
+**title?:** (`boolean | TitleExtractorsArgs`): Enable title extraction. Set to true for default settings, or provide custom configuration.
+
+**summary?:** (`boolean | SummaryExtractArgs`): Enable summary extraction. Set to true for default settings, or provide custom configuration.
+
+**questions?:** (`boolean | QuestionAnswerExtractArgs`): Enable question generation. Set to true for default settings, or provide custom configuration.
+
+**keywords?:** (`boolean | KeywordExtractArgs`): Enable keyword extraction. Set to true for default settings, or provide custom configuration.
+
+**schema?:** (`SchemaExtractArgs`): Enable structured metadata extraction using a Zod schema.
+
+## Extractor Arguments
+
+### TitleExtractorsArgs
+
+**llm?:** (`MastraLanguageModel`): AI SDK language model to use for title extraction
+
+**nodes?:** (`number`): Number of title nodes to extract
+
+**nodeTemplate?:** (`string`): Custom prompt template for title node extraction. Must include {context} placeholder
+
+**combineTemplate?:** (`string`): Custom prompt template for combining titles. Must include {context} placeholder
+
+### SummaryExtractArgs
+
+**llm?:** (`MastraLanguageModel`): AI SDK language model to use for summary extraction
+
+**summaries?:** (`('self' | 'prev' | 'next')[]`): List of summary types to generate. Can only include 'self' (current chunk), 'prev' (previous chunk), or 'next' (next chunk)
+
+**promptTemplate?:** (`string`): Custom prompt template for summary generation. Must include {context} placeholder
+
+### QuestionAnswerExtractArgs
+
+**llm?:** (`MastraLanguageModel`): AI SDK language model to use for question generation
+
+**questions?:** (`number`): Number of questions to generate
+
+**promptTemplate?:** (`string`): Custom prompt template for question generation. Must include both {context} and {numQuestions} placeholders
+
+**embeddingOnly?:** (`boolean`): If true, only generate embeddings without actual questions
+
+### KeywordExtractArgs
+
+**llm?:** (`MastraLanguageModel`): AI SDK language model to use for keyword extraction
+
+**keywords?:** (`number`): Number of keywords to extract
+
+**promptTemplate?:** (`string`): Custom prompt template for keyword extraction. Must include both {context} and {maxKeywords} placeholders
+
+### SchemaExtractArgs
+
+**schema:** (`ZodType`): Zod schema defining the structure of the data to extract.
+
+**llm?:** (`MastraLanguageModel`): AI SDK language model to use for extraction.
+
+**instructions?:** (`string`): Instructions for the LLM on what to extract.
+
+**metadataKey?:** (`string`): Key to nest extraction results under. If omitted, results are spread into the metadata object.
+
+## Advanced Example
+
+```typescript
+import { MDocument } from '@mastra/rag'
+
+const doc = MDocument.fromText(text)
+const chunks = await doc.chunk({
+  extract: {
+    // Title extraction with custom settings
+    title: {
+      nodes: 2, // Extract 2 title nodes
+      nodeTemplate: 'Generate a title for this: {context}',
+      combineTemplate: 'Combine these titles: {context}',
+    },
+
+    // Summary extraction with custom settings
+    summary: {
+      summaries: ['self'], // Generate summaries for current chunk
+      promptTemplate: 'Summarize this: {context}',
+    },
+
+    // Question generation with custom settings
+    questions: {
+      questions: 3, // Generate 3 questions
+      promptTemplate: 'Generate {numQuestions} questions about: {context}',
+      embeddingOnly: false,
+    },
+
+    // Keyword extraction with custom settings
+    keywords: {
+      keywords: 5, // Extract 5 keywords
+      promptTemplate: 'Extract {maxKeywords} key terms from: {context}',
+    },
+
+    // Schema extraction with Zod
+    schema: {
+      schema: z.object({
+        productName: z.string(),
+        category: z.enum(['electronics', 'clothing']),
+      }),
+      instructions: 'Extract product information.',
+      metadataKey: 'product',
+    },
+  },
+})
+
+// Example output:
+// chunks[0].metadata = {
+//   documentTitle: "AI in Modern Computing",
+//   sectionSummary: "Overview of AI concepts and their applications in computing",
+//   questionsThisExcerptCanAnswer: "1. What is machine learning?\n2. How do neural networks work?",
+//   excerptKeywords: "1. Machine learning\n2. Neural networks\n3. Training data",
+//   product: {
+//     productName: "Neural Net 2000",
+//     category: "electronics"
+//   }
+// }
+```
+
+## Document Grouping for Title Extraction
+
+When using the `TitleExtractor`, you can group multiple chunks together for title extraction by specifying a shared `docId` in the `metadata` field of each chunk. All chunks with the same `docId` will receive the same extracted title. If no `docId` is set, each chunk is treated as its own document for title extraction.
+
+**Example:**
+
+```ts
+import { MDocument } from '@mastra/rag'
+
+const doc = new MDocument({
+  docs: [
+    { text: 'chunk 1', metadata: { docId: 'docA' } },
+    { text: 'chunk 2', metadata: { docId: 'docA' } },
+    { text: 'chunk 3', metadata: { docId: 'docB' } },
+  ],
+  type: 'text',
+})
+
+await doc.extractMetadata({ title: true })
+// The first two chunks will share a title, while the third chunk will be assigned a separate title.
+```

package/dist/docs/references/reference-rag-graph-rag.md

@@ -0,0 +1,111 @@
+# GraphRAG
+
+The `GraphRAG` class implements a graph-based approach to retrieval augmented generation. It creates a knowledge graph from document chunks where nodes represent documents and edges represent semantic relationships, enabling both direct similarity matching and discovery of related content through graph traversal.
+
+## Basic Usage
+
+```typescript
+import { GraphRAG } from '@mastra/rag'
+
+const graphRag = new GraphRAG({
+  dimension: 1536,
+  threshold: 0.7,
+})
+
+// Create the graph from chunks and embeddings
+graphRag.createGraph(documentChunks, embeddings)
+
+// Query the graph with embedding
+const results = await graphRag.query({
+  query: queryEmbedding,
+  topK: 10,
+  randomWalkSteps: 100,
+  restartProb: 0.15,
+})
+```
+
+## Constructor Parameters
+
+**dimension?:** (`number`): Dimension of the embedding vectors (Default: `1536`)
+
+**threshold?:** (`number`): Similarity threshold for creating edges between nodes (0-1) (Default: `0.7`)
+
+## Methods
+
+### createGraph
+
+Creates a knowledge graph from document chunks and their embeddings.
+
+```typescript
+createGraph(chunks: GraphChunk[], embeddings: GraphEmbedding[]): void
+```
+
+#### Parameters
+
+**chunks:** (`GraphChunk[]`): Array of document chunks with text and metadata
+
+**embeddings:** (`GraphEmbedding[]`): Array of embeddings corresponding to chunks
+
+### query
+
+Performs a graph-based search combining vector similarity and graph traversal.
+
+```typescript
+query({
+  query,
+  topK = 10,
+  randomWalkSteps = 100,
+  restartProb = 0.15
+}: {
+  query: number[];
+  topK?: number;
+  randomWalkSteps?: number;
+  restartProb?: number;
+}): RankedNode[]
+```
+
+#### Parameters
+
+**query:** (`number[]`): Query embedding vector
+
+**topK?:** (`number`): Number of results to return (Default: `10`)
+
+**randomWalkSteps?:** (`number`): Number of steps in random walk (Default: `100`)
+
+**restartProb?:** (`number`): Probability of restarting walk from query node (Default: `0.15`)
+
+#### Returns
+
+Returns an array of `RankedNode` objects, where each node contains:
+
+**id:** (`string`): Unique identifier for the node
+
+**content:** (`string`): Text content of the document chunk
+
+**metadata:** (`Record<string, any>`): Additional metadata associated with the chunk
+
+**score:** (`number`): Combined relevance score from graph traversal
+
+## Advanced Example
+
+```typescript
+const graphRag = new GraphRAG({
+  dimension: 1536,
+  threshold: 0.8, // Stricter similarity threshold
+})
+
+// Create graph from chunks and embeddings
+graphRag.createGraph(documentChunks, embeddings)
+
+// Query with custom parameters
+const results = await graphRag.query({
+  query: queryEmbedding,
+  topK: 5,
+  randomWalkSteps: 200,
+  restartProb: 0.2,
+})
+```
+
+## Related
+
+- [createGraphRAGTool](https://mastra.ai/reference/tools/graph-rag-tool)

package/dist/docs/references/reference-rag-rerank.md

@@ -0,0 +1,75 @@
+# rerank()
+
+The `rerank()` function provides advanced reranking capabilities for vector search results by combining semantic relevance, vector similarity, and position-based scoring.
+
+```typescript
+function rerank(
+  results: QueryResult[],
+  query: string,
+  modelConfig: ModelConfig,
+  options?: RerankerFunctionOptions,
+): Promise<RerankResult[]>
+```
+
+## Usage Example
+
+```typescript
+import { rerank } from '@mastra/rag'
+
+const model = 'openai/gpt-5.1'
+
+const rerankedResults = await rerank(vectorSearchResults, 'How do I deploy to production?', model, {
+  weights: {
+    semantic: 0.5,
+    vector: 0.3,
+    position: 0.2,
+  },
+  topK: 3,
+})
+```
+
+## Parameters
+
+**results:** (`QueryResult[]`): The vector search results to rerank
+
+**query:** (`string`): The search query text used to evaluate relevance
+
+**model:** (`MastraLanguageModel`): The language Model to use for reranking
+
+**options?:** (`RerankerFunctionOptions`): Options for the reranking model
+
+The rerank function accepts any LanguageModel from the Vercel AI SDK. When using the Cohere model `rerank-v3.5`, it will automatically use Cohere's reranking capabilities.
+
+> **Note:** For semantic scoring to work properly during re-ranking, each result must include the text content in its `metadata.text` field.
+
+### RerankerFunctionOptions
+
+**weights?:** (`WeightConfig`): numbersemantic?:number (default: 0.4)Weight for semantic relevancenumbervector?:number (default: 0.4)Weight for vector similaritynumberposition?:number (default: 0.2)Weight for position-based scoring
+
+**queryEmbedding?:** (`number[]`): Embedding of the query
+
+**topK?:** (`number`): Number of top results to return (Default: `3`)
+
+## Returns
+
+The function returns an array of `RerankResult` objects:
+
+**result:** (`QueryResult`): The original query result
+
+**score:** (`number`): Combined reranking score (0-1)
+
+**details:** (`ScoringDetails`): Detailed scoring information
+
+### ScoringDetails
+
+**semantic:** (`number`): Semantic relevance score (0-1)
+
+**vector:** (`number`): Vector similarity score (0-1)
+
+**position:** (`number`): Position-based score (0-1)
+
+**queryAnalysis?:** (`object`): numbermagnitude:Magnitude of the querynumber\[]dominantFeatures:Dominant features of the query
+
+## Related
+
+- [createVectorQueryTool](https://mastra.ai/reference/tools/vector-query-tool)

package/dist/docs/references/reference-rag-rerankWithScorer.md

@@ -0,0 +1,80 @@
+# rerankWithScorer()
+
+The `rerankWithScorer()` function provides advanced reranking capabilities for vector search results by combining semantic relevance, vector similarity, and position-based scoring.
+
+```typescript
+function rerankWithScorer({
+  results: QueryResult[],
+  query: string,
+  scorer: RelevanceScoreProvider,
+  options?: RerankerFunctionOptions,
+}): Promise<RerankResult[]>;
+```
+
+## Usage Example
+
+```typescript
+import { rerankWithScorer as rerank, CohereRelevanceScorer } from '@mastra/rag'
+
+const scorer = new CohereRelevanceScorer('rerank-v3.5')
+
+const rerankedResults = await rerank({
+  results: vectorSearchResults,
+  query: 'How do I deploy to production?',
+  scorer,
+  options: {
+    weights: {
+      semantic: 0.5,
+      vector: 0.3,
+      position: 0.2,
+    },
+    topK: 3,
+  },
+})
+```
+
+## Parameters
+
+**results:** (`QueryResult[]`): The vector search results to rerank
+
+**query:** (`string`): The search query text used to evaluate relevance
+
+**scorer:** (`RelevanceScoreProvider`): The relevance scorer to use for reranking
+
+**options?:** (`RerankerFunctionOptions`): Options for the reranking model
+
+The `rerankWithScorer` function accepts any `RelevanceScoreProvider` from @mastra/rag.
+
+> **Note:** For semantic scoring to work properly during re-ranking, each result must include the text content in its `metadata.text` field.
+
+### RerankerFunctionOptions
+
+**weights?:** (`WeightConfig`): numbersemantic?:number (default: 0.4)Weight for semantic relevancenumbervector?:number (default: 0.4)Weight for vector similaritynumberposition?:number (default: 0.2)Weight for position-based scoring
+
+**queryEmbedding?:** (`number[]`): Embedding of the query
+
+**topK?:** (`number`): Number of top results to return (Default: `3`)
+
+## Returns
+
+The function returns an array of `RerankResult` objects:
+
+**result:** (`QueryResult`): The original query result
+
+**score:** (`number`): Combined reranking score (0-1)
+
+**details:** (`ScoringDetails`): Detailed scoring information
+
+### ScoringDetails
+
+**semantic:** (`number`): Semantic relevance score (0-1)
+
+**vector:** (`number`): Vector similarity score (0-1)
+
+**position:** (`number`): Position-based score (0-1)
+
+**queryAnalysis?:** (`object`): numbermagnitude:Magnitude of the querynumber\[]dominantFeatures:Dominant features of the query
+
+## Related
+
+- [createVectorQueryTool](https://mastra.ai/reference/tools/vector-query-tool)

package/dist/docs/references/reference-tools-document-chunker-tool.md

@@ -0,0 +1,89 @@
+# createDocumentChunkerTool()
+
+The `createDocumentChunkerTool()` function creates a tool for splitting documents into smaller chunks for efficient processing and retrieval. It supports different chunking strategies and configurable parameters.
+
+## Basic Usage
+
+```typescript
+import { createDocumentChunkerTool, MDocument } from '@mastra/rag'
+
+const document = new MDocument({
+  text: 'Your document content here...',
+  metadata: { source: 'user-manual' },
+})
+
+const chunker = createDocumentChunkerTool({
+  doc: document,
+  params: {
+    strategy: 'recursive',
+    size: 512,
+    overlap: 50,
+    separator: '\n',
+  },
+})
+
+const { chunks } = await chunker.execute()
+```
+
+## Parameters
+
+**doc:** (`MDocument`): The document to be chunked
+
+**params?:** (`ChunkParams`): Configuration parameters for chunking (Default: `Default chunking parameters`)
+
+### ChunkParams
+
+**strategy?:** (`'recursive'`): The chunking strategy to use (Default: `'recursive'`)
+
+**size?:** (`number`): Target size of each chunk in tokens/characters (Default: `512`)
+
+**overlap?:** (`number`): Number of overlapping tokens/characters between chunks (Default: `50`)
+
+**separator?:** (`string`): Character(s) to use as chunk separator (Default: `'\n'`)
+
+## Returns
+
+**chunks:** (`DocumentChunk[]`): Array of document chunks with their content and metadata
+
+## Example with Custom Parameters
+
+```typescript
+const technicalDoc = new MDocument({
+  text: longDocumentContent,
+  metadata: {
+    type: 'technical',
+    version: '1.0',
+  },
+})
+
+const chunker = createDocumentChunkerTool({
+  doc: technicalDoc,
+  params: {
+    strategy: 'recursive',
+    size: 1024, // Larger chunks
+    overlap: 100, // More overlap
+    separator: '\n\n', // Split on double newlines
+  },
+})
+
+const { chunks } = await chunker.execute()
+
+// Process the chunks
+chunks.forEach((chunk, index) => {
+  console.log(`Chunk ${index + 1} length: ${chunk.content.length}`)
+})
+```
+
+## Tool Details
+
+The chunker is created as a Mastra tool with the following properties:
+
+- **Tool ID**: `Document Chunker {strategy} {size}`
+- **Description**: `Chunks document using {strategy} strategy with size {size} and {overlap} overlap`
+- **Input Schema**: Empty object (no additional inputs required)
+- **Output Schema**: Object containing the chunks array
+
+## Related
+
+- [MDocument](https://mastra.ai/reference/rag/document)
+- [createVectorQueryTool](https://mastra.ai/reference/tools/vector-query-tool)