@mastra/mcp-docs-server 1.1.7 → 1.1.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.docs/docs/agents/agent-approval.md +61 -31
- package/.docs/docs/agents/supervisor-agents.md +1 -1
- package/.docs/docs/getting-started/manual-install.md +1 -1
- package/.docs/docs/index.md +68 -24
- package/.docs/docs/mastra-cloud/setup.md +1 -1
- package/.docs/docs/memory/observational-memory.md +9 -0
- package/.docs/docs/memory/semantic-recall.md +17 -1
- package/.docs/docs/workspace/skills.md +7 -5
- package/.docs/guides/deployment/aws-lambda.md +76 -165
- package/.docs/guides/deployment/azure-app-services.md +38 -61
- package/.docs/guides/deployment/cloudflare.md +1 -1
- package/.docs/guides/deployment/netlify.md +1 -1
- package/.docs/guides/deployment/vercel.md +1 -1
- package/.docs/models/gateways/netlify.md +5 -1
- package/.docs/models/index.md +1 -1
- package/.docs/models/providers/anthropic.md +9 -9
- package/.docs/models/providers/deepseek.md +1 -1
- package/.docs/models/providers/google.md +11 -11
- package/.docs/models/providers/openai.md +19 -19
- package/.docs/models/providers/opencode.md +33 -33
- package/.docs/models/providers/xai.md +3 -3
- package/.docs/reference/agents/agent.md +20 -20
- package/.docs/reference/agents/generate.md +200 -66
- package/.docs/reference/agents/generateLegacy.md +77 -35
- package/.docs/reference/agents/getDefaultGenerateOptions.md +4 -6
- package/.docs/reference/agents/getDefaultOptions.md +4 -6
- package/.docs/reference/agents/getDefaultStreamOptions.md +4 -6
- package/.docs/reference/agents/getDescription.md +1 -1
- package/.docs/reference/agents/getInstructions.md +4 -6
- package/.docs/reference/agents/getLLM.md +6 -8
- package/.docs/reference/agents/getMemory.md +4 -6
- package/.docs/reference/agents/getModel.md +4 -6
- package/.docs/reference/agents/getTools.md +5 -7
- package/.docs/reference/agents/getVoice.md +4 -6
- package/.docs/reference/agents/listAgents.md +4 -6
- package/.docs/reference/agents/listScorers.md +4 -6
- package/.docs/reference/agents/listTools.md +4 -6
- package/.docs/reference/agents/listWorkflows.md +4 -6
- package/.docs/reference/agents/network.md +69 -23
- package/.docs/reference/ai-sdk/chat-route.md +7 -7
- package/.docs/reference/ai-sdk/handle-chat-stream.md +12 -12
- package/.docs/reference/ai-sdk/handle-network-stream.md +4 -4
- package/.docs/reference/ai-sdk/handle-workflow-stream.md +11 -11
- package/.docs/reference/ai-sdk/network-route.md +3 -3
- package/.docs/reference/ai-sdk/to-ai-sdk-stream.md +9 -9
- package/.docs/reference/ai-sdk/to-ai-sdk-v4-messages.md +9 -9
- package/.docs/reference/ai-sdk/to-ai-sdk-v5-messages.md +5 -5
- package/.docs/reference/ai-sdk/with-mastra.md +12 -12
- package/.docs/reference/ai-sdk/workflow-route.md +3 -3
- package/.docs/reference/auth/auth0.md +6 -6
- package/.docs/reference/auth/better-auth.md +5 -5
- package/.docs/reference/auth/clerk.md +5 -5
- package/.docs/reference/auth/firebase.md +7 -7
- package/.docs/reference/auth/jwt.md +1 -1
- package/.docs/reference/auth/supabase.md +4 -4
- package/.docs/reference/auth/workos.md +6 -6
- package/.docs/reference/cli/create-mastra.md +14 -14
- package/.docs/reference/client-js/mastra-client.md +23 -23
- package/.docs/reference/client-js/workflows.md +3 -3
- package/.docs/reference/core/addGateway.md +2 -2
- package/.docs/reference/core/getAgent.md +2 -2
- package/.docs/reference/core/getAgentById.md +2 -2
- package/.docs/reference/core/getDeployer.md +1 -1
- package/.docs/reference/core/getGateway.md +2 -2
- package/.docs/reference/core/getGatewayById.md +2 -2
- package/.docs/reference/core/getLogger.md +1 -1
- package/.docs/reference/core/getMCPServer.md +2 -2
- package/.docs/reference/core/getMCPServerById.md +3 -3
- package/.docs/reference/core/getMemory.md +2 -2
- package/.docs/reference/core/getScorer.md +2 -2
- package/.docs/reference/core/getScorerById.md +2 -2
- package/.docs/reference/core/getServer.md +1 -1
- package/.docs/reference/core/getStorage.md +1 -1
- package/.docs/reference/core/getStoredAgentById.md +18 -20
- package/.docs/reference/core/getTelemetry.md +1 -1
- package/.docs/reference/core/getVector.md +2 -2
- package/.docs/reference/core/getWorkflow.md +3 -3
- package/.docs/reference/core/listAgents.md +1 -1
- package/.docs/reference/core/listGateways.md +1 -1
- package/.docs/reference/core/listLogs.md +9 -11
- package/.docs/reference/core/listLogsByRunId.md +9 -9
- package/.docs/reference/core/listMCPServers.md +1 -1
- package/.docs/reference/core/listMemory.md +1 -1
- package/.docs/reference/core/listScorers.md +1 -1
- package/.docs/reference/core/listStoredAgents.md +9 -11
- package/.docs/reference/core/listVectors.md +1 -1
- package/.docs/reference/core/listWorkflows.md +2 -2
- package/.docs/reference/core/mastra-class.md +17 -17
- package/.docs/reference/core/mastra-model-gateway.md +15 -15
- package/.docs/reference/core/setLogger.md +2 -4
- package/.docs/reference/core/setStorage.md +1 -1
- package/.docs/reference/datasets/addItem.md +20 -4
- package/.docs/reference/datasets/addItems.md +8 -2
- package/.docs/reference/datasets/compareExperiments.md +15 -3
- package/.docs/reference/datasets/create.md +6 -6
- package/.docs/reference/datasets/dataset.md +1 -1
- package/.docs/reference/datasets/delete.md +2 -2
- package/.docs/reference/datasets/deleteExperiment.md +2 -2
- package/.docs/reference/datasets/deleteItem.md +2 -2
- package/.docs/reference/datasets/deleteItems.md +2 -2
- package/.docs/reference/datasets/get.md +2 -2
- package/.docs/reference/datasets/getDetails.md +9 -9
- package/.docs/reference/datasets/getExperiment.md +2 -2
- package/.docs/reference/datasets/getItem.md +3 -3
- package/.docs/reference/datasets/getItemHistory.md +22 -2
- package/.docs/reference/datasets/list.md +7 -3
- package/.docs/reference/datasets/listExperimentResults.md +34 -4
- package/.docs/reference/datasets/listExperiments.md +41 -3
- package/.docs/reference/datasets/listItems.md +18 -6
- package/.docs/reference/datasets/listVersions.md +23 -3
- package/.docs/reference/datasets/startExperiment.md +62 -12
- package/.docs/reference/datasets/startExperimentAsync.md +5 -1
- package/.docs/reference/datasets/update.md +6 -6
- package/.docs/reference/datasets/updateItem.md +5 -5
- package/.docs/reference/deployer.md +8 -8
- package/.docs/reference/evals/answer-relevancy.md +11 -11
- package/.docs/reference/evals/answer-similarity.md +17 -19
- package/.docs/reference/evals/bias.md +10 -10
- package/.docs/reference/evals/completeness.md +3 -3
- package/.docs/reference/evals/content-similarity.md +6 -6
- package/.docs/reference/evals/context-precision.md +4 -4
- package/.docs/reference/evals/context-relevance.md +4 -4
- package/.docs/reference/evals/create-scorer.md +47 -49
- package/.docs/reference/evals/faithfulness.md +11 -11
- package/.docs/reference/evals/hallucination.md +17 -21
- package/.docs/reference/evals/keyword-coverage.md +4 -4
- package/.docs/reference/evals/mastra-scorer.md +14 -14
- package/.docs/reference/evals/noise-sensitivity.md +4 -4
- package/.docs/reference/evals/prompt-alignment.md +4 -4
- package/.docs/reference/evals/run-evals.md +16 -16
- package/.docs/reference/evals/scorer-utils.md +3 -3
- package/.docs/reference/evals/textual-difference.md +3 -3
- package/.docs/reference/evals/tone-consistency.md +3 -3
- package/.docs/reference/evals/tool-call-accuracy.md +5 -5
- package/.docs/reference/evals/toxicity.md +8 -8
- package/.docs/reference/harness/harness-class.md +34 -42
- package/.docs/reference/logging/pino-logger.md +5 -5
- package/.docs/reference/memory/clone-utilities.md +5 -5
- package/.docs/reference/memory/cloneThread.md +17 -21
- package/.docs/reference/memory/createThread.md +10 -10
- package/.docs/reference/memory/deleteMessages.md +2 -2
- package/.docs/reference/memory/getThreadById.md +2 -2
- package/.docs/reference/memory/listThreads.md +5 -5
- package/.docs/reference/memory/memory-class.md +12 -14
- package/.docs/reference/memory/observational-memory.md +102 -94
- package/.docs/reference/memory/recall.md +14 -16
- package/.docs/reference/observability/tracing/configuration.md +27 -10
- package/.docs/reference/observability/tracing/exporters/console-exporter.md +4 -7
- package/.docs/reference/processors/batch-parts-processor.md +8 -10
- package/.docs/reference/processors/language-detector.md +14 -16
- package/.docs/reference/processors/message-history-processor.md +7 -9
- package/.docs/reference/processors/moderation-processor.md +13 -15
- package/.docs/reference/processors/pii-detector.md +14 -16
- package/.docs/reference/processors/processor-interface.md +62 -62
- package/.docs/reference/processors/prompt-injection-detector.md +11 -13
- package/.docs/reference/processors/semantic-recall-processor.md +14 -16
- package/.docs/reference/processors/system-prompt-scrubber.md +12 -14
- package/.docs/reference/processors/token-limiter-processor.md +11 -13
- package/.docs/reference/processors/tool-call-filter.md +5 -7
- package/.docs/reference/processors/tool-search-processor.md +9 -11
- package/.docs/reference/processors/unicode-normalizer.md +8 -10
- package/.docs/reference/processors/working-memory-processor.md +14 -18
- package/.docs/reference/rag/chunk.md +38 -38
- package/.docs/reference/rag/database-config.md +11 -7
- package/.docs/reference/rag/document.md +2 -2
- package/.docs/reference/rag/embeddings.md +12 -12
- package/.docs/reference/rag/extract-params.md +23 -23
- package/.docs/reference/rag/graph-rag.md +12 -12
- package/.docs/reference/rag/rerank.md +25 -17
- package/.docs/reference/rag/rerankWithScorer.md +25 -17
- package/.docs/reference/server/create-route.md +14 -14
- package/.docs/reference/server/express-adapter.md +10 -10
- package/.docs/reference/server/fastify-adapter.md +10 -10
- package/.docs/reference/server/hono-adapter.md +10 -10
- package/.docs/reference/server/koa-adapter.md +10 -10
- package/.docs/reference/server/mastra-server.md +10 -10
- package/.docs/reference/server/register-api-route.md +13 -13
- package/.docs/reference/storage/cloudflare-d1.md +5 -5
- package/.docs/reference/storage/cloudflare.md +6 -6
- package/.docs/reference/storage/composite.md +9 -9
- package/.docs/reference/storage/convex.md +3 -3
- package/.docs/reference/storage/dynamodb.md +9 -9
- package/.docs/reference/storage/lance.md +3 -3
- package/.docs/reference/storage/libsql.md +2 -2
- package/.docs/reference/storage/mongodb.md +5 -5
- package/.docs/reference/storage/mssql.md +2 -2
- package/.docs/reference/storage/postgresql.md +25 -25
- package/.docs/reference/storage/upstash.md +3 -3
- package/.docs/reference/streaming/ChunkType.md +251 -59
- package/.docs/reference/streaming/agents/MastraModelOutput.md +86 -16
- package/.docs/reference/streaming/agents/stream.md +155 -43
- package/.docs/reference/streaming/agents/streamLegacy.md +79 -39
- package/.docs/reference/streaming/workflows/resumeStream.md +18 -8
- package/.docs/reference/streaming/workflows/stream.md +21 -9
- package/.docs/reference/streaming/workflows/timeTravelStream.md +4 -4
- package/.docs/reference/tools/create-tool.md +25 -21
- package/.docs/reference/tools/document-chunker-tool.md +7 -7
- package/.docs/reference/tools/graph-rag-tool.md +16 -18
- package/.docs/reference/tools/mcp-client.md +38 -27
- package/.docs/reference/tools/mcp-server.md +45 -45
- package/.docs/reference/tools/vector-query-tool.md +34 -22
- package/.docs/reference/vectors/astra.md +22 -22
- package/.docs/reference/vectors/chroma.md +44 -44
- package/.docs/reference/vectors/convex.md +26 -26
- package/.docs/reference/vectors/couchbase.md +30 -30
- package/.docs/reference/vectors/duckdb.md +29 -29
- package/.docs/reference/vectors/elasticsearch.md +27 -27
- package/.docs/reference/vectors/lance.md +46 -38
- package/.docs/reference/vectors/libsql.md +31 -31
- package/.docs/reference/vectors/mongodb.md +32 -32
- package/.docs/reference/vectors/opensearch.md +27 -27
- package/.docs/reference/vectors/pg.md +60 -44
- package/.docs/reference/vectors/pinecone.md +38 -38
- package/.docs/reference/vectors/qdrant.md +36 -36
- package/.docs/reference/vectors/s3vectors.md +24 -24
- package/.docs/reference/vectors/turbopuffer.md +29 -29
- package/.docs/reference/vectors/upstash.md +25 -25
- package/.docs/reference/vectors/vectorize.md +27 -27
- package/.docs/reference/voice/azure.md +19 -15
- package/.docs/reference/voice/cloudflare.md +10 -12
- package/.docs/reference/voice/composite-voice.md +10 -10
- package/.docs/reference/voice/deepgram.md +20 -14
- package/.docs/reference/voice/elevenlabs.md +20 -22
- package/.docs/reference/voice/google-gemini-live.md +42 -44
- package/.docs/reference/voice/google.md +25 -21
- package/.docs/reference/voice/mastra-voice.md +20 -20
- package/.docs/reference/voice/murf.md +24 -28
- package/.docs/reference/voice/openai-realtime.md +26 -26
- package/.docs/reference/voice/openai.md +14 -12
- package/.docs/reference/voice/playai.md +13 -15
- package/.docs/reference/voice/sarvam.md +21 -25
- package/.docs/reference/voice/speechify.md +11 -13
- package/.docs/reference/voice/voice.addInstructions.md +1 -1
- package/.docs/reference/voice/voice.addTools.md +1 -1
- package/.docs/reference/voice/voice.answer.md +1 -1
- package/.docs/reference/voice/voice.connect.md +3 -3
- package/.docs/reference/voice/voice.events.md +11 -11
- package/.docs/reference/voice/voice.getSpeakers.md +30 -30
- package/.docs/reference/voice/voice.listen.md +9 -9
- package/.docs/reference/voice/voice.off.md +2 -2
- package/.docs/reference/voice/voice.on.md +2 -2
- package/.docs/reference/voice/voice.send.md +1 -1
- package/.docs/reference/voice/voice.speak.md +11 -11
- package/.docs/reference/voice/voice.updateConfig.md +3 -3
- package/.docs/reference/workflows/run-methods/cancel.md +2 -2
- package/.docs/reference/workflows/run-methods/restart.md +17 -5
- package/.docs/reference/workflows/run-methods/resume.md +23 -9
- package/.docs/reference/workflows/run-methods/start.md +22 -8
- package/.docs/reference/workflows/run-methods/startAsync.md +12 -6
- package/.docs/reference/workflows/run-methods/timeTravel.md +29 -13
- package/.docs/reference/workflows/run.md +12 -12
- package/.docs/reference/workflows/step.md +24 -26
- package/.docs/reference/workflows/workflow-methods/branch.md +2 -2
- package/.docs/reference/workflows/workflow-methods/commit.md +1 -1
- package/.docs/reference/workflows/workflow-methods/create-run.md +4 -4
- package/.docs/reference/workflows/workflow-methods/dountil.md +3 -3
- package/.docs/reference/workflows/workflow-methods/dowhile.md +3 -3
- package/.docs/reference/workflows/workflow-methods/foreach.md +9 -9
- package/.docs/reference/workflows/workflow-methods/map.md +2 -2
- package/.docs/reference/workflows/workflow-methods/parallel.md +2 -2
- package/.docs/reference/workflows/workflow-methods/sleep.md +2 -2
- package/.docs/reference/workflows/workflow-methods/sleepUntil.md +2 -2
- package/.docs/reference/workflows/workflow-methods/then.md +2 -2
- package/.docs/reference/workflows/workflow.md +40 -50
- package/.docs/reference/workspace/daytona-sandbox.md +33 -33
- package/.docs/reference/workspace/e2b-sandbox.md +20 -20
- package/.docs/reference/workspace/filesystem.md +22 -22
- package/.docs/reference/workspace/gcs-filesystem.md +15 -15
- package/.docs/reference/workspace/local-filesystem.md +35 -35
- package/.docs/reference/workspace/local-sandbox.md +26 -26
- package/.docs/reference/workspace/s3-filesystem.md +18 -18
- package/.docs/reference/workspace/sandbox.md +8 -8
- package/.docs/reference/workspace/workspace-class.md +30 -34
- package/CHANGELOG.md +16 -0
- package/package.json +6 -6
- package/.docs/docs/getting-started/start.md +0 -28
|
@@ -15,25 +15,23 @@ const processor = new UnicodeNormalizer({
|
|
|
15
15
|
|
|
16
16
|
## Constructor parameters
|
|
17
17
|
|
|
18
|
-
**options
|
|
18
|
+
**options** (`Options`): Configuration options for Unicode text normalization
|
|
19
19
|
|
|
20
|
-
|
|
20
|
+
**options.stripControlChars** (`boolean`): Whether to strip control characters. When enabled, removes control characters except , ,
|
|
21
21
|
|
|
22
|
-
**
|
|
22
|
+
**options.preserveEmojis** (`boolean`): Whether to preserve emojis. When disabled, emojis may be removed if they contain control characters
|
|
23
23
|
|
|
24
|
-
**
|
|
24
|
+
**options.collapseWhitespace** (`boolean`): Whether to collapse consecutive whitespace. When enabled, multiple spaces/tabs/newlines are collapsed to single instances
|
|
25
25
|
|
|
26
|
-
**
|
|
27
|
-
|
|
28
|
-
**trim?:** (`boolean`): Whether to trim leading and trailing whitespace
|
|
26
|
+
**options.trim** (`boolean`): Whether to trim leading and trailing whitespace
|
|
29
27
|
|
|
30
28
|
## Returns
|
|
31
29
|
|
|
32
|
-
**id
|
|
30
|
+
**id** (`string`): Processor identifier set to 'unicode-normalizer'
|
|
33
31
|
|
|
34
|
-
**name
|
|
32
|
+
**name** (`string`): Optional processor display name
|
|
35
33
|
|
|
36
|
-
**processInput
|
|
34
|
+
**processInput** (`(args: { messages: MastraDBMessage[]; abort: (reason?: string) => never }) => MastraDBMessage[]`): Processes input messages to normalize Unicode text
|
|
37
35
|
|
|
38
36
|
## Extended usage example
|
|
39
37
|
|
|
@@ -23,39 +23,35 @@ const processor = new WorkingMemory({
|
|
|
23
23
|
|
|
24
24
|
## Constructor parameters
|
|
25
25
|
|
|
26
|
-
**options
|
|
26
|
+
**options** (`Options`): Configuration options for the working memory processor
|
|
27
27
|
|
|
28
|
-
|
|
28
|
+
**options.storage** (`MemoryStorage`): Storage instance for retrieving working memory data
|
|
29
29
|
|
|
30
|
-
**
|
|
30
|
+
**options.template** (`WorkingMemoryTemplate`): Template defining the format and structure of working memory
|
|
31
31
|
|
|
32
|
-
**template
|
|
32
|
+
**options.template.format** (`'markdown' | 'json'`): Format of the working memory content
|
|
33
33
|
|
|
34
|
-
**
|
|
34
|
+
**options.template.content** (`string`): Template content defining the structure of working memory data
|
|
35
35
|
|
|
36
|
-
**
|
|
36
|
+
**options.scope** (`'thread' | 'resource'`): Scope of working memory. 'thread' scopes to current thread, 'resource' shares across all threads for the resource
|
|
37
37
|
|
|
38
|
-
**
|
|
38
|
+
**options.useVNext** (`boolean`): Use the next-generation instruction format with improved guidelines
|
|
39
39
|
|
|
40
|
-
**
|
|
40
|
+
**options.readOnly** (`boolean`): When true, working memory is provided as read-only context. The data is injected into the conversation but without the updateWorkingMemory tool or update instructions. Useful for agents that should reference working memory without modifying it.
|
|
41
41
|
|
|
42
|
-
**
|
|
42
|
+
**options.templateProvider** (`{ getWorkingMemoryTemplate(args: { memoryConfig?: MemoryConfig }): Promise<WorkingMemoryTemplate | null> }`): Dynamic template provider for runtime template resolution
|
|
43
43
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
**format:** (`'markdown' | 'json'`): Format of the working memory content
|
|
47
|
-
|
|
48
|
-
**content:** (`string`): Template content defining the structure of working memory data
|
|
44
|
+
**options.logger** (`IMastraLogger`): Optional logger instance for structured logging
|
|
49
45
|
|
|
50
46
|
## Returns
|
|
51
47
|
|
|
52
|
-
**id
|
|
48
|
+
**id** (`string`): Processor identifier set to 'working-memory'
|
|
53
49
|
|
|
54
|
-
**name
|
|
50
|
+
**name** (`string`): Processor display name set to 'WorkingMemory'
|
|
55
51
|
|
|
56
|
-
**defaultWorkingMemoryTemplate
|
|
52
|
+
**defaultWorkingMemoryTemplate** (`string`): The default markdown template used when no custom template is provided
|
|
57
53
|
|
|
58
|
-
**processInput
|
|
54
|
+
**processInput** (`(args: { messages: MastraDBMessage[]; messageList: MessageList; abort: (reason?: string) => never; requestContext?: RequestContext }) => Promise<MessageList | MastraDBMessage[]>`): Retrieves working memory and adds it as a system message to the message list
|
|
59
55
|
|
|
60
56
|
## Extended usage example
|
|
61
57
|
|
|
@@ -39,21 +39,21 @@ const chunksWithMetadata = await doc.chunk({
|
|
|
39
39
|
|
|
40
40
|
The following parameters are available for all chunking strategies. **Important:** Each strategy will only utilize a subset of these parameters relevant to its specific use case.
|
|
41
41
|
|
|
42
|
-
**strategy
|
|
42
|
+
**strategy** (`'recursive' | 'character' | 'token' | 'markdown' | 'semantic-markdown' | 'html' | 'json' | 'latex' | 'sentence'`): The chunking strategy to use. If not specified, defaults based on document type. Depending on the chunking strategy, there are additional optionals. Defaults: .md files → 'markdown', .html/.htm → 'html', .json → 'json', .tex → 'latex', others → 'recursive'
|
|
43
43
|
|
|
44
|
-
**maxSize
|
|
44
|
+
**maxSize** (`number`): Maximum size of each chunk. \*\*Note:\*\* Some strategy configurations (markdown with headers, HTML with headers) ignore this parameter. (Default: `4000`)
|
|
45
45
|
|
|
46
|
-
**overlap
|
|
46
|
+
**overlap** (`number`): Number of characters/tokens that overlap between chunks. (Default: `50`)
|
|
47
47
|
|
|
48
|
-
**lengthFunction
|
|
48
|
+
**lengthFunction** (`(text: string) => number`): Function to calculate text length. Defaults to character count.
|
|
49
49
|
|
|
50
|
-
**separatorPosition
|
|
50
|
+
**separatorPosition** (`'start' | 'end'`): Where to position the separator in chunks. 'start' attaches to beginning of next chunk, 'end' attaches to end of current chunk. If not specified, separators are discarded.
|
|
51
51
|
|
|
52
|
-
**addStartIndex
|
|
52
|
+
**addStartIndex** (`boolean`): Whether to add start index metadata to chunks. (Default: `false`)
|
|
53
53
|
|
|
54
|
-
**stripWhitespace
|
|
54
|
+
**stripWhitespace** (`boolean`): Whether to strip whitespace from chunks. (Default: `true`)
|
|
55
55
|
|
|
56
|
-
**extract
|
|
56
|
+
**extract** (`ExtractParams`): Metadata extraction configuration.
|
|
57
57
|
|
|
58
58
|
See [ExtractParams reference](https://mastra.ai/reference/rag/extract-params) for details on the `extract` parameter.
|
|
59
59
|
|
|
@@ -126,83 +126,83 @@ The options documented below are passed directly at the top level of the configu
|
|
|
126
126
|
|
|
127
127
|
### Character
|
|
128
128
|
|
|
129
|
-
**separators
|
|
129
|
+
**separators** (`string[]`): Array of separators to try in order of preference. The strategy will attempt to split on the first separator, then fall back to subsequent ones.
|
|
130
130
|
|
|
131
|
-
**isSeparatorRegex
|
|
131
|
+
**isSeparatorRegex** (`boolean`): Whether the separator is a regex pattern (Default: `false`)
|
|
132
132
|
|
|
133
133
|
### Recursive
|
|
134
134
|
|
|
135
|
-
**separators
|
|
135
|
+
**separators** (`string[]`): Array of separators to try in order of preference. The strategy will attempt to split on the first separator, then fall back to subsequent ones.
|
|
136
136
|
|
|
137
|
-
**isSeparatorRegex
|
|
137
|
+
**isSeparatorRegex** (`boolean`): Whether the separators are regex patterns (Default: `false`)
|
|
138
138
|
|
|
139
|
-
**language
|
|
139
|
+
**language** (`Language`): Programming or markup language for language-specific splitting behavior. See Language enum for supported values.
|
|
140
140
|
|
|
141
141
|
### Sentence
|
|
142
142
|
|
|
143
|
-
**maxSize
|
|
143
|
+
**maxSize** (`number`): Maximum size of each chunk (required for sentence strategy)
|
|
144
144
|
|
|
145
|
-
**minSize
|
|
145
|
+
**minSize** (`number`): Minimum size of each chunk. Chunks smaller than this will be merged with adjacent chunks when possible. (Default: `50`)
|
|
146
146
|
|
|
147
|
-
**targetSize
|
|
147
|
+
**targetSize** (`number`): Preferred target size for chunks. Defaults to 80% of maxSize. The strategy will try to create chunks close to this size.
|
|
148
148
|
|
|
149
|
-
**sentenceEnders
|
|
149
|
+
**sentenceEnders** (`string[]`): Array of characters that mark sentence endings for splitting boundaries. (Default: `['.', '!', '?']`)
|
|
150
150
|
|
|
151
|
-
**fallbackToWords
|
|
151
|
+
**fallbackToWords** (`boolean`): Whether to fall back to word-level splitting for sentences that exceed maxSize. (Default: `true`)
|
|
152
152
|
|
|
153
|
-
**fallbackToCharacters
|
|
153
|
+
**fallbackToCharacters** (`boolean`): Whether to fall back to character-level splitting for words that exceed maxSize. Only applies if fallbackToWords is enabled. (Default: `true`)
|
|
154
154
|
|
|
155
155
|
### HTML
|
|
156
156
|
|
|
157
|
-
**headers
|
|
157
|
+
**headers** (`Array<[string, string]>`): Array of \[selector, metadata key] pairs for header-based splitting
|
|
158
158
|
|
|
159
|
-
**sections
|
|
159
|
+
**sections** (`Array<[string, string]>`): Array of \[selector, metadata key] pairs for section-based splitting
|
|
160
160
|
|
|
161
|
-
**returnEachLine
|
|
161
|
+
**returnEachLine** (`boolean`): Whether to return each line as a separate chunk
|
|
162
162
|
|
|
163
163
|
**Important:** When using the HTML strategy, all general options are ignored. Use `headers` for header-based splitting or `sections` for section-based splitting. If used together, `sections` will be ignored.
|
|
164
164
|
|
|
165
165
|
### Markdown
|
|
166
166
|
|
|
167
|
-
**headers
|
|
167
|
+
**headers** (`Array<[string, string]>`): Array of \[header level, metadata key] pairs
|
|
168
168
|
|
|
169
|
-
**stripHeaders
|
|
169
|
+
**stripHeaders** (`boolean`): Whether to remove headers from the output
|
|
170
170
|
|
|
171
|
-
**returnEachLine
|
|
171
|
+
**returnEachLine** (`boolean`): Whether to return each line as a separate chunk
|
|
172
172
|
|
|
173
173
|
**Important:** When using the `headers` option, the markdown strategy ignores all general options and content is split based on the markdown header structure. To use size-based chunking with markdown, omit the `headers` parameter.
|
|
174
174
|
|
|
175
175
|
### Semantic Markdown
|
|
176
176
|
|
|
177
|
-
**joinThreshold
|
|
177
|
+
**joinThreshold** (`number`): Maximum token count for merging related sections. Sections exceeding this limit individually are left intact, but smaller sections are merged with siblings or parents if the combined size stays under this threshold. (Default: `500`)
|
|
178
178
|
|
|
179
|
-
**modelName
|
|
179
|
+
**modelName** (`string`): Name of the model for tokenization. If provided, the model's underlying tokenization \`encodingName\` will be used.
|
|
180
180
|
|
|
181
|
-
**encodingName
|
|
181
|
+
**encodingName** (`string`): Name of the token encoding to use. Derived from \`modelName\` if available. (Default: `cl100k_base`)
|
|
182
182
|
|
|
183
|
-
**allowedSpecial
|
|
183
|
+
**allowedSpecial** (`Set<string> | 'all'`): Set of special tokens allowed during tokenization, or 'all' to allow all special tokens
|
|
184
184
|
|
|
185
|
-
**disallowedSpecial
|
|
185
|
+
**disallowedSpecial** (`Set<string> | 'all'`): Set of special tokens to disallow during tokenization, or 'all' to disallow all special tokens (Default: `all`)
|
|
186
186
|
|
|
187
187
|
### Token
|
|
188
188
|
|
|
189
|
-
**encodingName
|
|
189
|
+
**encodingName** (`string`): Name of the token encoding to use
|
|
190
190
|
|
|
191
|
-
**modelName
|
|
191
|
+
**modelName** (`string`): Name of the model for tokenization
|
|
192
192
|
|
|
193
|
-
**allowedSpecial
|
|
193
|
+
**allowedSpecial** (`Set<string> | 'all'`): Set of special tokens allowed during tokenization, or 'all' to allow all special tokens
|
|
194
194
|
|
|
195
|
-
**disallowedSpecial
|
|
195
|
+
**disallowedSpecial** (`Set<string> | 'all'`): Set of special tokens to disallow during tokenization, or 'all' to disallow all special tokens
|
|
196
196
|
|
|
197
197
|
### JSON
|
|
198
198
|
|
|
199
|
-
**maxSize
|
|
199
|
+
**maxSize** (`number`): Maximum size of each chunk
|
|
200
200
|
|
|
201
|
-
**minSize
|
|
201
|
+
**minSize** (`number`): Minimum size of each chunk
|
|
202
202
|
|
|
203
|
-
**ensureAscii
|
|
203
|
+
**ensureAscii** (`boolean`): Whether to ensure ASCII encoding
|
|
204
204
|
|
|
205
|
-
**convertLists
|
|
205
|
+
**convertLists** (`boolean`): Whether to convert lists in the JSON
|
|
206
206
|
|
|
207
207
|
### Latex
|
|
208
208
|
|
|
@@ -19,9 +19,13 @@ export type DatabaseConfig = {
|
|
|
19
19
|
|
|
20
20
|
Configuration options specific to Pinecone vector store.
|
|
21
21
|
|
|
22
|
-
**namespace
|
|
22
|
+
**namespace** (`string`): Pinecone namespace for organizing and isolating vectors within the same index. Useful for multi-tenancy or environment separation.
|
|
23
23
|
|
|
24
|
-
**sparseVector
|
|
24
|
+
**sparseVector** (`{ indices: number[]; values: number[]; }`): Sparse vector for hybrid search combining dense and sparse embeddings. Enables better search quality for keyword-based queries. The indices and values arrays must be the same length.
|
|
25
|
+
|
|
26
|
+
**sparseVector.indices** (`number[]`): Array of indices for sparse vector components
|
|
27
|
+
|
|
28
|
+
**sparseVector.values** (`number[]`): Array of values corresponding to the indices
|
|
25
29
|
|
|
26
30
|
**Use Cases:**
|
|
27
31
|
|
|
@@ -33,11 +37,11 @@ Configuration options specific to Pinecone vector store.
|
|
|
33
37
|
|
|
34
38
|
Configuration options specific to PostgreSQL with pgvector extension.
|
|
35
39
|
|
|
36
|
-
**minScore
|
|
40
|
+
**minScore** (`number`): Minimum similarity score threshold for results. Only vectors with similarity scores above this value will be returned.
|
|
37
41
|
|
|
38
|
-
**ef
|
|
42
|
+
**ef** (`number`): HNSW search parameter that controls the size of the dynamic candidate list during search. Higher values improve accuracy at the cost of speed. Typically set between topK and 200.
|
|
39
43
|
|
|
40
|
-
**probes
|
|
44
|
+
**probes** (`number`): IVFFlat probe parameter that specifies the number of index cells to visit during search. Higher values improve recall at the cost of speed.
|
|
41
45
|
|
|
42
46
|
**Performance Guidelines:**
|
|
43
47
|
|
|
@@ -55,9 +59,9 @@ Configuration options specific to PostgreSQL with pgvector extension.
|
|
|
55
59
|
|
|
56
60
|
Configuration options specific to Chroma vector store.
|
|
57
61
|
|
|
58
|
-
**where
|
|
62
|
+
**where** (`Record<string, any>`): Metadata filtering conditions using MongoDB-style query syntax. Filters results based on metadata fields.
|
|
59
63
|
|
|
60
|
-
**whereDocument
|
|
64
|
+
**whereDocument** (`Record<string, any>`): Document content filtering conditions. Allows filtering based on the actual document text content.
|
|
61
65
|
|
|
62
66
|
**Filter Syntax Examples:**
|
|
63
67
|
|
|
@@ -4,9 +4,9 @@ The MDocument class processes documents for RAG applications. The main methods a
|
|
|
4
4
|
|
|
5
5
|
## Constructor
|
|
6
6
|
|
|
7
|
-
**docs
|
|
7
|
+
**docs** (`Array<{ text: string, metadata?: Record<string, any> }>`): Array of document chunks with their text content and optional metadata
|
|
8
8
|
|
|
9
|
-
**type
|
|
9
|
+
**type** (`'text' | 'html' | 'markdown' | 'json' | 'latex'`): Type of document content
|
|
10
10
|
|
|
11
11
|
## Static Methods
|
|
12
12
|
|
|
@@ -19,19 +19,19 @@ const result = await embed({
|
|
|
19
19
|
|
|
20
20
|
### Parameters
|
|
21
21
|
|
|
22
|
-
**model
|
|
22
|
+
**model** (`EmbeddingModel`): The embedding model to use (e.g. openai.embedding('text-embedding-3-small'))
|
|
23
23
|
|
|
24
|
-
**value
|
|
24
|
+
**value** (`string | Record<string, any>`): The text content or object to embed
|
|
25
25
|
|
|
26
|
-
**maxRetries
|
|
26
|
+
**maxRetries** (`number`): Maximum number of retries per embedding call. Set to 0 to disable retries. (Default: `2`)
|
|
27
27
|
|
|
28
|
-
**abortSignal
|
|
28
|
+
**abortSignal** (`AbortSignal`): Optional abort signal to cancel the request
|
|
29
29
|
|
|
30
|
-
**headers
|
|
30
|
+
**headers** (`Record<string, string>`): Additional HTTP headers for the request (only for HTTP-based providers)
|
|
31
31
|
|
|
32
32
|
### Return Value
|
|
33
33
|
|
|
34
|
-
**embedding
|
|
34
|
+
**embedding** (`number[]`): The embedding vector for the input
|
|
35
35
|
|
|
36
36
|
## Multiple Embeddings
|
|
37
37
|
|
|
@@ -49,19 +49,19 @@ const result = await embedMany({
|
|
|
49
49
|
|
|
50
50
|
### Parameters
|
|
51
51
|
|
|
52
|
-
**model
|
|
52
|
+
**model** (`EmbeddingModel`): The embedding model to use (e.g. openai.embedding('text-embedding-3-small'))
|
|
53
53
|
|
|
54
|
-
**values
|
|
54
|
+
**values** (`string[] | Record<string, any>[]`): Array of text content or objects to embed
|
|
55
55
|
|
|
56
|
-
**maxRetries
|
|
56
|
+
**maxRetries** (`number`): Maximum number of retries per embedding call. Set to 0 to disable retries. (Default: `2`)
|
|
57
57
|
|
|
58
|
-
**abortSignal
|
|
58
|
+
**abortSignal** (`AbortSignal`): Optional abort signal to cancel the request
|
|
59
59
|
|
|
60
|
-
**headers
|
|
60
|
+
**headers** (`Record<string, string>`): Additional HTTP headers for the request (only for HTTP-based providers)
|
|
61
61
|
|
|
62
62
|
### Return Value
|
|
63
63
|
|
|
64
|
-
**embeddings
|
|
64
|
+
**embeddings** (`number[][]`): Array of embedding vectors corresponding to the input values
|
|
65
65
|
|
|
66
66
|
## Example Usage
|
|
67
67
|
|
|
@@ -28,63 +28,63 @@ const chunks = await doc.chunk({
|
|
|
28
28
|
|
|
29
29
|
The `extract` parameter accepts the following fields:
|
|
30
30
|
|
|
31
|
-
**title
|
|
31
|
+
**title** (`boolean | TitleExtractorsArgs`): Enable title extraction. Set to true for default settings, or provide custom configuration.
|
|
32
32
|
|
|
33
|
-
**summary
|
|
33
|
+
**summary** (`boolean | SummaryExtractArgs`): Enable summary extraction. Set to true for default settings, or provide custom configuration.
|
|
34
34
|
|
|
35
|
-
**questions
|
|
35
|
+
**questions** (`boolean | QuestionAnswerExtractArgs`): Enable question generation. Set to true for default settings, or provide custom configuration.
|
|
36
36
|
|
|
37
|
-
**keywords
|
|
37
|
+
**keywords** (`boolean | KeywordExtractArgs`): Enable keyword extraction. Set to true for default settings, or provide custom configuration.
|
|
38
38
|
|
|
39
|
-
**schema
|
|
39
|
+
**schema** (`SchemaExtractArgs`): Enable structured metadata extraction using a Zod schema.
|
|
40
40
|
|
|
41
41
|
## Extractor Arguments
|
|
42
42
|
|
|
43
43
|
### TitleExtractorsArgs
|
|
44
44
|
|
|
45
|
-
**llm
|
|
45
|
+
**llm** (`MastraLanguageModel`): AI SDK language model to use for title extraction
|
|
46
46
|
|
|
47
|
-
**nodes
|
|
47
|
+
**nodes** (`number`): Number of title nodes to extract
|
|
48
48
|
|
|
49
|
-
**nodeTemplate
|
|
49
|
+
**nodeTemplate** (`string`): Custom prompt template for title node extraction. Must include {context} placeholder
|
|
50
50
|
|
|
51
|
-
**combineTemplate
|
|
51
|
+
**combineTemplate** (`string`): Custom prompt template for combining titles. Must include {context} placeholder
|
|
52
52
|
|
|
53
53
|
### SummaryExtractArgs
|
|
54
54
|
|
|
55
|
-
**llm
|
|
55
|
+
**llm** (`MastraLanguageModel`): AI SDK language model to use for summary extraction
|
|
56
56
|
|
|
57
|
-
**summaries
|
|
57
|
+
**summaries** (`('self' | 'prev' | 'next')[]`): List of summary types to generate. Can only include 'self' (current chunk), 'prev' (previous chunk), or 'next' (next chunk)
|
|
58
58
|
|
|
59
|
-
**promptTemplate
|
|
59
|
+
**promptTemplate** (`string`): Custom prompt template for summary generation. Must include {context} placeholder
|
|
60
60
|
|
|
61
61
|
### QuestionAnswerExtractArgs
|
|
62
62
|
|
|
63
|
-
**llm
|
|
63
|
+
**llm** (`MastraLanguageModel`): AI SDK language model to use for question generation
|
|
64
64
|
|
|
65
|
-
**questions
|
|
65
|
+
**questions** (`number`): Number of questions to generate
|
|
66
66
|
|
|
67
|
-
**promptTemplate
|
|
67
|
+
**promptTemplate** (`string`): Custom prompt template for question generation. Must include both {context} and {numQuestions} placeholders
|
|
68
68
|
|
|
69
|
-
**embeddingOnly
|
|
69
|
+
**embeddingOnly** (`boolean`): If true, only generate embeddings without actual questions
|
|
70
70
|
|
|
71
71
|
### KeywordExtractArgs
|
|
72
72
|
|
|
73
|
-
**llm
|
|
73
|
+
**llm** (`MastraLanguageModel`): AI SDK language model to use for keyword extraction
|
|
74
74
|
|
|
75
|
-
**keywords
|
|
75
|
+
**keywords** (`number`): Number of keywords to extract
|
|
76
76
|
|
|
77
|
-
**promptTemplate
|
|
77
|
+
**promptTemplate** (`string`): Custom prompt template for keyword extraction. Must include both {context} and {maxKeywords} placeholders
|
|
78
78
|
|
|
79
79
|
### SchemaExtractArgs
|
|
80
80
|
|
|
81
|
-
**schema
|
|
81
|
+
**schema** (`ZodType`): Zod schema defining the structure of the data to extract.
|
|
82
82
|
|
|
83
|
-
**llm
|
|
83
|
+
**llm** (`MastraLanguageModel`): AI SDK language model to use for extraction.
|
|
84
84
|
|
|
85
|
-
**instructions
|
|
85
|
+
**instructions** (`string`): Instructions for the LLM on what to extract.
|
|
86
86
|
|
|
87
|
-
**metadataKey
|
|
87
|
+
**metadataKey** (`string`): Key to nest extraction results under. If omitted, results are spread into the metadata object.
|
|
88
88
|
|
|
89
89
|
## Advanced Example
|
|
90
90
|
|
|
@@ -26,9 +26,9 @@ const results = await graphRag.query({
|
|
|
26
26
|
|
|
27
27
|
## Constructor Parameters
|
|
28
28
|
|
|
29
|
-
**dimension
|
|
29
|
+
**dimension** (`number`): Dimension of the embedding vectors (Default: `1536`)
|
|
30
30
|
|
|
31
|
-
**threshold
|
|
31
|
+
**threshold** (`number`): Similarity threshold for creating edges between nodes (0-1) (Default: `0.7`)
|
|
32
32
|
|
|
33
33
|
## Methods
|
|
34
34
|
|
|
@@ -42,9 +42,9 @@ createGraph(chunks: GraphChunk[], embeddings: GraphEmbedding[]): void
|
|
|
42
42
|
|
|
43
43
|
#### Parameters
|
|
44
44
|
|
|
45
|
-
**chunks
|
|
45
|
+
**chunks** (`GraphChunk[]`): Array of document chunks with text and metadata
|
|
46
46
|
|
|
47
|
-
**embeddings
|
|
47
|
+
**embeddings** (`GraphEmbedding[]`): Array of embeddings corresponding to chunks
|
|
48
48
|
|
|
49
49
|
### query
|
|
50
50
|
|
|
@@ -66,25 +66,25 @@ query({
|
|
|
66
66
|
|
|
67
67
|
#### Parameters
|
|
68
68
|
|
|
69
|
-
**query
|
|
69
|
+
**query** (`number[]`): Query embedding vector
|
|
70
70
|
|
|
71
|
-
**topK
|
|
71
|
+
**topK** (`number`): Number of results to return (Default: `10`)
|
|
72
72
|
|
|
73
|
-
**randomWalkSteps
|
|
73
|
+
**randomWalkSteps** (`number`): Number of steps in random walk (Default: `100`)
|
|
74
74
|
|
|
75
|
-
**restartProb
|
|
75
|
+
**restartProb** (`number`): Probability of restarting walk from query node (Default: `0.15`)
|
|
76
76
|
|
|
77
77
|
#### Returns
|
|
78
78
|
|
|
79
79
|
Returns an array of `RankedNode` objects, where each node contains:
|
|
80
80
|
|
|
81
|
-
**id
|
|
81
|
+
**id** (`string`): Unique identifier for the node
|
|
82
82
|
|
|
83
|
-
**content
|
|
83
|
+
**content** (`string`): Text content of the document chunk
|
|
84
84
|
|
|
85
|
-
**metadata
|
|
85
|
+
**metadata** (`Record<string, any>`): Additional metadata associated with the chunk
|
|
86
86
|
|
|
87
|
-
**score
|
|
87
|
+
**score** (`number`): Combined relevance score from graph traversal
|
|
88
88
|
|
|
89
89
|
## Advanced Example
|
|
90
90
|
|
|
@@ -30,45 +30,53 @@ const rerankedResults = await rerank(vectorSearchResults, 'How do I deploy to pr
|
|
|
30
30
|
|
|
31
31
|
## Parameters
|
|
32
32
|
|
|
33
|
-
**results
|
|
33
|
+
**results** (`QueryResult[]`): The vector search results to rerank
|
|
34
34
|
|
|
35
|
-
**query
|
|
35
|
+
**query** (`string`): The search query text used to evaluate relevance
|
|
36
36
|
|
|
37
|
-
**model
|
|
37
|
+
**model** (`MastraLanguageModel`): The language Model to use for reranking
|
|
38
38
|
|
|
39
|
-
**options
|
|
39
|
+
**options** (`RerankerFunctionOptions`): Options for the reranking model
|
|
40
40
|
|
|
41
|
-
|
|
41
|
+
**options.weights** (`WeightConfig`): Weights for different scoring components (must add up to 1)
|
|
42
42
|
|
|
43
|
-
|
|
43
|
+
**options.weights.semantic** (`number (default: 0.4)`): Weight for semantic relevance
|
|
44
|
+
|
|
45
|
+
**options.weights.vector** (`number (default: 0.4)`): Weight for vector similarity
|
|
46
|
+
|
|
47
|
+
**options.weights.position** (`number (default: 0.2)`): Weight for position-based scoring
|
|
44
48
|
|
|
45
|
-
|
|
49
|
+
**options.queryEmbedding** (`number[]`): Embedding of the query
|
|
46
50
|
|
|
47
|
-
**
|
|
51
|
+
**options.topK** (`number`): Number of top results to return
|
|
48
52
|
|
|
49
|
-
|
|
53
|
+
The rerank function accepts any LanguageModel from the Vercel AI SDK. When using the Cohere model `rerank-v3.5`, it will automatically use Cohere's reranking capabilities.
|
|
50
54
|
|
|
51
|
-
**
|
|
55
|
+
> **Note:** For semantic scoring to work properly during re-ranking, each result must include the text content in its `metadata.text` field.
|
|
52
56
|
|
|
53
57
|
## Returns
|
|
54
58
|
|
|
55
59
|
The function returns an array of `RerankResult` objects:
|
|
56
60
|
|
|
57
|
-
**result
|
|
61
|
+
**result** (`QueryResult`): The original query result
|
|
58
62
|
|
|
59
|
-
**score
|
|
63
|
+
**score** (`number`): Combined reranking score (0-1)
|
|
60
64
|
|
|
61
|
-
**details
|
|
65
|
+
**details** (`ScoringDetails`): Detailed scoring information
|
|
62
66
|
|
|
63
67
|
### ScoringDetails
|
|
64
68
|
|
|
65
|
-
**semantic
|
|
69
|
+
**semantic** (`number`): Semantic relevance score (0-1)
|
|
70
|
+
|
|
71
|
+
**vector** (`number`): Vector similarity score (0-1)
|
|
72
|
+
|
|
73
|
+
**position** (`number`): Position-based score (0-1)
|
|
66
74
|
|
|
67
|
-
**
|
|
75
|
+
**queryAnalysis** (`object`): Query analysis details
|
|
68
76
|
|
|
69
|
-
**
|
|
77
|
+
**queryAnalysis.magnitude**: Magnitude of the query
|
|
70
78
|
|
|
71
|
-
**queryAnalysis
|
|
79
|
+
**queryAnalysis.dominantFeatures**: Dominant features of the query
|
|
72
80
|
|
|
73
81
|
## Related
|
|
74
82
|
|