@mindstudio-ai/agent 0.1.89 → 0.1.91

package/llms.txt

@@ -417,8 +417,9 @@ Download a file from a URL and extract its text content. Supports PDFs (includin
 - Handles both text-layer PDFs and image-based/scanned PDFs (e.g. Figma/Canva exports, scanned documents). Image-based PDFs are processed with OCR automatically — there is no need to convert PDF pages to images first.
 - Accepts a single URL, a comma-separated list of URLs, or a JSON array of URLs.
 - Files are rehosted on the MindStudio CDN before extraction.
+- Optionally set `model` to a specific document-extraction model (`mistral-ocr-latest`, `llamaparse`, `google-document-ai`); omit to use the platform default.
 - Maximum file size is 50MB per URL.
-- Input: `{ url: string | string[] }`
+- Input: `{ url: string | string[], model?: string }`
 - Output: `{ text: string | string[] }`
 
 #### fetchDataSourceDocument
@@ -1856,11 +1857,21 @@ export const Policies = dataSources.defineDataSource('policies');
 
 // search
 const { results } = await Policies.search('what are the payment terms?', { topK: 5 });
-// options: topK (default 5, max 50), scoreThreshold, rerank, hybrid — the last two default on
+// options: topK (default 5, max 50), scoreThreshold, filter, mode, maxPerDocument, highlight, rerank, hybrid
 const context = results.map((r) => r.text).join(String.fromCharCode(10, 10));
 
+// narrow before ranking: metadata tags, filename, pages, required words
+await Policies.search('termination clause', {
+  filter: { metadata: { department: 'legal', year: [2025, 2026] }, phrase: 'notice period' },
+  maxPerDocument: 2,  // stop one document monopolizing the results
+});
+
+// mode: 'hybrid' (default) | 'semantic' | 'lexical'. Lexical skips the query
+// embedding — fastest, right for identifier-shaped queries (error codes, SKUs).
+await Policies.search('ERR-7741X', { mode: 'lexical' });
+
 // add a document (background — poll documents() for status)
-await Policies.add(buffer, { filename: 'policy.pdf', contentType: 'application/pdf' });
+await Policies.add(buffer, { filename: 'policy.pdf', contentType: 'application/pdf', metadata: { department: 'legal' } });
 const docs = await Policies.documents();
 
 // introspection — what is in the corpus, and how one document was split
@@ -1870,14 +1881,18 @@ const chunks = await Policies.chunks(docs[0].id);       // why a document does o
 
 `SearchHit`: `{ score, text, citation: { documentId, filename, pageNumber, chunkIndex, headingPath, boundingBox?, url } }`. `citation.url` is a stable on-domain link to the source document — drop it in an `<a href>`. `(documentId, chunkIndex)` is a chunk's stable identity — key an eval set on that, not on `text`.
 
-Also on each hit: `retrievalRank`/`retrievalScore` (position before reranking — compare with the final position to see what reranking did). Two opt-in search options for debugging: `explain: true` adds `explain.{dense,lexical,matchedVia}` so you can see which half of hybrid found a hit, and `expand: 1` adds `neighbors.{before,after}` for surrounding context. Neither changes the results or their order.
+Also on each hit: `retrievalRank`/`retrievalScore` (position before reranking — compare with the final position to see what reranking did). Three opt-in search options for debugging and display: `explain: true` adds `explain.{dense,lexical,matchedVia}` so you can see which half of hybrid found a hit; `expand: 1` adds `neighbors.{before,after}` for surrounding context; `highlight: true` adds `matches` — `{start, end}` offsets of query terms in `text`, for rendering highlights. None of them changes the results or their order.
+
+`filter` fields (all AND together, and only narrow): `metadata` (per-key equals, or any-of via an array), `filename`, `documentIds`, `pages: {min?, max?}`, `contains` (all words, any order), `phrase` (exact adjacent sequence). Document metadata is set at add time: scalars only, ≤16 keys, keys `[a-zA-Z0-9_-]`. Re-adding the same bytes with different metadata updates the tags in place — no re-processing.
 
 Search is deterministic for a fixed corpus and configuration — same query, same order — so regression checks are meaningful. `search()` also returns `latencyMs`.
 
-Supported: pdf, docx, pptx, xlsx, odt, rtf, epub, images, txt, md, json, csv, html.
+Other methods: `stats()`, `documents()`, `chunks(documentId, {vectors?})`, `remove(documentId)`, `ensure(name?)` (rarely needed — `add` and `search` create-on-reference), and `DataSource.contentHash(bytes)` to check for an existing document before adding.
+
+Supported: pdf, docx, pptx, xlsx, odt, rtf, epub, images, txt, md, json, csv, tsv, log, html.
 
 Adding the same bytes twice is free — documents are content-addressed, so a re-add is a no-op when this source has already processed those exact bytes under its current configuration. Only reconfiguring the source makes a re-add do work again, and that is an explicit owner-triggered migration, never a side effect of a deploy.
 
-`rerank` and `hybrid` are free to change and take effect on the next search. How documents are chunked and embedded is configured on the corpus itself with `mindstudio-prod datasources config` — not in code — and changing it on a populated source is rejected rather than silently rebuilding.
+Everything in `SearchOptions` — `mode`, `filter`, `rerank`, `hybrid`, `maxPerDocument`, `highlight` — is free to change and takes effect on the next search. How documents are chunked and embedded is configured on the corpus itself with `mindstudio-prod datasources config` — not in code — and changing it on a populated source is rejected rather than silently rebuilding.
 
 **A data source is live and shared.** Unlike database tables there is no dev copy and no per-release isolation: adding or removing a document changes what the deployed app retrieves, immediately. Scenarios never reset a data source, and re-ingesting a large corpus costs real money.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@mindstudio-ai/agent",
-  "version": "0.1.89",
+  "version": "0.1.91",
   "description": "TypeScript SDK for MindStudio direct step execution",
   "type": "module",
   "main": "./dist/index.js",
@@ -26,6 +26,7 @@
     "codegen": "tsx scripts/codegen.ts",
     "typecheck": "tsc --noEmit",
     "postinstall": "node dist/postinstall.js || true",
+    "bump-consumers": "node scripts/bump-consumers.mjs",
     "prepare": "npm run build",
     "prepublishOnly": "npm run build"
   },