@mindstudio-ai/agent 0.1.121 → 0.1.123

package/llms.txt

@@ -935,8 +935,14 @@ Generate an audio file from provided text using a speech model.
 #### transcribeAudio
 Convert an audio file to text using a transcription model.
 - The prompt field provides optional context to improve transcription accuracy (e.g. language, speaker names, domain).
+- Detail beyond the plain transcript is requested through `transcriptionModelOverride.config`, and support varies by model:
+- `includeSpeakers: 'yes'` — label each passage with a speaker. Supported by elevenlabs-scribe-v1, elevenlabs-scribe-v2, deepgram-nova-3 and grok-stt. Also switches `text` to a timestamped, speaker-labelled block format.
+- `includeWordTimings: 'yes'` — add per-word timings and confidence inside each segment. Supported by elevenlabs-scribe-v1, elevenlabs-scribe-v2, deepgram-nova-3 and whisper-1. Off by default because an hour of speech is roughly ten thousand words.
+- `detectLanguage: 'yes'` — detect the spoken language instead of assuming English (deepgram-nova-3 only; Scribe and the whisper models always report it).
+- The `segments` output carries timings, confidence and speakers as structured data, so there is no need to slice the audio up to recover them. It is populated for elevenlabs-scribe-v1/v2, deepgram-nova-3, whisper-1 and the DeepInfra-served whisper/voxtral/qwen models, and absent for models that return nothing but text (gpt-transcribe, gpt-4o-transcribe, grok-stt).
+- `confidence` is always 0..1 with higher meaning more confident, whichever model produced it.
 - Input: `{ audioUrl: string, prompt: string, transcriptionModelOverride?: { model: string, config?: object } }`
-- Output: `{ text: string }`
+- Output: `{ text: string, segments?: { start: number, end: number, text: string, confidence?: number, speaker?: string, language?: string, words?: { text: string, start?: number, end?: number, confidence?: number, speaker?: string }[] }[], language?: string, durationSeconds?: number }`
 
 #### trimMedia
 Trim an audio or video clip
@@ -1909,6 +1915,11 @@ for await (const doc of Policies.allDocuments()) { /* doc.id, doc.metadata, doc.
 // introspection — what is in the corpus, and how one document was split
 const { chunkCount, pipeline } = await Policies.stats();  // counts + the config actually in effect
 const chunks = await Policies.chunks(docs[0].id);       // why a document does or doesn't match
+
+// how many passages contain the words — exact, same filter grammar as search. Say
+// "N passages mention these words", never "N relevant results": no such total exists,
+// and search's hits are not this set (semantic hits need not contain the words).
+const { chunks: mentions } = await Policies.count({ contains: 'client secret rotation' });
 ```
 
 `SearchHit`: `{ score, text, citation: { documentId, filename, pageNumber, chunkIndex, headingPath, boundingBox?, url } }`. `citation.url` is a stable on-domain link to the source document for a signed-in user — drop it in an `<a href>`. A document is a cousin of a file and shares its share verbs: for a public app or another origin, `search(q, { shareCitations: 3600 })` returns every `citation.url` as an absolute signed link that needs no session, and `Policies.shareUrl(hit.citation, { expiresIn })` signs one document on demand (default 24h). `(documentId, chunkIndex)` is a chunk's stable identity — key an eval set on that, not on `text`.
@@ -1925,7 +1936,7 @@ Search is deterministic for a fixed corpus and configuration — same query, sam
 
 On dedicated capacity `search()` throws `index_building` (HTTP 503) instead, while a bulk load runs and while the index is built after one: the index is built once at the end of a load rather than behind every write, and the message carries "N of M vectors indexed". Handle it like `index_warming` — the knowledge base is being built, never empty; say so and retry later.
 
-Other methods: `stats()`, `documents()` (the first thousand, or `documents({ ids })` to poll what `add` returned), `documentsPage({ cursor?, limit? })` and `allDocuments()` (walk a corpus of any size, oldest first — how an app builds its own timeline or index of a big source after ingest, in a background task), `chunks(documentId, {vectors?})`, `remove(documentId)`, `removeWhere(selector)`, `ensure(name?)` (rarely needed — `add` and `search` create-on-reference), and `DataSource.contentHash(bytes)` to check for an existing document before adding.
+Other methods: `stats()`, `documents()` (the first thousand, or `documents({ ids })` to poll what `add` returned), `documentsPage({ cursor?, limit?, order? })` (one page; `order: "newest"` is what just arrived) and `allDocuments()` (walk a corpus of any size, oldest first — how an app builds its own timeline or index of a big source after ingest, in a background task), `chunks(documentId, {vectors?})`, `remove(documentId)`, `removeWhere(selector)`, `ensure(name?)` (rarely needed — `add` and `search` create-on-reference), and `DataSource.contentHash(bytes)` to check for an existing document before adding.
 
 `removeWhere({ metadata?, filename?, documentIds?, externalIdPrefix? })` removes every matching document (vectors, row and bytes) in pages of a thousand and returns `{ deleted, remaining }`. Chunk-level filter fields and an empty selector are refused; deleting a whole source is an owner operation (`remy-admin datasources delete`), not something app code does.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@mindstudio-ai/agent",
-  "version": "0.1.121",
+  "version": "0.1.123",
   "description": "TypeScript SDK for MindStudio direct step execution",
   "type": "module",
   "main": "./dist/index.js",