@mindstudio-ai/remy 0.1.335 → 0.1.337

package/dist/prompt/compiled/sdk-actions.md

@@ -22,6 +22,20 @@ result.content;              // step-specific output
 result.$billingCost;         // cost in credits (if applicable)
 ```
 
+## Where files an action produces end up
+
+Any action that returns a file URL writes to the shared **public** MindStudio CDN by default: unlisted, but readable by anyone holding the URL, and it never expires. Pass a file store as `store` in the options object to write into the app's own storage instead — a private store means the URL only loads for an authorized app session.
+
+```typescript
+const Reports = files.defineStore('reports', { access: 'private' });
+
+const { imageUrl } = await mindstudio.generateImage({ prompt }, { store: Reports });
+```
+
+**Use a store whenever the file belongs to a specific user or is confidential** (a client document, a participant recording, anything the app wouldn't publish). The public default is for marketing images and other genuinely world-readable assets. See the `files` skill.
+
+Inputs are safe either way: a private file or a `shareUrl` handed to an action is read where it sits, and any intermediate the platform needs (transcoding a video, rasterizing a PDF) is staged privately and deleted when the call finishes.
+
 ## Available Actions
 
 ### AI Generation

package/dist/prompt/skills/dataSources.md

@@ -20,6 +20,7 @@ Don't use one when: the data is structured (`db`), you only need to store files
 - **Ingest is async.** `add()` returns once queued; poll `documents()`, or use `--wait` from the CLI.
 - **Reprocessing costs real money**, so changing how a corpus is built is always explicit.
 - **Limits apply**: 25 data sources per app, 10,000 chunks per document, 300 searches/minute, and 5,000 documents per source when documents are added one at a time (bulk jobs and connectors, below, are how a corpus grows past that). Well clear of normal use — but **source names must be fixed, not computed per user or per request**, since referencing one creates it. Partition inside a source with document metadata instead: tag at add time (`add(bytes, { filename, metadata: { userId } })`), narrow at search time (`search(q, { filter: { metadata: { userId } } })`).
+- **Whole documents, not excerpts.** Nothing needs trimming before it is added: a document becomes as many chunks as it needs, up to the 10,000 per document above (about 20M characters). Truncating to keep chunks manageable throws away retrievable text, and is never the fix for an ingest problem.
 - **Credentials never appear in code or in chat.** A bucket's keys are app secrets (`remy-admin secrets set NAME --prod <value>`, or the dashboard); everything else refers to them by NAME. If the user pastes a key into the conversation, set it as a secret for them to use moving forward.
 
 ## Defining and searching
@@ -46,7 +47,7 @@ Created on first use, so searching a source the build hasn't populated returns n
 
 Search is deterministic for a fixed corpus and configuration, so eval sets and regression checks are meaningful — key them on `(documentId, chunkIndex)` rather than on chunk text.
 
-**Debugging retrieval.** Two opt-in options, neither of which changes the results or their order: `explain: true` adds `explain.{dense, lexical, matchedVia}` (which half of hybrid found each hit; costs two extra round trips), and `expand: 1` adds `neighbors.{before, after}` for surrounding context. When a document never comes back at all, `Policies.stats()` reports the config actually in effect and `Policies.chunks(documentId)` shows exactly how it was split.
+**Debugging retrieval.** Two opt-in options, neither of which changes the results or their order: `explain: true` adds `explain.{dense, lexical, matchedVia}` (which half of hybrid found each hit; costs two extra round trips), and `expand: 1` adds `neighbors.{before, after}` for surrounding context. **When a document never comes back, or its chunk count looks wrong, read the split before theorising about the file.** `Policies.chunks(documentId)` shows exactly how that document was divided and `Policies.stats()` reports the config actually in effect; from the CLI, `datasources status` gives per-document state with its ingest error. One chunk for a long document, or a chunk much larger than the configured `--max-chars`, is a fact about the split, not about the file.
 
 **A cold index (shared capacity).** By default a data source lives on shared retrieval capacity: its vectors sit in their own isolated partition of a pool many apps share, and the pool keeps only a working set resident. A source nobody has searched for a while is unloaded to make room and reloaded from durable storage on the next search. A small corpus reloads inside that search and nobody notices; a large one (hundreds of thousands of chunks) reloads in the background for a minute or two, and `search()` throws `index_warming` (HTTP 503) until it lands. That means *loading*, never *empty*: catch it, tell the user the knowledge base is warming up, and retry shortly. Before a demo, `remy-admin datasources hydrate --source <slug> --wait` reloads it ahead of time. The way out of the cycle is dedicated capacity (below): a source on its own provisioned retrieval is never unloaded. It has one pause of its own instead: after a large load the index catches up behind the writes, and until it has, `search()` throws `index_building` (HTTP 503) with "N of M vectors indexed" and the time left in the message. Same handling as `index_warming`: the knowledge base is being built, never empty. `remy-admin datasources list` shows the index as `building` or `ready`, with its rate and time left while it builds. A resource that resumes from hibernation, or whose instance the platform replaced, rebuilds every source placed on it from durable storage in the background; searches on those sources answer `index_warming` with the rebuild's progress until it lands.
 
@@ -118,7 +119,7 @@ for await (const doc of Policies.allDocuments()) { /* ... */ }    // walk a corp
 await Policies.remove(documentId);
 ```
 
-Formats: pdf, docx, pptx, xlsx, odt, rtf, epub, images, txt, md, json, csv, tsv, log, html. When the file is not the document (a JSON record, a JSONL bundle of articles, a kill notice), the source needs a mapper — see below.
+Formats: pdf, docx, pptx, xlsx, odt, rtf, epub, images, txt, md, json, csv, tsv, log, html. **Add the source bytes, not your own rendering of them** — built-in extraction reads every one of these, HTML included, so fetching a page and converting it to markdown yourself swaps a tested extractor for an untested one and makes its quirks the corpus's quirks. When the file is not the document (a JSON record, a JSONL bundle of articles, a kill notice), the source needs a mapper — see below.
 
 Removing many documents at once: `Policies.removeWhere({ metadata: { year: 2019 } })` or `{ externalIdPrefix: 'archive/2019/' }` (the key a job or connector recorded) removes every match, vectors and bytes included, in pages of a thousand. From the CLI, `datasources rm --source policies --filter year=2019`. An empty filter is refused; deleting a whole source is `datasources delete`, never something app code does.
 

package/dist/prompt/skills/files.md

@@ -114,13 +114,15 @@ remy-admin files rm --store handoff --key <key> --private               # revoke
 
 ## Generated assets
 
-MindStudio SDK Actions that produce a file (`generateImage`, `generateVideo`, `generateSpeech`, `generatePdf`, `upscaleImage`, …) can optionally write straight into a store — pass the handle as `store` in the options object:
+Every MindStudio SDK Action that produces a file (`generateImage`, `generateVideo`, `generateSpeech`, `generatePdf`, `upscaleImage`, `trimMedia`, `convertPdfToImages`, `screenshotUrl`, `getGmailAttachments`, …) can write straight into a store — pass the handle as `store` in the options object:
 
 ```typescript
 const { imageUrl } = await mindstudio.generateImage({ prompt }, { store: Assets });
 ```
 
-If omitted, files are written to the default global, public MindStudio CDN.
+If omitted, the file goes to the shared public MindStudio CDN, where the URL is unlisted but readable by anyone who has it, forever. **For anything confidential, pass a private store.** That's the one thing to get right here — the default is fine for marketing images and wrong for client material.
+
+**Actions never publish what you give them.** An input that's already one of the app's own files — a private object, a `shareUrl` link, a `/_/files/...` path — is read where it sits, and anything the platform has to convert first (a non-mp4 video, the pages of a PDF, a document an extraction model must fetch) is staged privately and discarded when the call finishes. So handing `extractText` a private contract, or `trimMedia` a private `.mov`, leaves no public copy behind.
 
 ## When public vs private
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@mindstudio-ai/remy",
-  "version": "0.1.335",
+  "version": "0.1.337",
   "description": "Remy coding agent",
   "repository": {
     "type": "git",