@mindstudio-ai/remy 0.1.327 → 0.1.329

package/dist/prompt/compiled/sdk-actions.md

@@ -42,7 +42,7 @@ result.$billingCost;         // cost in credits (if applicable)
 |--------|-------------|-----------|------------|
 | `analyzeImage` | Vision model analysis | `prompt`, `imageUrl` | `analysis` |
 | `analyzeVideo` | Video analysis | `prompt`, `videoUrl` | `analysis` |
-| `transcribeAudio` | Audio to text | `audioUrl` | `transcription` |
+| `transcribeAudio` | Audio to text | `audioUrl` | `text`, `segments` |
 | `extractText` | Extract text from documents/images | `url` | `text` |
 | `detectPII` | Find personal data | `text` | `entities` |
 

package/dist/prompt/skills/dataSources.md

@@ -40,13 +40,15 @@ Created on first use, so searching a source the build hasn't populated returns n
 
 **Filtering** narrows a search before ranking, and every condition only narrows: `filter: { metadata: { department: 'legal', year: [2025, 2026], signedAt: { gte: 20250101 } }, filename, documentIds, pages: { min?, max? }, contains: 'all these words', phrase: 'exact adjacent sequence' }`. Metadata matches per key: scalar = equals, array = any-of, `{ gte?, lte? }` = numeric range — ranges are numeric only, so store dates as sortable integers at add time (YYYYMMDD or epoch seconds) to range on them. Metadata is tagged at add time (scalars only, ≤16 keys); re-adding the same bytes with different metadata updates the tags in place, free. Filters are the right tool for scoping retrieval (per-user, per-category, a date window); they are NOT a substitute for a `db` query over structured data.
 
+**Counting matches.** `Policies.count(filter)` returns `{ chunks }`: how many passages match a filter, exactly, over the whole corpus, in the same grammar as `search`'s filter. `count({ contains: query })` is the honest number to show beside a search's hits: "8 of 3,891 passages mention these words". It is not a relevance count (none exists; similarity is a continuous score over every chunk) and it is not the set `search` returns (semantic hits need not contain the words, lexical hits may contain only some), so never label it "relevant results" or "matches for your query". Chunks, not documents. Counts share the 300/minute search limit; a count beside every search spends two. Same `index_warming` / `index_building` handling as search.
+
 **Modes**: `mode: 'hybrid'` (default) fuses semantic and keyword retrieval; `'semantic'` is the embedding alone; `'lexical'` is keyword-only with **no query embedding** — cheapest and fastest, right when the query is an identifier (an error code, a SKU, a name) rather than a meaning. `maxPerDocument: 2` stops one document monopolizing the results when the answer should draw on several. `highlight: true` adds `matches` (`{start, end}` offsets into `text`) for rendering highlighted excerpts.
 
 Search is deterministic for a fixed corpus and configuration, so eval sets and regression checks are meaningful — key them on `(documentId, chunkIndex)` rather than on chunk text.
 
 **Debugging retrieval.** Two opt-in options, neither of which changes the results or their order: `explain: true` adds `explain.{dense, lexical, matchedVia}` (which half of hybrid found each hit; costs two extra round trips), and `expand: 1` adds `neighbors.{before, after}` for surrounding context. When a document never comes back at all, `Policies.stats()` reports the config actually in effect and `Policies.chunks(documentId)` shows exactly how it was split.
 
-**A cold index (shared capacity).** By default a data source lives on shared retrieval capacity: its vectors sit in their own isolated partition of a pool many apps share, and the pool keeps only a working set resident. A source nobody has searched for a while is unloaded to make room and reloaded from durable storage on the next search. A small corpus reloads inside that search and nobody notices; a large one (hundreds of thousands of chunks) reloads in the background for a minute or two, and `search()` throws `index_warming` (HTTP 503) until it lands. That means *loading*, never *empty*: catch it, tell the user the knowledge base is warming up, and retry shortly. Before a demo, `remy-admin datasources hydrate --source <slug> --wait` reloads it ahead of time. The way out of the cycle is dedicated capacity (below): a source on its own provisioned retrieval is never unloaded and never warms. It has one pause of its own instead: while a bulk load runs, the index is not built behind every write, it is built once when the load finishes, and until then `search()` throws `index_building` (HTTP 503) with "N of M vectors indexed" in the message. Same handling as `index_warming`: the knowledge base is being built, never empty. `remy-admin datasources list` shows the index as `deferred`, `building` or `ready`.
+**A cold index (shared capacity).** By default a data source lives on shared retrieval capacity: its vectors sit in their own isolated partition of a pool many apps share, and the pool keeps only a working set resident. A source nobody has searched for a while is unloaded to make room and reloaded from durable storage on the next search. A small corpus reloads inside that search and nobody notices; a large one (hundreds of thousands of chunks) reloads in the background for a minute or two, and `search()` throws `index_warming` (HTTP 503) until it lands. That means *loading*, never *empty*: catch it, tell the user the knowledge base is warming up, and retry shortly. Before a demo, `remy-admin datasources hydrate --source <slug> --wait` reloads it ahead of time. The way out of the cycle is dedicated capacity (below): a source on its own provisioned retrieval is never unloaded. It has one pause of its own instead: after a large load the index catches up behind the writes, and until it has, `search()` throws `index_building` (HTTP 503) with "N of M vectors indexed" and the time left in the message. Same handling as `index_warming`: the knowledge base is being built, never empty. `remy-admin datasources list` shows the index as `building` or `ready`, with its rate and time left while it builds. A resource that resumes from hibernation, or whose instance the platform replaced, rebuilds every source placed on it from durable storage in the background; searches on those sources answer `index_warming` with the rebuild's progress until it lands.
 
 **Configuration is not declared in code** — chunking and embedding settings live on the corpus and are set with the CLI, so code and reality can't drift.
 
@@ -206,7 +208,9 @@ remy-admin datasources remap --source archive --wait
 
 **A mapper is a pure transform: one object in, documents out, nothing else.** `remap` and `jobs replay` run it again over the raw copies, and a frame runs in the context of the release that compiled it, so anything a mapper writes on the side is written twice and possibly into the wrong data plane. The per-document facts an app needs later belong in `metadata`; an app that wants its own view of a big corpus (a timeline, counts by year, a table of ids) builds it after ingest by walking `Source.allDocuments()` in a background task, and keeps it current from what each sync adds.
 
-A mapper runs on the platform, so the platform has to build it. Any push builds it, and a branch push is a private preview build, which is all a mapper needs. `map deploy` then makes that build's mapper the source's active one: jobs, syncs and `add()` run it from then on, whether or not the app has ever been published. Publishing to main activates the mapper main declares as well, so merge the branch before you publish. `jobs start` refuses with `mapper_not_deployed` while the dev session declares a mapper that is not yet active, because the job would otherwise load the raw records as documents.
+On a job, mapping is its own stage. The mapper turns each object into documents; the platform then ingests those documents in parallel batches of fifty across its workers, whatever one object became. So the size of an object does not set the pace, and a bundle of a thousand records is fine; only the number of objects sets how wide the mapping stage itself runs (three huge files map on three workers, the plan says so as a warning). `jobs status` reads "mapping N of M objects" until that stage is through, then counts documents.
+
+A mapper runs on the platform, so the platform has to build it. Any push builds it, and a branch push is a private preview build, which is all a mapper needs. `map deploy` then makes that build's mapper the source's active one: jobs, syncs and `add()` run it from then on, whether or not the app has ever been published. Publishing activates the mapper the live release declares — which is the one you deployed, since publishing fast-forwards the default branch to your branch. So there is nothing extra to do at publish time, and nothing to merge by hand: publishing is the merge (see the publishing skill). `jobs start` refuses with `mapper_not_deployed` while the dev session declares a mapper that is not yet active, because the job would otherwise load the raw records as documents.
 
 `map test --dev` needs the dev session running (`npx mindstudio dev`); it runs the mapper from local source through the tunnel and prints every outcome with markdown previews. The plan of a mapped job records the mapper's outcome mix on its sample; a run whose skip share climbs past twice that pauses with `pauseReason: 'skips'` for a look at the quarantine. `remap` reads the platform's own raw copies — no origin traffic — skips unchanged markdown by hash, and supersedes changed documents, so a metadata tweak on a million-document source costs frames and little else. `externalId` is the identity everything replaces by; choose it deliberately (the record's stable id, never the key of a file that gets rewritten in place).
 
@@ -214,6 +218,10 @@ A mapper runs on the platform, so the platform has to build it. Any push builds
 
 The shared pool holds a source up to a per-source cap of chunks. A corpus beyond it — a plan that answers `plan_requires_dedicated` — runs on dedicated retrieval capacity the workspace provisions and pays for hourly: `remy-admin infra list` shows the offering catalog with prices and any resources the app has, `infra provision --offering <id> --name <n> --wait` creates one, and `datasources move --source archive --to <resource-id> --wait` puts the source on it with its data intact (`create --placement <resource-id>` starts a new source there). Provisioning bills the workspace; **never provision without the user's explicit confirmation**, and show them the offering's price first. `infra --help` covers hibernate, resume, resize and destroy.
 
+A resource's capacity is bytes, not a chunk count: the catalog quotes each size as chunks at each embedding dimension on offer (a 2560-dimension chunk takes two and a half times a 1024-dimension one), and a plan or move that does not fit says how much room it would need against what the size holds. `infra get <id>` shows what the placed sources take against that, measured from the instance itself once it is up.
+
+Nothing long-running here is silent. Every `--wait` (`datasources move`, `hydrate`, `jobs approve`, `infra provision|resume|resize`) prints the phase, what is done of what, the measured rate and the time left as it goes, and keeps waiting as long as progress advances; the dashboard shows the same numbers. `infra get <id>` carries a resource's progress and `infra logs <id> --follow` streams the platform's narration of what it is doing. Hibernating releases the instance and stops billing; resuming brings a new one up and rebuilds every placed source's index from durable storage in the background, with its progress on the resource and `index_warming` from searches until it lands. Nothing is re-embedded.
+
 ## Answering from results
 
 Retrieve → join passages as context → have a model answer *from that context* → render citations. Never paste raw chunks at the user; they're fragments. For agentic flows, give the model `search` as a tool so it can query repeatedly and refine, rather than retrieving once up front.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@mindstudio-ai/remy",
-  "version": "0.1.327",
+  "version": "0.1.329",
   "description": "Remy coding agent",
   "repository": {
     "type": "git",