@mindstudio-ai/remy 0.1.269 → 0.1.270

package/README.md

@@ -43,13 +43,13 @@ Remy saves conversation history to `.remy-session.json` in the working directory
 
 ## Tools
 
-Tool availability depends on the project's onboarding state, sent by the sandbox on each message.
+The full tool set is always available — it is deliberately invariant (no onboarding-state gating) so the provider's tools-tier cache prefix stays identical across turns and sessions. The sections below are thematic groupings only.
 
-### Common Tools (all onboarding states)
+### Common Tools
 
 | Tool | Description |
 |------|-------------|
-| `setProjectOnboardingState` | Advance the onboarding flow (intake → initialSpecReview → initialCodegen → onboardingFinished) |
+| `setProjectOnboardingState` | Advance the onboarding flow (intake → building → buildComplete → onboardingFinished) |
 | `setProjectName` | Set the project name |
 | `promptUser` | Ask the user structured questions (form or inline display) |
 | `confirmDestructiveAction` | Confirm a destructive or irreversible action with the user |
@@ -74,8 +74,6 @@ Available in all onboarding states. Used for authoring and editing MSFM specs in
 
 ### Code Tools
 
-Available from `initialCodegen` onward.
-
 | Tool | Description |
 |------|-------------|
 | `readFile` | Read a file with line numbers |
@@ -100,9 +98,7 @@ Available when `--lsp-url` is passed.
 | `lspDiagnostics` | Type errors and warnings for a file, with suggested quick fixes |
 | `restartProcess` | Restart a managed sandbox process (e.g., dev server after npm install) |
 
-### Post-Onboarding Tools
-
-Available only when `onboardingState` is `onboardingFinished`.
+### Development & Publishing Tools
 
 | Tool | Description |
 |------|-------------|
@@ -174,11 +170,7 @@ src/
   logger.ts              Structured logging
 
   prompt/
-    index.ts             System prompt builder (onboarding-state-aware)
-    actions/             Built-in prompts for runCommand actions
-      sync.md
-      publish.md
-      buildFromInitialSpec.md
+    index.ts             System prompt builder (static prefix + small dynamic tail)
     static/              Behavioral instruction fragments
       identity.md
       intake.md
@@ -187,7 +179,7 @@ src/
       instructions.md
       team.md
       lsp.md
-      projectContext.ts  Reads manifest, spec metadata, file listing at runtime
+      projectContext.ts  Reads project root, app identity, plan status at runtime
     compiled/            Platform docs distilled for agent consumption
     sources/             Prompt source material (hand-maintained)
 
@@ -267,7 +259,7 @@ The headless IPC protocol uses request correlation and a unified response patter
 - Messages sent while a turn is running are queued. When the turn ends, all contiguous queued user messages and background results are delivered together as **one merged turn**: the first queued message's `requestId` becomes the turn's primary id (stamped on `turn_started` and all streaming events), each absorbed message echoes its own `user_message` with its original `requestId` and `queued: true`, and at turn end the primary `completed` is emitted first, followed immediately by one `completed {…same outcome, absorbed: true}` per other absorbed `requestId`. Automated-action (`@@automated::…@@`) messages and chain steps never merge — they always run one turn each.
 - A queued **user** message can be promoted to ASAP delivery via `setQueuedDelivery`. ASAP items are pulled into the **running** turn at its next tool boundary (injected as plain user messages — no abort, no restart), echo `user_message` with `queued: true` and their own `requestId`, and get a `{…, absorbed: true}` completed with the turn's outcome at turn end. Promotion deliberately jumps ahead of anything else in the queue, including chain steps. If the turn ends before injection, the tag is ignored and the item drains in normal FIFO order.
 - Background tool completions have three delivery classes (per-tool `backgroundNotify` on the tool definition). `wake` (default): the result is queued as a `background_results` message and may start a turn when the agent is idle. `passive` (e.g. `specSync`): the result never enters the queue and never wakes the agent — it parks in a persisted holding pen and rides the next real turn as a hidden `background_results` entry (so it never appears in `queuedMessages`, never affects queue-derived busy state, and never triggers resume-on-restart). `silent` (e.g. `compactConversation`): the tool block is updated for the UI and the model is never told — its outcome reaches the model by another mechanism. All classes emit `tool_background_complete` immediately. `specSync` also takes a `refreshBuildOverview` flag (set post-deploy / post-milestone, prompted via the publish flow): the run gains the `writeBuildOverview` tool and re-authors the Build Overview from the freshly-reconciled spec — the design-expert render runs foreground within specSync's already-detached run, never nested-background.
-- Compaction lifecycle rides `compaction_started {blocking}` / `compaction_complete {error?}` system events. Every compaction also renders as a normal `compactConversation` tool call: the model-invoked tool has its own block, and user (`/compact`) or gate-initiated compactions get a **synthesized UI-only tool block** in history (standard `tool_start {background: true}` → `tool_background_complete` lifecycle; the summary lands in the block's `backgroundResult`, and the block is excluded from every API payload — the model receives the summary via the checkpoint prefix instead). Every turn passes a **compaction gate**: it waits for any in-flight compaction and applies the finished checkpoint before running, so no turn ever bills at the uncompacted context. Messages received while a compaction is in flight are queued exactly like mid-turn messages (they appear in `queuedMessages` and drain — with merged-turn semantics — when the compaction completes, on success and failure alike). A `compact` command received **mid-turn** queues too: it's acked immediately with `completed {success: true, queued: true}` (duplicate clicks coalesce onto the queued item), appears in `queuedMessages` as a removable `@@automated::compact@@` user item, and runs as its own drain step when the turn ends — everything queued behind it runs against the compacted history.
+- Compaction lifecycle rides `compaction_started {blocking}` / `compaction_complete {error?}` system events. Every compaction also renders as a normal `compactConversation` tool call. The **model-invoked** tool is a real background tool (its block completes via `tool_background_complete`; the summary lands in `backgroundResult`). User (`/compact`) and gate-initiated compactions instead get a **synthesized UI-only foreground block** in history — the user is actively waiting on these, so they follow the normal foreground lifecycle: `tool_start` with no result, then a (late) `tool_done {result, isError}` carrying the summary or the error when the compaction finishes. Either way the block is excluded from every API payload — the model receives the summary via the checkpoint prefix instead. Every turn passes a **compaction gate**: it waits for any in-flight compaction and applies the finished checkpoint before running, so no turn ever bills at the uncompacted context. Messages received while a compaction is in flight are queued exactly like mid-turn messages (they appear in `queuedMessages` and drain — with merged-turn semantics — when the compaction completes, on success and failure alike). A `compact` command received **mid-turn** queues too: it's acked immediately with `completed {success: true, queued: true}` (duplicate clicks coalesce onto the queued item), appears in `queuedMessages` as a removable `@@automated::compact@@` user item, and runs as its own drain step when the turn ends — everything queued behind it runs against the compacted history.
 - The caller distinguishes command responses from system events with a single check: `if (msg.requestId)`
 
 This enables a simple promise-based RPC layer: send a command with a unique ID, store a pending promise keyed by that ID, resolve it when you see `completed` with the matching ID.
@@ -287,8 +279,7 @@ Send a user message to the agent.
 Fields:
 - `requestId` — caller-provided correlation ID (echoed on all response events)
 - `text` — the user message (required unless `runCommand` is set)
-- `onboardingState` — controls tool availability and prompt context. One of: `intake`, `initialSpecAuthoring`, `initialCodegen`, `onboardingFinished` (default: `onboardingFinished`)
-- `viewContext` — `{ mode, openFiles?, activeFile? }` for prompt context
+- `onboardingState` — the project's onboarding phase, reflected in the system prompt's dynamic tail and plan-status behavior. One of: `intake`, `building`, `buildComplete`, `onboardingFinished` (default: `onboardingFinished`)
 - `attachments` — array of `{ url, extractedTextUrl? }` for file attachments
 - `runCommand` — triggers a built-in action prompt (`"sync"`, `"publish"`, `"buildFromInitialSpec"`)
 
@@ -379,7 +370,8 @@ All command responses include the `requestId` from the originating command.
 | `user_message` | `text`, `attachments?`, `queued?`, `hidden?` | Echo of a user message entering the turn. Queue-delivered messages (including ASAP items injected mid-turn) carry `queued: true` and their own original `requestId` (a merged turn emits one per absorbed message); idle sends echo with the turn's requestId and no `queued` flag. `hidden: true` marks internal entries (e.g. passive background results) that should not render. |
 | `tool_start` | `id`, `name`, `input`, `partial?`, `parentToolId?` | Tool execution started. `partial: true` means more `tool_start` events will follow for this id (progressive input streaming). |
 | `tool_input_delta` | `id`, `name`, `result`, `parentToolId?` | Progressive tool content (streaming tools only) |
-| `tool_done` | `id`, `name`, `result`, `isError`, `parentToolId?` | Tool execution completed |
+| `tool_done` | `id`, `name`, `result`, `isError`, `parentToolId?` | Tool execution completed. For a background tool this carries the synchronous ack; for a synthesized user/gate compaction block it arrives late, when the compaction finishes, carrying the summary (or error). |
+| `tool_background_complete` | `id`, `name`, `result`, `parentToolId?` | A background tool's detached work finished; `result` belongs in the block's `backgroundResult`. Emitted for all `backgroundNotify` classes. |
 | `status` | `message` | Contextual status label (e.g., "Writing files...") |
 | `error` | `error` | Error message (may precede `completed`) |
 | `history` | `messages` | Response to `get_history` |

package/dist/headless.js

@@ -628,7 +628,7 @@ function loadProjectRoot() {
 ## Project Root
 \`${PROJECT_ROOT}\`
 
-File paths are relative to this directory. Every tool operates here and bash commands run it it.`;
+File paths are relative to this directory. Every tool operates here and bash commands run in it.`;
 }
 function loadAppIdentity() {
   try {
@@ -797,21 +797,9 @@ function buildSystemPrompt(onboardingState) {
   {{compiled/design.md}}
   </design>
 
-  <app_files>
-  {{compiled/files.md}}
-  </app_files>
-
   <interfaces>
   {{compiled/interfaces.md}}
   </interfaces>
-
-  <scenarios>
-  {{compiled/scenarios.md}}
-  </scenarios>
-
-  <secrets>
-  {{compiled/secrets.md}}
-  </secrets>
 </platform_docs>
 
 ${loadSkillsCatalog()}
@@ -824,12 +812,12 @@ ${loadSkillsCatalog()}
   {{compiled/msfm.md}}
 </mindstudio_flavored_markdown_spec_docs>
 
-<intake_mode_instructions>
+${onboardingState === "intake" ? `<intake_mode_instructions>
   {{static/intake.md}}
-</intake_mode_instructions>
+</intake_mode_instructions>` : ""}
 
 <spec_authoring_instructions>
-  {{static/authoring.md}}
+  ${onboardingState !== void 0 && onboardingState !== "onboardingFinished" ? "{{static/authoring.md}}" : "{{static/spec-maintenance.md}}"}
 </spec_authoring_instructions>
 
 <team>
@@ -851,7 +839,7 @@ Tool results generally persist in the conversation until a compaction summarizes
 </conversation_summaries>
 
 <project_onboarding>
-New projects progress through three onboarding states. The user might skip this entirely and jump straight into working on the existing scaffold (which defaults to onboardingFinished), but ideally new projects move through each phase:
+New projects progress through four onboarding states. The user might skip this entirely and jump straight into working on the existing scaffold (which defaults to onboardingFinished), but ideally new projects move through each phase:
 
 - **intake**: Gathering requirements. The project has scaffold code (a "hello world" starter) but it's not the user's app yet. Focus on understanding what they want to build, not on the existing code. Intake ends with a plan proposal via writePlan.
 - **building**: The user approved the initial plan. The agent is writing the spec and building the app. This can take a while and involves heavy tool use (spec authoring, design expert consultation, code generation, verification, polishing).
@@ -1378,8 +1366,33 @@ var presentPublishPlanTool = {
   }
 };
 
+// src/atomicWrite.ts
+import fs10 from "fs";
+import fsp from "fs/promises";
+var writeFileAtomicSync = (file, data) => {
+  const tmp = `${file}.${process.pid}.tmp`;
+  const fd2 = fs10.openSync(tmp, "w");
+  try {
+    fs10.writeSync(fd2, data);
+    fs10.fsyncSync(fd2);
+  } finally {
+    fs10.closeSync(fd2);
+  }
+  fs10.renameSync(tmp, file);
+};
+var writeFileAtomic = async (file, data) => {
+  const tmp = `${file}.${process.pid}.tmp`;
+  const handle = await fsp.open(tmp, "w");
+  try {
+    await handle.writeFile(data);
+    await handle.sync();
+  } finally {
+    await handle.close();
+  }
+  await fsp.rename(tmp, file);
+};
+
 // src/tools/spec/writePlan.ts
-import fs10 from "fs/promises";
 var PLAN_FILE = ".remy-plan.md";
 var writePlanTool = {
   definition: {
@@ -1403,7 +1416,7 @@ status: pending
 ---
 
 ${content}`;
-    await fs10.writeFile(PLAN_FILE, file, "utf-8");
+    await writeFileAtomic(PLAN_FILE, file);
     return "Plan written to .remy-plan.md. Waiting for user approval.";
   }
 };
@@ -1439,13 +1452,13 @@ var updatePlanStatusTool = {
       return "No plan file found.";
     }
     if (status === "rejected") {
-      await fs11.unlink(PLAN_FILE2);
+      await fs11.unlink(PLAN_FILE2).catch(() => {
+      });
       return "Plan rejected and removed.";
     }
-    await fs11.writeFile(
+    await writeFileAtomic(
       PLAN_FILE2,
-      content.replace(/^status:\s*\w+/m, `status: ${status}`),
-      "utf-8"
+      content.replace(/^status:\s*\w+/m, `status: ${status}`)
     );
     return "Plan approved. Proceeding with implementation.";
   }
@@ -2975,7 +2988,8 @@ var CONTENT_TYPES = {
   ".jpg": "image/jpeg",
   ".jpeg": "image/jpeg",
   ".gif": "image/gif",
-  ".webp": "image/webp"
+  ".webp": "image/webp",
+  ".svg": "image/svg+xml"
 };
 var hosted = /* @__PURE__ */ new Map();
 function isFetchableUrl(ref) {
@@ -3582,7 +3596,8 @@ async function runSubAgent(config) {
     background,
     acquireLock,
     onBackgroundComplete,
-    captureArtifacts
+    captureArtifacts,
+    cachePolicy = "run"
   } = config;
   const artifacts = {};
   const bgAbort = background ? new AbortController() : null;
@@ -3677,6 +3692,7 @@ ${partial}` : "[INTERRUPTED] Agent was interrupted before producing output.",
               system: fullSystem,
               messages: cleanMessagesForApi(messages),
               tools: tools2,
+              cachePolicy,
               signal
             },
             {
@@ -5243,7 +5259,7 @@ function load() {
 }
 function save(indices) {
   try {
-    fs18.writeFileSync(SAMPLE_FILE, JSON.stringify(indices));
+    writeFileAtomicSync(SAMPLE_FILE, JSON.stringify(indices));
   } catch {
   }
 }
@@ -5494,6 +5510,8 @@ async function runDesignExpert(opts, context) {
     }),
     task: opts.task,
     history: history.length > 0 ? history : void 0,
+    // History-carrying thread re-sent across parent turns — 1h-TTL profile.
+    cachePolicy: "conversation",
     tools: opts.render ? DESIGN_EXPERT_RENDER_TOOLS : DESIGN_EXPERT_TOOLS,
     externalTools: /* @__PURE__ */ new Set(),
     executeTool: (name, input, toolCallId, onLog, sams) => {
@@ -5761,6 +5779,8 @@ var productVisionTool = {
       system: getProductVisionPrompt(),
       task: input.task,
       history: history.length > 0 ? history : void 0,
+      // History-carrying thread re-sent across parent turns — 1h-TTL profile.
+      cachePolicy: "conversation",
       tools: VISION_TOOLS,
       externalTools: /* @__PURE__ */ new Set(),
       executeTool: (name, input2, toolCallId, _onLog, sams) => {
@@ -6228,7 +6248,7 @@ var SUBAGENT_TOOL_NAMES = /* @__PURE__ */ new Set([
   "runAutomatedBrowserTest",
   "askMindStudioSdk"
 ]);
-function getToolDefinitions(_onboardingState) {
+function getToolDefinitions() {
   return ALL_TOOLS.map((t) => t.definition);
 }
 function getToolByName(name) {
@@ -6612,7 +6632,11 @@ Write the summary of the conversation above, following your instructions.`;
     messages: [{ role: "user", content: userContent }],
     // Always empty. With a toolset available the model picks `tool_use` over
     // producing a summary, leaving summaryText empty.
-    tools: []
+    tools: [],
+    // Each summary call carries a unique 100-200KB chunk that is never
+    // re-read (parallel siblings can't read each other's in-flight writes
+    // either) — a cache write here is pure waste.
+    cachePolicy: "oneshot"
   })) {
     if (event.type === "text") {
       summaryText += event.text;
@@ -6648,11 +6672,6 @@ import path11 from "path";
 var log10 = createLogger("session");
 var SESSION_FILE = ".remy-session.json";
 var ARCHIVE_DIR = ".logs/sessions";
-function writeFileAtomicSync(file, data) {
-  const tmp = `${file}.tmp`;
-  fs21.writeFileSync(tmp, data, "utf-8");
-  fs21.renameSync(tmp, file);
-}
 var ROTATE_THRESHOLD_BYTES = 32 * 1024 * 1024;
 var RETAIN_TAIL_BYTES = 16 * 1024 * 1024;
 var ARCHIVE_RETENTION_BYTES = 64 * 1024 * 1024;
@@ -6681,6 +6700,12 @@ function loadSession(state) {
       return true;
     }
   } catch {
+    try {
+      const quarantine = `${SESSION_FILE}.corrupt-${Date.now()}`;
+      fs21.renameSync(SESSION_FILE, quarantine);
+      log10.warn(`Session file unreadable \u2014 quarantined to ${quarantine}`);
+    } catch {
+    }
   }
   return false;
 }
@@ -7210,7 +7235,10 @@ async function extractBrand(apiConfig, model) {
       subAgentId: "brandExtractor",
       system: EXTRACT_PROMPT,
       messages: [{ role: "user", content: corpus }],
-      tools: []
+      tools: [],
+      // One call per extraction over a corpus that changes with every spec
+      // edit — never re-read, so a cache write is pure waste.
+      cachePolicy: "oneshot"
     })) {
       if (event.type === "text") {
         responseText += event.text;
@@ -7392,11 +7420,9 @@ function pickFont(raw) {
   return out;
 }
 function persistBrand(brand, inputHash) {
-  const tmp = `${BRAND_FILE}.tmp`;
-  fs22.writeFileSync(tmp, JSON.stringify(brand, null, 2), "utf-8");
-  fs22.renameSync(tmp, BRAND_FILE);
+  writeFileAtomicSync(BRAND_FILE, JSON.stringify(brand, null, 2));
   const cache = { inputHash, generatedAt: Date.now() };
-  fs22.writeFileSync(CACHE_FILE, JSON.stringify(cache, null, 2), "utf-8");
+  writeFileAtomicSync(CACHE_FILE, JSON.stringify(cache, null, 2));
 }
 function readCache() {
   try {
@@ -7737,7 +7763,7 @@ async function runTurn(params) {
     toolRegistry,
     onBackgroundComplete
   } = params;
-  const tools2 = getToolDefinitions(onboardingState);
+  const tools2 = getToolDefinitions();
   const buildModelOverride = buildModel ? filterModelPicks({ parent: buildModel }).parent : void 0;
   const baseline = resolveModel("parent", state.models, model);
   const parentModel = buildModelOverride ?? baseline;
@@ -7945,6 +7971,9 @@ async function runTurn(params) {
             system,
             messages: cleanMessagesForApi(state.messages),
             tools: tools2,
+            // The parent is one long-lived conversation re-read across turns
+            // with multi-minute gaps — the 1h-TTL profile.
+            cachePolicy: "conversation",
             signal
           },
           {
@@ -8481,16 +8510,15 @@ ${lines.join("\n")}`;
 }
 
 // src/headless/planFile.ts
-import { readFileSync, writeFileSync, unlinkSync } from "fs";
+import { readFileSync, unlinkSync } from "fs";
 var PLAN_FILE3 = ".remy-plan.md";
 function applyPlanFileSideEffect(rawText) {
   if (hasSentinel(rawText, "approvePlan") || hasSentinel(rawText, "approveInitialPlan")) {
     try {
       const plan = readFileSync(PLAN_FILE3, "utf-8");
-      writeFileSync(
+      writeFileAtomicSync(
         PLAN_FILE3,
-        plan.replace(/^status:\s*pending/m, "status: approved"),
-        "utf-8"
+        plan.replace(/^status:\s*pending/m, "status: approved")
       );
     } catch {
     }
@@ -8503,7 +8531,7 @@ function applyPlanFileSideEffect(rawText) {
 }
 
 // src/headless/stats.ts
-import { readFileSync as readFileSync2, writeFileSync as writeFileSync2 } from "fs";
+import { readFileSync as readFileSync2 } from "fs";
 var STATS_FILE = ".remy-stats.json";
 function createSessionStats() {
   return {
@@ -8540,7 +8568,7 @@ function loadPassiveResults() {
 }
 function writeStats(stats, queue, passiveResults) {
   try {
-    writeFileSync2(
+    writeFileAtomicSync(
       STATS_FILE,
       JSON.stringify({
         ...stats,
@@ -8769,9 +8797,7 @@ var HeadlessSession = class {
                 name: "compactConversation",
                 input: {},
                 startedAt: Date.now(),
-                background: true,
-                uiOnly: true,
-                result: "Compaction started in the background."
+                uiOnly: true
               }
             ]
           });
@@ -8780,15 +8806,7 @@ var HeadlessSession = class {
             type: "tool_start",
             id,
             name: "compactConversation",
-            input: {},
-            background: true
-          });
-          this.onEvent({
-            type: "tool_done",
-            id,
-            name: "compactConversation",
-            result: "Compaction started in the background.",
-            isError: false
+            input: {}
           });
         }
       } else {
@@ -8797,11 +8815,31 @@ var HeadlessSession = class {
         if (this.syntheticCompactionId) {
           const id = this.syntheticCompactionId;
           this.syntheticCompactionId = null;
-          this.onBackgroundComplete(
+          const result = event.error ? `Error: ${event.error}` : formatSummariesResult(event.summaries ?? []);
+          const isError = !!event.error;
+          for (let i = this.state.messages.length - 1; i >= 0; i--) {
+            const msg = this.state.messages[i];
+            if (msg.role !== "assistant" || !Array.isArray(msg.content)) {
+              continue;
+            }
+            const block = msg.content.find(
+              (b) => b.type === "tool" && b.id === id
+            );
+            if (block && block.type === "tool") {
+              block.result = result;
+              block.isError = isError;
+              block.completedAt = Date.now();
+              saveSession(this.state);
+              break;
+            }
+          }
+          this.onEvent({
+            type: "tool_done",
             id,
-            "compactConversation",
-            event.error ? `Error: ${event.error}` : formatSummariesResult(event.summaries ?? [])
-          );
+            name: "compactConversation",
+            result,
+            isError
+          });
         }
         this.sessionStats.compactionInProgress = false;
         if (!event.error) {