@garuda-agent/garuda 0.18.0 → 0.18.2

package/CHANGELOG.md

@@ -1,5 +1,161 @@
 # Changelog
 
+## 0.18.2 (2026-11-06)
+
+A fix release for repo evals (`garuda eval -s repo`, `garuda advise`). On macOS, 0.18.1 failed
+every check of a project whose tools live in `~/Library` (pnpm does), and Vite projects failed
+before a test ran. It also makes `garuda advise` faster and cheaper to repeat: runs in parallel,
+reuse of earlier runs, a cost estimate from real requests, and the command to go on after a stop
+at the cap. A provider outage no longer counts against a model.
+
+- Fix: repo-task evals (`-s repo`, `garuda advise`) on macOS failed every check in 0.18.1 when the
+  project's tools live in `~/Library`. pnpm runs the version that a project asks for from
+  `~/Library/pnpm`, and the 0.18.1 rules denied all of the home folder except hidden entries. The
+  check stopped with "EPERM: process.cwd failed", so both models passed 0 of 30 in the user's run.
+  `~/Library` is readable again in these runs; Mail, Messages, Safari, iCloud Drive, the other
+  personal parts and the keychains stay denied for every sandboxed command. Run 0.18.1 repo evals
+  again.
+- `advise` gives no advice when neither model passes a task: the check itself may fail.
+- Evals: a timeout while the model's API is silent is a new failure kind, `provider`. It applies
+  when a request was open at the limit, and the stream had broken and been retried or the API had
+  sent nothing for 2 minutes. Such a run is not scored, and it runs once more by itself; its
+  spending is kept. In the user's Sonnet run, 4 of 5 timeouts were such stalls, and they counted
+  against the model. A timeout also shows its steps now, not 0.
+- Fix: in repo-task evals, Vite could not write its compiled `vitest.config.ts` to
+  `node_modules/.vite-temp`, because the task's `node_modules` links to the project's and was
+  read-only. In the user's suite 10 of 30 tasks failed for both models before a test ran.
+  Commands may now write `.vite-temp`, `.vite` and `.cache` there, and nothing else.
+- `garuda eval --from-git` checks each kept commit a third time, as an eval runs it (a copy, in the
+  sandbox), and leaves out a commit whose tests fail there. Before, it checked only on the host,
+  and a task could pass the build but never an eval (2 of 30 in the user's suite: tests that start
+  their own sandbox). Build your suite again.
+- The Homebrew formula in the repository names 0.18.1 and its tarball hash.
+- `garuda advise` runs 4 tasks at a time (`--parallel <n>`); each result line starts with the
+  model and a count, as in `[haiku-5-5 12/30] ✓ fix-parser: pass`. The first real run, one at a
+  time, took 64 minutes for 30 tasks.
+- `advise` reuses an earlier run of a model when the model, the task hashes, `--repeat`, `--seed`
+  and the Garuda version and build commit are the same and every run is there. Runs of 0.18.0 and
+  before are not reused: they could read your checkout (the 0.18.1 fix).
+- The cost estimate comes from the mean cost of earlier runs of the model on the same tasks. With
+  none, it prices each request of the other model's runs (from their session files) at this
+  model's prices; else the token guess. Each line says which. (A first version scaled by the ratio
+  of list prices: 20 for Sonnet 5.5 over Haiku 5.5, but Haiku costs 5 times as much for a prompt
+  over 100 000 tokens. On a real run it estimated $19.16 where the requests give $12.62.)
+- The repriced estimate is scaled by what both models really cost on shared repo tasks before:
+  Sonnet 5.5 needs fewer steps and tokens than Haiku 5.5, and cost $6.30 where Haiku's requests at
+  its prices gave $12.62 (× 0.52). Without the factor, the user's 90-run estimate was $33.62; with
+  it, $17.37. Runs where no task passed, and built-in suites, are left out.
+- When the estimate or the first model's real cost passes `--max-cost`, `advise` prints the
+  command that goes on (`garuda advise --models a,b --repeat R --seed S --max-cost N`). The first
+  model's run is kept and reused.
+- `advise` stops when a run fails to start (for example a refused key), not at a missing report.
+
+## 0.18.1 (2026-11-06)
+
+A security release. In 0.18.0 and earlier, a command in the OS sandbox could read your personal
+folders (Documents, Downloads, Pictures and the Photos library, Mail and the like), and on macOS it
+could reach denied paths through `/System/Volumes/Data`. Update to 0.18.1. It also has the UX work
+of docs/lld/ux.md (receipts, a network question, `/rewind`, word-level diffs, the morning brief,
+`/export --html`, `garuda advise` and more) and eval failure kinds.
+
+- Security: sandboxed commands can no longer read your personal folders: Desktop, Documents,
+  Downloads, Movies, Music, Pictures (with the Photos library), Public, and Mail, Messages, Safari,
+  Calendars, Contacts, iCloud Drive and other cloud folders in `~/Library`. A project inside one of
+  them stays readable, the rest of that folder does not. On macOS each read and write rule also
+  covers the same path under `/System/Volumes/Data`, where a command could reach it before. Found
+  in a live run: a model's `find /` in an eval made macOS ask the user to let iTerm into Downloads
+  and Photos.
+- Evals: a repo task now runs in a copy of the project with no history, not a git worktree, and
+  its commands cannot read the project's checkout (except `node_modules` and venvs) or the rest of
+  the home folder except hidden entries. In a live run, a model read the user's checkout, which
+  holds the commit that the task asks for. Repo results from before this change may have read the
+  answer; run them again.
+- UX, patch 7 (docs/lld/ux.md U11, U8): `/export --html [file]` writes the session as one page:
+  your requests, the answers, each tool call that opens to its output (an edit opens to its
+  change), and the receipts. Every text is escaped; the page has no script and no link, and its
+  own policy forbids them. `garuda advise` runs the repo suite (`garuda eval --from-git`) on two
+  models (default Haiku 5.5 and Sonnet 5.5, 2 runs a task, at most $2), compares them as
+  `eval compare` does, and advises: the model that passes clearly more, or on a tie the cheaper
+  one. It asks first with a cost guess, and writes `advice.txt` and `report.html`.
+- UX, patch 6 (docs/lld/ux.md U7, U9): the morning brief. The first chat after jobs ran shows them
+  once: each job's verdict, its tests ("tests 120 → 126" when the runner's summary gives counts)
+  and its branch, and the total cost. `.garuda/jobs/brief.seen` keeps the time of the last brief.
+  The first-run tour: the first chat in a project with code names its stack ("TypeScript with
+  vitest (pnpm)") and offers three first tasks; typing 1, 2 or 3 runs one. It shows once per
+  project (`~/.garuda/ui.json` keeps a hash of the folder, not its path).
+- UX, patch 5 (docs/lld/ux.md U5): diffs mark the changed words of a changed line (inverse video),
+  in approvals, in hunk review and in `/diff`. From 140 columns a diff is side by side, with the
+  line numbers of both files. An approval that can review hunks says so in its first line:
+  "3 hunks · h reviews them one by one". Syntax colours are not in this patch (decision D10).
+  Only a real diff (with a hunk header) gets the marks and the two columns; other previews, such
+  as the project-settings question, keep their simple colours (the first live run showed that
+  question twice, side by side).
+- UX, patch 4 (docs/lld/ux.md U6): `/rewind` lists the turns that undo can reach (files, cost, age).
+  `/rewind <n>` goes back to the start of turn n in one step, with one question; `/redo` brings the
+  turns back one at a time. `/rewind fork <n>` starts a new session with the conversation up to
+  turn n and the files as they were then; the old session does not change. A new `fork` record
+  starts such a session.
+- UX, patch 3 (docs/lld/ux.md U4): in the chat, calls of one read-only tool in a row become one
+  line, for example `● Read 5 files  docs/architecture.md, SECURITY.md, +3`; a lone call keeps its
+  line and its result. Edits, commands and failed calls always keep a line of their own. Ctrl-O
+  still shows the last call's output. The model's plan (`todo_write`) is drawn live above the
+  input during the turn and printed once at the end, instead of after every update.
+- The chat's wordmark has a drop shadow (a 3D look), drawn twice as large from 130 columns, with
+  the card as wide as the wordmark; a tall screen gets some space above it. A screen reader gets no
+  wordmark (the card says the name).
+- The chat starts on a clear screen, with the banner centred, and you type in a visible chat box
+  with a hint ("Type a task · / for commands · @ for files"). The old screen goes into the
+  scrollback, not away; `"ui": { "clearOnStart": false }` in settings keeps it on screen. Garuda
+  stays in the terminal's normal screen, so its scrollback, search and copy still work. The box's
+  border dims while a turn runs. With `GARUDA_SCREEN_READER=1`, or under 40 columns, there is no box
+  and the banner is not centred.
+- UX, patch 2 (docs/lld/ux.md U2): a chat with no network allowlist starts Garuda's proxy with no
+  host on it ("watch mode"). A command in the sandbox that needs a host now asks for it (once, this
+  session, or no) instead of failing with a DNS error; an empty list needs no consent. After "this
+  session", Garuda says which line in `network.allow` keeps it for the project (a preset such as
+  `npm` when the host belongs to one). The receipt names a blocked host. `-p`, jobs and evals keep
+  no network, as before.
+- The chat shows your requests with a bar (`▌`) on every line, and Garuda's answer starts with `✦`,
+  so requests and answers differ at a glance. With `GARUDA_SCREEN_READER=1` they read "You:" and
+  "Garuda:".
+- The chat warns at start when `dist/` is older than its checkout ("run pnpm build first"), as the
+  evals do. `garudaCommit` and `staleBuildNote` moved to `src/buildCheck.ts`.
+- Docs: `architecture.md`, `hld.md` and the docs index describe Garuda as it is now. The release
+  history in their prose (version lists, "since 0.x", test counts per release) is gone; the
+  CHANGELOG keeps it.
+- UX, patch 1 (docs/lld/ux.md): a receipt after each turn that changed files, ran tests, ran a command outside the sandbox, or had a host blocked: the
+  files with their lines (from the undo snapshots), the last test run with its count for vitest,
+  jest, node:test, pytest, Maven and `go test -v`, the commands in and outside the sandbox, blocked
+  hosts, what `/undo` puts back, and the session's cost per kept change. A turn that only answered
+  keeps the usage line, also when it ran read-only commands in the sandbox (the first live run: a
+  `find` gave a receipt with no news). The session records the receipt (`receipt`), `/export` shows it, and
+  `-p --json` has it as `receipt`. Under 60 columns, in a pipe and with `GARUDA_SCREEN_READER=1` it
+  is plain lines.
+- The chat asks how to go on after the step limit: go on for as many steps again, summarise what is
+  left, or stop. After the same tool call 3 times it says to type a hint, and no longer names
+  `--resume`.
+- The chat's footer shows the context as a bar and a number (yellow from 70%, red from 80%). The key
+  hints show in the first 5 chats; then the footer says `? keys`, and `?` on an empty line lists them.
+- Design: the UX note (docs/lld/ux.md), with mock-ups and no code yet. A receipt after each
+  turn (files, tests, sandbox, undo, cost per kept change), a blocked host as a one-key fix,
+  next steps after a stop, a live plan, better diffs, rewind and fork, a morning brief from the
+  night shift, `garuda advise` (which model for this repo), a first-run tour, a quieter footer,
+  HTML export, and reduced motion and screen-reader modes. Ten decisions for the user.
+- Blog: "Garuda against Claude Code, with intervals": the eval method, the Claude Code
+  baseline, the faults the live runs found in the tasks, the Graphify A/B, the tuned prompt, and
+  the limits.
+- The Homebrew formula in the repository names 0.18.0 and its tarball hash.
+- Failure kinds: the eval report gives one line, "Failures by kind", that counts the failed runs
+  as `check` (the check failed), `protected-files`, `step-limit`, `token-limit`,
+  `repeated-calls`, `refusal`, `timeout`, `error` or `setup` (the key, the connection or the MCP
+  server), and names up to three runs of each kind. A limit or a timeout wins over the check that
+  failed after it. `report.json` has the same in `evidence.failures`, with each failed run's
+  steps, tokens and time.
+- The reports of the runs in the blog post are in `docs/quality-baseline/evals/0.18/`, with a
+  README that maps each run to the post and gives the `eval compare` commands. Local paths are
+  removed. A test checks that the files hold no local path and that `compare` gives the post's
+  numbers. The post links to them. Biome and the docs site leave the folder out.
+
 ## 0.18.0 (2026-11-04)
 
 - Biome 2.6.2 (with @inquirer/prompts 8.7.4, react 19.3.1 and smol-toml 1.9.1, from Dependabot's

package/README.md

@@ -7,7 +7,7 @@ Website and docs: https://madhu-sv.github.io/garuda/
 Garuda is a coding agent for the terminal, and for editors over the Agent Client Protocol. It reads
 and edits your code, runs commands in an OS sandbox, and keeps an audit log of its permission
 decisions. It works with Claude and with OpenAI-compatible models, local ones too. This is version
-0.18.0. Release notes: [CHANGELOG.md](CHANGELOG.md).
+0.18.2. Release notes: [CHANGELOG.md](CHANGELOG.md).
 
 ## What Garuda does on your machine
 
@@ -15,7 +15,8 @@ decisions. It works with Claude and with OpenAI-compatible models, local ones to
 - `write_file` and `edit_file` show a diff first. You allow once, allow for this session, or deny.
 - Commands (`bash`) run in an OS sandbox when your machine has one: Seatbelt on macOS, bubblewrap
   on Linux. There they run with no question, but they can write only in the project, temp folders
-  and package caches, cannot read secrets in your home folder (such as `~/.ssh`), and have no
+  and package caches, cannot read secrets in your home folder (such as `~/.ssh`) or your personal folders (Documents,
+  Downloads, Pictures and the like, outside the project), and have no
   network unless you open an allowlist. So a command can change project files without a question;
   `/undo` restores the files in its snapshot (ignored files, such as build output, are not in it).
 - A command outside the sandbox asks first, unless one of your allow rules covers it. The strict

package/dist/{acpCommand-IUZPLPLI.js → acpCommand-PUJHFWYU.js}

@@ -9,27 +9,29 @@ import {
 import {
   Runtime,
   auditDirFor
-} from "./chunk-NHWHG3SW.js";
-import "./chunk-65NUEOI7.js";
+} from "./chunk-STVFWSIL.js";
 import "./chunk-G2AX7MGS.js";
+import "./chunk-ULAJPMY7.js";
+import "./chunk-65NUEOI7.js";
+import "./chunk-4QRTTRVK.js";
 import "./chunk-A2QAIRQL.js";
 import {
   FileSessionStore
-} from "./chunk-UKIE4AAM.js";
+} from "./chunk-NLS7GMF7.js";
 import "./chunk-7KBOIPUT.js";
 import {
   loadSearchConfig
 } from "./chunk-B25VL4RU.js";
-import "./chunk-23FXBNBB.js";
+import "./chunk-MNE7HHX2.js";
 import {
   VERSION
-} from "./chunk-C57GHLKP.js";
+} from "./chunk-CFDRP6XW.js";
 import "./chunk-5DEWP3YX.js";
 import "./chunk-PZS3V6P3.js";
 import "./chunk-QANRZW7W.js";
 import "./chunk-ATW7BNIG.js";
 import "./chunk-AOGFYUKG.js";
-import "./chunk-B4SJY7KT.js";
+import "./chunk-LMKGSYJC.js";
 import "./chunk-BHZ6G65V.js";
 import "./chunk-P6XWWIEK.js";
 import "./chunk-UIRJOQQS.js";
@@ -52,7 +54,7 @@ async function acpCommand(options) {
   }
   const output = protocolOutput(process.stdout, process.stderr);
   const input = Readable.toWeb(process.stdin);
-  const { acpServer, setupNeeded } = await import("./server-VEYIJHEU.js");
+  const { acpServer, setupNeeded } = await import("./server-ASM73QVY.js");
   const server = acpServer({
     version: VERSION,
     // Not on native Windows: the setup there could not keep a key private (no 0600).
@@ -190,4 +192,4 @@ export {
   prepare,
   protocolOutput
 };
-//# sourceMappingURL=acpCommand-IUZPLPLI.js.map
+//# sourceMappingURL=acpCommand-PUJHFWYU.js.map

package/dist/advise-C777SZ5M.js

@@ -0,0 +1,433 @@
+#!/usr/bin/env node
+import {
+  taskHash
+} from "./chunk-6JQTPW3P.js";
+import {
+  garudaCommit
+} from "./chunk-YAQ6QLKB.js";
+import {
+  escapeHtml
+} from "./chunk-4BELTWP4.js";
+import {
+  BUILD_COMMIT
+} from "./chunk-E6YRCDSF.js";
+import {
+  compareReports,
+  costPerPass,
+  formatComparison,
+  loadReport,
+  repoEvalTasks
+} from "./chunk-A5DLAXB4.js";
+import {
+  loadModelsConfig,
+  resolveModel
+} from "./chunk-2FIV2VZP.js";
+import "./chunk-AX3A5KPH.js";
+import "./chunk-2OYTNNUJ.js";
+import "./chunk-NXJ4XLB5.js";
+import "./chunk-STVFWSIL.js";
+import "./chunk-G2AX7MGS.js";
+import "./chunk-ULAJPMY7.js";
+import "./chunk-65NUEOI7.js";
+import "./chunk-4QRTTRVK.js";
+import "./chunk-A2QAIRQL.js";
+import {
+  newSessionId
+} from "./chunk-NLS7GMF7.js";
+import {
+  costOf,
+  responseCost
+} from "./chunk-7KBOIPUT.js";
+import "./chunk-B25VL4RU.js";
+import "./chunk-MNE7HHX2.js";
+import {
+  VERSION
+} from "./chunk-CFDRP6XW.js";
+import "./chunk-5DEWP3YX.js";
+import "./chunk-PZS3V6P3.js";
+import "./chunk-QANRZW7W.js";
+import "./chunk-ATW7BNIG.js";
+import "./chunk-WK5NE6RY.js";
+import "./chunk-AOGFYUKG.js";
+import "./chunk-QXTHEFKP.js";
+import "./chunk-LMKGSYJC.js";
+import "./chunk-BHZ6G65V.js";
+import "./chunk-P6XWWIEK.js";
+import "./chunk-UIRJOQQS.js";
+import "./chunk-4LFJFDCF.js";
+import "./chunk-3NJNVZTK.js";
+
+// src/cli/advise.ts
+import { mkdirSync, readdirSync, readFileSync, statSync, writeFileSync } from "fs";
+import { basename, join } from "path";
+import { createInterface } from "readline/promises";
+var ADVISE_MODELS = ["claude-haiku-5-5", "claude-sonnet-5-5"];
+var ADVISE_REPEAT = 2;
+var ADVISE_MAX_COST = 2;
+var GUESS_INPUT_TOKENS = 6e4;
+var GUESS_OUTPUT_TOKENS = 1500;
+function estimateUsd(price, runs) {
+  if (price === void 0) return void 0;
+  return runs * (GUESS_INPUT_TOKENS * price.input + GUESS_OUTPUT_TOKENS * price.output) / 1e6;
+}
+function adviceText(a, b, c, tasks) {
+  const rough = tasks < 10 ? ` With ${tasks} task${tasks === 1 ? "" : "s"}, this is a rough guide.` : "";
+  if (c.verdict === "B passes more")
+    return `Advice: ${b} for this repo. It passes more often.${rough}`;
+  if (c.verdict === "B passes less")
+    return `Advice: ${a} for this repo. It passes more often.${rough}`;
+  if (c.rateA === 0 && c.rateB === 0) {
+    return `Advice: none. Neither model passed a task, so the check itself may fail: read a run's "reason" in report.json before you trust this.`;
+  }
+  const ratio = c.costRatio;
+  if (ratio === void 0) {
+    return `Advice: no clear difference in passes, and the cost could not be compared.${rough}`;
+  }
+  const times = (x) => `${x.toFixed(1)} times`;
+  if (ratio.verdict === "B costs less per accepted change") {
+    return `Advice: ${b} for this repo. It passes as often, and ${a} costs ${times(1 / ratio.value)} as much per accepted change.${rough}`;
+  }
+  if (ratio.verdict === "B costs more per accepted change") {
+    return `Advice: ${a} for this repo. It passes as often, and ${b} costs ${times(ratio.value)} as much per accepted change.${rough}`;
+  }
+  const cheaper = ratio.value <= 1 ? b : a;
+  return `Advice: either. No clear difference in passes or in cost; ${cheaper} was cheaper in these runs.${rough}`;
+}
+function adviceHtml(comparison, advice) {
+  return [
+    "<!doctype html>",
+    '<html lang="en"><head><meta charset="utf-8">',
+    `<meta http-equiv="Content-Security-Policy" content="default-src 'none'; style-src 'unsafe-inline'">`,
+    "<title>Garuda advise</title>",
+    "<style>body{font:15px/1.5 system-ui,sans-serif;max-width:900px;margin:32px auto;padding:0 16px}",
+    "pre{font:13px/1.45 ui-monospace,monospace;white-space:pre-wrap}.advice{font-weight:600}</style>",
+    "</head><body><h1>Which model for this repository</h1>",
+    `<p class="advice">${escapeHtml(advice)}</p>`,
+    `<pre>${escapeHtml(comparison)}</pre>`,
+    "</body></html>",
+    ""
+  ].join("\n");
+}
+var ADVISE_PARALLEL = 4;
+function pastRuns(evalsDir) {
+  const folders = [];
+  try {
+    for (const name of readdirSync(evalsDir)) {
+      const path = join(evalsDir, name);
+      folders.push(path, join(path, "a"), join(path, "b"));
+    }
+  } catch {
+    return [];
+  }
+  const out = [];
+  for (const folder of folders) {
+    const file = join(folder, "report.json");
+    try {
+      const data = JSON.parse(readFileSync(file, "utf8"));
+      const garuda = data.garuda;
+      out.push({
+        path: folder,
+        model: typeof data.spec === "string" ? data.spec : typeof data.model === "string" ? data.model : void 0,
+        version: typeof garuda?.version === "string" ? garuda.version : void 0,
+        commit: typeof garuda?.commit === "string" ? garuda.commit : void 0,
+        suite: typeof data.suite === "string" ? data.suite : void 0,
+        repeat: typeof data.repeat === "number" ? data.repeat : void 0,
+        seed: typeof data.seed === "number" ? data.seed : void 0,
+        tasks: typeof data.tasks === "object" && data.tasks !== null ? data.tasks : void 0,
+        results: Array.isArray(data.results) ? data.results : [],
+        mtimeMs: statSync(file).mtimeMs
+      });
+    } catch {
+    }
+  }
+  return out.sort((x, y) => y.mtimeMs - x.mtimeMs);
+}
+var sameTasks = (a, b) => a !== void 0 && Object.keys(a).length === Object.keys(b).length && Object.entries(b).every(([id, hash]) => a[id] === hash);
+function reusableRun(runs, want) {
+  const count = Object.keys(want.tasks).length * want.repeat;
+  return runs.find(
+    (r) => r.model === want.model && r.version === want.version && r.commit === want.commit && r.suite === "repo" && r.repeat === want.repeat && r.seed === want.seed && sameTasks(r.tasks, want.tasks) && r.results.length === count
+  );
+}
+function pastCostPerRun(runs, model, tasks) {
+  const results = runs.filter((r) => r.model === model && sameTasks(r.tasks, tasks)).flatMap((r) => r.results);
+  if (results.length === 0 || results.some((r) => r.costUsd === void 0)) return void 0;
+  return results.reduce((sum, r) => sum + (r.costUsd ?? 0), 0) / results.length;
+}
+function repricedSessionCost(file, price) {
+  let text;
+  try {
+    text = readFileSync(file, "utf8");
+  } catch {
+    return void 0;
+  }
+  let usd = 0;
+  for (const line of text.split("\n")) {
+    if (line.trim() === "") continue;
+    let record;
+    try {
+      record = JSON.parse(line);
+    } catch {
+      return void 0;
+    }
+    if (record.type === "unreported" || record.type === "child_debit") return void 0;
+    if (line.includes('"subagent":{')) return void 0;
+    if (record.type === "assistant") {
+      const cost = responseCost(record.response, price);
+      if (cost === void 0) return void 0;
+      usd += cost;
+    }
+    if (record.type === "side_usage") usd += costOf(record.usage, price);
+  }
+  return usd;
+}
+function repricedCostPerRun(runs, model, tasks, price) {
+  if (price === void 0) return void 0;
+  const costs = [];
+  for (const run of runs.filter((r) => r.model === model && sameTasks(r.tasks, tasks))) {
+    for (const result of run.results) {
+      const file = result.sessionFile;
+      if (file === void 0) {
+        costs.push(void 0);
+        continue;
+      }
+      const near = join(run.path, basename(file));
+      costs.push(repricedSessionCost(near, price) ?? repricedSessionCost(file, price));
+    }
+  }
+  const known = costs.filter((c) => c !== void 0);
+  if (known.length === 0 || known.length * 2 < costs.length) return void 0;
+  return known.reduce((sum, c) => sum + c, 0) / known.length;
+}
+function repriceFactor(runs, source, target, price) {
+  const useful = runs.filter((r) => r.suite === "repo" && r.results.some((x) => x.passed));
+  const sets = [];
+  for (const run of useful.filter((r) => r.model === source && r.tasks !== void 0)) {
+    const tasks = run.tasks;
+    if (sets.some((t) => sameTasks(t, tasks))) continue;
+    if (useful.some((r) => r.model === target && sameTasks(r.tasks, tasks))) sets.push(tasks);
+  }
+  const ratios = [];
+  for (const tasks of sets) {
+    const real = pastCostPerRun(useful, target, tasks);
+    const repriced = repricedCostPerRun(useful, source, tasks, price);
+    if (real !== void 0 && repriced !== void 0 && repriced > 0) ratios.push(real / repriced);
+  }
+  if (ratios.length === 0) return void 0;
+  return { value: ratios.reduce((sum, r) => sum + r, 0) / ratios.length, sets: ratios.length };
+}
+function costEstimate(runs, own, repriced, price) {
+  if (own !== void 0) return { usd: own * runs, how: "from earlier runs" };
+  if (repriced.perRun !== void 0) {
+    const f = repriced.factor;
+    return {
+      usd: repriced.perRun * (f?.value ?? 1) * runs,
+      how: f === void 0 ? `from ${repriced.from}'s requests at this model's prices` : `from ${repriced.from}'s requests at this model's prices, \xD7 ${f.value.toFixed(2)} as in ${f.sets} earlier comparison${f.sets === 1 ? "" : "s"}`
+    };
+  }
+  return { usd: estimateUsd(price, runs), how: "a guess" };
+}
+var sumKnown = (values) => values.some((v) => v === void 0) ? void 0 : values.reduce((sum, v) => sum + (v ?? 0), 0);
+function continueCommand(a, b, repeat, seed, maxCost) {
+  return `garuda advise --models ${a},${b} --repeat ${repeat} --seed ${seed} --max-cost ${Math.ceil(maxCost)}`;
+}
+async function runAdvise(options) {
+  const [a, b] = options.models ?? ADVISE_MODELS;
+  if (a === void 0 || b === void 0 || options.models?.length === 1 || (options.models?.length ?? 2) > 2) {
+    process.stderr.write("Give two models: --models <a>,<b>.\n");
+    return 1;
+  }
+  const repeat = options.repeat ?? ADVISE_REPEAT;
+  const maxCost = options.maxCost ?? ADVISE_MAX_COST;
+  const seed = options.seed ?? 1;
+  const parallel = options.parallel ?? ADVISE_PARALLEL;
+  if (!Number.isInteger(repeat) || repeat < 1 || !(maxCost > 0) || !Number.isInteger(parallel) || parallel < 1) {
+    process.stderr.write(
+      "--repeat and --parallel must be whole numbers of 1 or more, and --max-cost above 0.\n"
+    );
+    return 1;
+  }
+  const { repoSuite, runEvalCommand } = await import("./evalCommand-MRO6ENCA.js");
+  const suite = await repoSuite();
+  if (typeof suite === "string") {
+    process.stderr.write(`${suite}
+`);
+    return 1;
+  }
+  const repoTasks = repoEvalTasks(suite);
+  const tasks = repoTasks.length;
+  const hashes = Object.fromEntries(repoTasks.map((t) => [t.id, taskHash(t)]));
+  const models = await loadModelsConfig();
+  if (models.problem !== void 0) {
+    process.stderr.write(`${models.problem}
+`);
+    return 1;
+  }
+  let prices;
+  try {
+    prices = [resolveModel(a, models.config).info.price, resolveModel(b, models.config).info.price];
+  } catch (error) {
+    process.stderr.write(`${error.message}
+`);
+    return 1;
+  }
+  const evalsDir = join(process.cwd(), ".garuda", "evals");
+  const past = pastRuns(evalsDir);
+  const commit = BUILD_COMMIT ?? await garudaCommit();
+  const want = { repeat, seed, tasks: hashes, version: VERSION, commit };
+  const reuse = [
+    reusableRun(past, { ...want, model: a }),
+    reusableRun(past, { ...want, model: b })
+  ];
+  const runs = tasks * repeat;
+  const perRun = [pastCostPerRun(past, a, hashes), pastCostPerRun(past, b, hashes)];
+  const repriced = (i, from) => perRun[i] === void 0 ? repricedCostPerRun(past, from, hashes, prices[i]) : void 0;
+  const factors = [
+    perRun[0] === void 0 ? repriceFactor(past, b, a, prices[0]) : void 0,
+    perRun[1] === void 0 ? repriceFactor(past, a, b, prices[1]) : void 0
+  ];
+  const estimates = [
+    costEstimate(
+      runs,
+      perRun[0],
+      { perRun: repriced(0, b), from: b, factor: factors[0] },
+      prices[0]
+    ),
+    costEstimate(
+      runs,
+      perRun[1],
+      { perRun: repriced(1, a), from: a, factor: factors[1] },
+      prices[1]
+    )
+  ];
+  const line = (model, i) => {
+    const r = reuse[i];
+    const e = estimates[i];
+    return r !== void 0 ? `  ${model}: reuses ${r.path} (Garuda ${r.version}, ${r.results.length} runs)` : `  ${model}: ${runs} runs, ${e?.usd === void 0 ? "cost unknown" : `about $${e.usd.toFixed(2)} (${e.how})`}`;
+  };
+  const toSpend = sumKnown(
+    [0, 1].filter((i) => reuse[i] === void 0).map((i) => estimates[i]?.usd)
+  );
+  process.stderr.write(
+    [
+      `Your repo: ${tasks} task${tasks === 1 ? "" : "s"} from its history (tests that fail before the commit and pass after it).`,
+      `${repeat} run${repeat === 1 ? "" : "s"} a task, ${parallel} at a time, seed ${seed}:`,
+      line(a, 0),
+      line(b, 1),
+      `Cap: --max-cost $${maxCost}; it stops before a model that would pass it.`,
+      ""
+    ].join("\n")
+  );
+  if (toSpend !== void 0 && toSpend > maxCost) {
+    process.stderr.write(
+      `The estimate passes the cap. To go on: ${continueCommand(a, b, repeat, seed, toSpend * 1.25)}
+`
+    );
+    return 1;
+  }
+  if (options.yes !== true && (reuse[0] === void 0 || reuse[1] === void 0)) {
+    if (!process.stdin.isTTY) {
+      process.stderr.write("Add --yes to start without a question.\n");
+      return 1;
+    }
+    const rl = createInterface({ input: process.stdin, output: process.stderr });
+    const answer = await rl.question("Start? (y/N) ");
+    rl.close();
+    if (!/^y(es)?$/i.test(answer.trim())) return 1;
+  }
+  const dir = join(evalsDir, `advise-${newSessionId()}`);
+  mkdirSync(dir, { recursive: true });
+  const runModel = async (model, label, i) => {
+    const earlier = reuse[i];
+    if (earlier !== void 0) {
+      process.stderr.write(`Reusing ${earlier.path} for ${model}.
+`);
+      return earlier.path;
+    }
+    const outDir = join(dir, label);
+    const code = await runEvalCommand({
+      model,
+      suite: "repo",
+      repeat,
+      shuffle: true,
+      seed,
+      parallel,
+      outDir,
+      progress: model.replace(/^claude-/, "")
+    });
+    return code === 1 ? void 0 : outDir;
+  };
+  const pathA = await runModel(a, "a", 0);
+  if (pathA === void 0) return 1;
+  const first = await loadReport(pathA);
+  if (reuse[0] === void 0 && reuse[1] === void 0) {
+    const spent = costPerPass(first.results);
+    const spentA = spent.kind === "unknown" ? void 0 : spent.totalUsd;
+    const nextB = costEstimate(
+      runs,
+      perRun[1],
+      {
+        perRun: repricedCostPerRun(pastRuns(evalsDir), a, hashes, prices[1]),
+        from: a,
+        factor: factors[1]
+      },
+      prices[1]
+    ).usd;
+    if (spentA !== void 0 && nextB !== void 0 && spentA + nextB > maxCost) {
+      process.stderr.write(
+        [
+          `Stopped before ${b}: ${a} cost $${spentA.toFixed(2)}, and ${b} would cost about $${nextB.toFixed(2)}, past $${maxCost}.`,
+          `${a}'s run is kept and will be reused. To go on: ${continueCommand(a, b, repeat, seed, nextB * 1.25)}`,
+          ""
+        ].join("\n")
+      );
+      return 1;
+    }
+  }
+  const pathB = await runModel(b, "b", 1);
+  if (pathB === void 0) return 1;
+  const second = await loadReport(pathB);
+  const result = compareReports(first, second, { seed });
+  if ("problems" in result) {
+    process.stderr.write(`The two runs cannot be compared:
+${result.problems.join("\n")}
+`);
+    return 1;
+  }
+  const comparison = formatComparison(first, second, result.comparison);
+  const advice = adviceText(a, b, result.comparison, tasks);
+  writeFileSync(join(dir, "advice.txt"), `${advice}
+
+${comparison}
+`);
+  writeFileSync(join(dir, "report.html"), adviceHtml(comparison, advice));
+  process.stdout.write(`
+${comparison}
+
+${advice}
+Report: ${join(dir, "report.html")}
+`);
+  return 0;
+}
+export {
+  ADVISE_MAX_COST,
+  ADVISE_MODELS,
+  ADVISE_PARALLEL,
+  ADVISE_REPEAT,
+  GUESS_INPUT_TOKENS,
+  GUESS_OUTPUT_TOKENS,
+  adviceHtml,
+  adviceText,
+  continueCommand,
+  costEstimate,
+  estimateUsd,
+  pastCostPerRun,
+  pastRuns,
+  repriceFactor,
+  repricedCostPerRun,
+  repricedSessionCost,
+  reusableRun,
+  runAdvise,
+  sumKnown
+};
+//# sourceMappingURL=advise-C777SZ5M.js.map