model-orchestrator 0.1.21 → 0.1.23

package/CHANGELOG.md

@@ -4,6 +4,28 @@ All notable changes to this project are documented here. The format follows [Kee
 
 ## [Unreleased]
 
+## [0.1.23] - 2026-09-15
+
+### Added
+
+- **`cli-run` says WHY a lane failed.** Every run lands in one class with its own exit code: `auth` 14, `quota` 15, `rejected` 16, `refused` 17, `cut_short` 18, next to the existing `empty` 10, `no_output` 11, `timeout` 12 and `unavailable` 13. A missing API key, a spent quota and an unknown model id used to share exit 10 or the vendor's own code, and each needs a different response: a missing key is not a model fault, and retrying a spent quota cannot help. Signals are read from each lane's authoritative error fields only, never the model's prose, with precedence auth, quota, rejected, refused, cut_short, empty.
+- **`refused=N` on every run.** Tool calls a hook or deny rule blocked, counted from qwen's `permission_denials`, agy's deny-rule steps, codex's router `Rejected(` lines, and grok's session transcript (its `sessionId` is charset-checked, the path is contained to the sessions root, lines must name the same session, and the read is capped at 5 MiB). `null` when a lane gives no signal. A deliverable with refused calls is still exit 0.
+- **A problem line and a fix line** on the terminal for every failure, and for an `ok` run with refused calls, so a calling agent can relay "this is what went wrong, this is the fix" and ask.
+- **Terminal output is redacted** before it prints: JSON credential keys, `Authorization:` values, bearer values, URL query credentials, and common key prefixes, redacted before any clipping.
+- **The durable log gains `class` and `refused`.** Both are fixed values; the log still never holds provider text, the problem line or stderr.
+
+### Changed
+
+- **A nonzero vendor exit is no longer passed through as cli-run's exit code.** The class owns the code, and the vendor's own code stays in the log as `cli_rc`. A nonzero exit nothing else explains is `cut_short` (18), and it is still never `ok`. If a script compared `cli-run`'s exit code with a specific vendor code, compare `cli_rc` in the log instead; `!= 0` checks are unaffected.
+- **A lane killed by a signal, or output past the 16 MiB buffer, is `cut_short` (18)**, not 10. Exit 10 now means only an empty run or an unmet `--expect-*` contract.
+- **agy's judge refuses a non-object terminal `result` as `bad_last_event`** (was `bad_status`), and **qwen's judge refuses a non-string `error.message` as `error_message_not_string`**, so both classify as `cut_short`. hermes' stderr cause (degraded free tier, bad `--toolsets`) is now named in its detail line.
+
+## [0.1.22] - 2026-09-12
+
+### Added
+
+- **SuperGrok Plus as a `grok` plan (high headroom).** xAI's pricing page lists it with "Significantly higher usage across Chat, Imagine, Voice & Build", so `--plans grok=supergrok-plus` now works and counts as a high-headroom lane for plan guidance and `--effort-auto`. SuperGrok Heavy stays out: the page states no Build usage for it. A test holds both.
+
 ## [0.1.21] - 2026-09-12
 
 ### Added
@@ -329,7 +351,9 @@ First release.
 - Tests: a case per fix, judges proven to go red, mutation checks; `npm test` prints the current count.
 - Adversarial audit: two Codex rounds plus a two-engine review (Codex, Antigravity); findings and fixes in `docs/audit-brief.md`. After the review: subagents go to the project root (`--project`), snippet paths computed from `--dir`, lane sections rendered from the selection, a primary agent required, level 3 asks for API keys separately from CLIs, images and CLI installs pinned, an activation summary at the end of every install.
 
-[Unreleased]: https://github.com/aunysillyme/model-orchestrator/compare/v0.1.21...HEAD
+[Unreleased]: https://github.com/aunysillyme/model-orchestrator/compare/v0.1.23...HEAD
+[0.1.23]: https://github.com/aunysillyme/model-orchestrator/compare/v0.1.22...v0.1.23
+[0.1.22]: https://github.com/aunysillyme/model-orchestrator/compare/v0.1.21...v0.1.22
 [0.1.21]: https://github.com/aunysillyme/model-orchestrator/compare/v0.1.20...v0.1.21
 [0.1.20]: https://github.com/aunysillyme/model-orchestrator/compare/v0.1.19...v0.1.20
 [0.1.19]: https://github.com/aunysillyme/model-orchestrator/compare/v0.1.18...v0.1.19

package/bin/cli-run.mjs

@@ -27,18 +27,30 @@
 // bounded by the OS ARG_MAX, so: no secrets in a prompt, and very large briefs
 // should be referenced by path in the prompt rather than pasted into it.
 //
-// Exit codes
-//   0   structurally accepted non-empty response (and every --expect-* contract met)
-//   10  ran, produced no deliverable, or a contract was not met, or killed by signal
-//   11  produced no output at all
-//   12  timed out (the lane AND its descendants are killed as a process group)
-//   13  lane unavailable (missing binary, disabled in lanes.json, or lanes.json malformed)
+// Exit codes: one per failure CLASS, so the code says what to do next.
+//   0   ok: structurally accepted non-empty response (and every --expect-* contract met)
+//   10  empty: ran and delivered nothing, or a contract was not met
+//   11  no_output: produced no output at all
+//   12  timeout: the lane AND its descendants are killed as a process group
+//   13  unavailable: missing binary, disabled in lanes.json, or lanes.json malformed
+//   14  auth: the lane's own error says a credential is missing or not logged in
+//   15  quota: the lane's own error says usage limit, credits or rate limit
+//   16  rejected: the upstream rejected the request (bad model id, bad request)
+//   17  refused: no deliverable, and the lane reports tool calls a hook or deny rule blocked
+//   18  cut_short: no trustworthy finish: a missing or non-success terminal event, a lane
+//       killed by a signal, output past the 16 MiB buffer, or a nonzero vendor exit
 //   130 / 143  cli-run itself received SIGINT / SIGTERM: the lane's process group was killed first
 //   2   usage error in cli-run itself
-//   N   the lane exited N != 0: passed through, verdict exit_nonzero, even if text came back
+// A run that delivered AND had tool calls refused is still 0, with refused=N and a
+// problem/fix pair on the terminal. The vendor's own exit code is logged as cli_rc.
+// Precedence when several signals are present: auth, quota, rejected, refused,
+// cut_short, empty. Only the lane's authoritative error fields are searched,
+// never the model's prose, so an answer that merely mentions "rate limit" is
+// not a quota failure.
 //
-// The durable log stores a FIXED reason code per run (see REASONS), never a
-// provider-supplied string. Bounded vendor stderr goes to your terminal only.
+// The durable log stores a FIXED reason code and class per run (see REASONS,
+// CLASS_CODES), never a provider-supplied string. Bounded vendor stderr and the
+// problem/fix lines go to your terminal only, redacted.
 //
 // ROUTE: which model and reasoning effort a lane ran with.
 // A lane with no --model and no lanes.json default inherits whatever its own
@@ -55,12 +67,21 @@ import { spawn, spawnSync } from 'node:child_process';
 import { StringDecoder } from 'node:string_decoder';
 import { createHash } from 'node:crypto';
 import { readFileSync, existsSync, mkdirSync, appendFileSync, mkdtempSync, rmSync, accessSync, constants, realpathSync, statSync, lstatSync, openSync, readSync, closeSync } from 'node:fs';
-import { join, dirname, delimiter, resolve } from 'node:path';
+import { join, dirname, delimiter, resolve, relative, isAbsolute, sep } from 'node:path';
 import { tmpdir, homedir } from 'node:os';
 import { fileURLToPath, pathToFileURL } from 'node:url';
 
 export const LANES = ['grok', 'codex', 'agy', 'hermes', 'qwen'];
 export const OK = 0, NO_DELIVERABLE = 10, NO_OUTPUT = 11, TIMEOUT = 12, UNAVAILABLE = 13, USAGE = 2;
+export const AUTH = 14, QUOTA = 15, REJECTED = 16, REFUSED = 17, CUT_SHORT = 18;
+
+// The closed set of failure classes. Every judged run lands in exactly one, and
+// the class owns the exit code. `interrupted` (cli-run itself was signalled)
+// is logged as a class but exits 130 or 143.
+export const CLASS_CODES = {
+  ok: OK, empty: NO_DELIVERABLE, no_output: NO_OUTPUT, timeout: TIMEOUT, unavailable: UNAVAILABLE,
+  auth: AUTH, quota: QUOTA, rejected: REJECTED, refused: REFUSED, cut_short: CUT_SHORT
+};
 
 // Every reason that may reach the durable log. A judge or the wrapper picks
 // one of these; anything else is written as 'unknown'. Provider text never
@@ -69,8 +90,8 @@ export const REASONS = new Set([
   'ok', 'not_json', 'bad_stop_reason', 'empty_text', 'no_terminal_event', 'empty_output_file',
   'bad_status', 'empty_response', 'exit_nonzero', 'empty_stdout', 'bad_event_array', 'bad_last_event',
   'not_result', 'bad_subtype', 'is_error', 'result_not_string', 'empty_result', 'api_error_in_result',
-  'telemetry_absent', 'total_errors_unreadable', 'total_errors', 'contract_unmet',
-  'timeout', 'unavailable', 'killed', 'disabled', 'lanes_json_malformed', 'no_output', 'unknown'
+  'telemetry_absent', 'total_errors_unreadable', 'total_errors', 'contract_unmet', 'error_message_not_string',
+  'judge_raised', 'timeout', 'unavailable', 'killed', 'disabled', 'lanes_json_malformed', 'no_output', 'unknown'
 ]);
 
 const LOG = join(homedir(), '.ai-orchestrator', 'cli-run.log.jsonl');
@@ -163,7 +184,8 @@ export function judgeCodex(rc, out, err, fileText) {
 export function judgeAgy(rc, out) {
   const ev = lastJsonLine(out, '"result"', (o) => o && o.event === 'result');
   if (ev === null) return fail('no_terminal_event', 'no terminal result event');
-  const term = ev.result && typeof ev.result === 'object' && !Array.isArray(ev.result) ? ev.result : {};
+  if (!ev.result || typeof ev.result !== 'object' || Array.isArray(ev.result)) return fail('bad_last_event', 'terminal result was not an object');
+  const term = ev.result;
   const status = term.status;
   const text = typeof term.response === 'string' ? term.response.trim() : '';
   if (status !== 'SUCCESS') return fail('bad_status', `status=${JSON.stringify(status)}`);
@@ -173,7 +195,12 @@ export function judgeAgy(rc, out) {
 export function judgeHermes(rc, out, err) {
   const text = String(out || '').trim();
   if (rc !== 0) {
-    const why = { 1: 'no final response (agent produced nothing)', 2: 'bad args, or completed with an empty response' }[rc] || 'unknown failure';
+    let why = { 1: 'no final response (agent produced nothing)', 2: 'bad args, or completed with an empty response' }[rc] || 'unknown failure';
+    // hermes collapses every upstream failure into one exit code; its stderr
+    // is the only place the cause is named.
+    const blob = String(err || '').toLowerCase(); // stderr only: stdout is the agent's own prose
+    if (hermesQuotaText(blob)) why += ': upstream free tier degraded or limited, a retry is reasonable';
+    else if (blob.includes('toolset')) why += ': invalid --toolsets value, a caller bug and not a lane fault';
     return fail('exit_nonzero', `hermes exit ${rc}: ${why}`);
   }
   return text ? pass(text, 'exit 0') : fail('empty_stdout', 'exit 0 but empty stdout');
@@ -192,8 +219,11 @@ export function judgeQwen(rc, out) {
   if (term.type !== 'result') return fail('not_result', `last event was ${JSON.stringify(term.type)}, not result`);
   if (term.subtype !== 'success') {
     const e = term.error;
+    if (e && typeof e === 'object' && e.message !== undefined && typeof e.message !== 'string') {
+      return fail('error_message_not_string', `subtype=${JSON.stringify(term.subtype)}, error.message was not a string`);
+    }
     const msg = e && typeof e === 'object' && typeof e.message === 'string' ? e.message : e ? String(e) : '';
-    return fail('bad_subtype', `subtype=${JSON.stringify(term.subtype)}` + (msg ? `: ${msg.slice(0, 120)}` : ''));
+    return fail('bad_subtype', `subtype=${JSON.stringify(term.subtype)}` + (msg ? `: ${redact(msg).slice(0, 120)}` : ''));
   }
   if (term.is_error) return fail('is_error', 'is_error true');
   if (term.result != null && typeof term.result !== 'string') return fail('result_not_string', `result was ${typeof term.result}, not a string`);
@@ -201,7 +231,7 @@ export function judgeQwen(rc, out) {
   if (!text) return fail('empty_result', 'success but empty result');
   // qwen reports success even when the upstream API rejected the call; the
   // error text lands in `result`. These two checks are the honest ones.
-  if (text.startsWith('[API Error:')) return fail('api_error_in_result', `success flag lied, result is an API error: ${text.slice(0, 140)}`);
+  if (text.startsWith('[API Error:')) return fail('api_error_in_result', `success flag lied, result is an API error: ${redact(text).slice(0, 140)}`);
   const stats = term.stats;
   const models = stats && typeof stats === 'object' ? stats.models : null;
   if (!models || typeof models !== 'object' || Array.isArray(models) || Object.keys(models).length === 0) {
@@ -383,6 +413,406 @@ export function judge(lane, rc, out, err, outFile) {
   }
 }
 
+// A judge is type-guarded, but this is the backstop: an exception from any
+// judge would escape main() as cli-run's own crash and be misread as a
+// wrapper bug. It becomes a classifiable verdict instead (class cut_short).
+export function safeJudge(lane, rc, out, err, outFile) {
+  try {
+    return judge(lane, rc, out, err, outFile);
+  } catch (e) {
+    return fail('judge_raised', `judge raised ${(e && e.name) || 'Error'}: ${(e && e.message) || e}`);
+  }
+}
+
+// --- failure classes: WHY a run failed --------------------------------------
+// Every function here returns a safe default instead of throwing. Signals are
+// read from a lane's authoritative error fields only, never from assistant
+// prose, and each one was taken from a captured vendor shape, not guessed.
+
+// Terminal output only; nothing redacted here reaches the durable log anyway.
+// Redact BEFORE clipping: a clip first can cut a long token so its tail no
+// longer matches any pattern.
+export const SECRET_PATTERNS = [
+  // A JSON string value, escapes included; an unterminated value (a clipped line) runs to the end.
+  // The optional backslashes also catch a JSON body escaped inside another string.
+  [/(\\?"(?:api[_-]?key|token|secret|password|access[_-]?token)\\?"\s*:\s*)\\?"(?:[^"\\]|\\.)*(?:"|$)/gi, '$1"[REDACTED]"'],
+  [/(authorization\s*:\s*)(\S+)\s+\S+/gi, '$1$2 [REDACTED]'],
+  [/([?&](?:token|key|api_key|access_token|sig)=)[^&\s"'<>]+/gi, '$1[REDACTED]'],
+  [/(bearer\s+)[A-Za-z0-9._+/=-]{8,}/gi, '$1[REDACTED]'],
+  [/sk-[A-Za-z0-9_-]{8,}/g, '[REDACTED]'],
+  [/xai-[A-Za-z0-9_-]{8,}/g, '[REDACTED]'],
+  [/ghp_[A-Za-z0-9_-]{8,}/g, '[REDACTED]'],
+  [/AIza[A-Za-z0-9_-]{8,}/g, '[REDACTED]']
+];
+export function redact(text) {
+  if (typeof text !== 'string') return text;
+  let s = text;
+  for (const [pat, repl] of SECRET_PATTERNS) s = s.replace(pat, repl);
+  return s;
+}
+
+function* jsonLines(out) {
+  for (const raw of String(out || '').split('\n')) {
+    const line = raw.trim();
+    if (!line.startsWith('{') || line.length > 1_000_000) continue;
+    let o;
+    try {
+      o = JSON.parse(line);
+    } catch {
+      continue;
+    }
+    if (o && typeof o === 'object' && !Array.isArray(o)) yield o;
+  }
+}
+
+const isObj = (v) => !!v && typeof v === 'object' && !Array.isArray(v);
+
+// codex: only its own `error`, `turn.failed` and error-item events. Never an
+// agent_message, which is the model talking.
+export function codexErrorEventsText(out) {
+  const parts = [];
+  for (const o of jsonLines(out)) {
+    if (o.type === 'error' && typeof o.message === 'string') parts.push(o.message);
+    else if (o.type === 'turn.failed') {
+      if (isObj(o.error) && typeof o.error.message === 'string') parts.push(o.error.message);
+      else if (typeof o.error === 'string') parts.push(o.error);
+    } else if (o.type === 'item.completed' && isObj(o.item) && o.item.type === 'error' && typeof o.item.message === 'string') {
+      parts.push(o.item.message);
+    }
+  }
+  return parts.join('\n');
+}
+
+// agy: only the terminal result event's own status and error fields.
+export function agyResultFieldsText(out) {
+  const parts = [];
+  for (const o of jsonLines(out)) {
+    if (o.event !== 'result' || !isObj(o.result)) continue;
+    if (typeof o.result.status === 'string') parts.push(o.result.status);
+    const e = o.result.error;
+    if (typeof e === 'string') parts.push(e);
+    else if (isObj(e) && typeof e.message === 'string') parts.push(e.message);
+  }
+  return parts.join('\n');
+}
+
+// codex often carries the upstream's JSON error body as a string inside
+// `message`; one level is unwrapped to the human-readable text underneath.
+function unwrapJsonMessage(msg) {
+  try {
+    const p = JSON.parse(msg);
+    if (isObj(p) && isObj(p.error) && typeof p.error.message === 'string') return p.error.message;
+  } catch {
+    /* not JSON */
+  }
+  return msg;
+}
+
+// The single most specific native codex error, preferring a top-level `error`
+// event over `turn.failed`, so a problem line names the real cause instead of
+// "no terminal turn.completed event".
+export function codexPrimaryError(out) {
+  const errors = [], failed = [];
+  for (const o of jsonLines(out)) {
+    if (o.type === 'error' && typeof o.message === 'string' && o.message) errors.push(unwrapJsonMessage(o.message));
+    else if (o.type === 'turn.failed') {
+      const m = isObj(o.error) ? o.error.message : o.error;
+      if (typeof m === 'string' && m) failed.push(unwrapJsonMessage(m));
+    }
+  }
+  return errors[0] || failed[0] || null;
+}
+
+function hermesQuotaText(lower) {
+  return lower.includes('no usable content') || lower.includes('limit') || lower.includes('degraded');
+}
+
+// qwen: the terminal event's full error text, and a result that is an API error.
+// Never the display detail, which is clipped and can carry a model name.
+export function qwenErrorText(out) {
+  let events;
+  try {
+    events = JSON.parse(out);
+  } catch {
+    return '';
+  }
+  const term = Array.isArray(events) && events.length ? events[events.length - 1] : null;
+  if (!isObj(term)) return '';
+  const parts = [];
+  if (typeof term.error === 'string') parts.push(term.error);
+  else if (isObj(term.error) && typeof term.error.message === 'string') parts.push(term.error.message);
+  if (typeof term.result === 'string' && term.result.trim().startsWith('[API Error:')) parts.push(term.result);
+  return parts.join('\n');
+}
+
+function authoritativeBlob(lane, out, err, detail, rc) {
+  if (lane === 'codex') return `${codexErrorEventsText(out)}\n${err || ''}`;
+  if (lane === 'agy') return `${agyResultFieldsText(out)}\n${err || ''}`;
+  if (lane === 'hermes') return rc !== 0 ? String(err || '') : '';
+  if (lane === 'qwen') return `${qwenErrorText(out)}\n${err || ''}`;
+  return `${detail || ''}\n${err || ''}`; // grok: no auth, quota or rejected signal is defined
+}
+
+export function sigAuth(lane, blob) {
+  const b = blob.toLowerCase();
+  if (lane === 'qwen') return b.includes('missing api key');
+  if (lane === 'agy') return b.includes('you are not logged into antigravity') || b.includes('not authenticated');
+  return false; // codex, grok, hermes: no documented native auth signal
+}
+
+export function sigQuota(lane, blob) {
+  const b = blob.toLowerCase();
+  if (lane === 'qwen') return b.includes('[api error: 402') || b.includes('requires more credits') || blob.includes(' 429') || b.includes('rate limit');
+  if (lane === 'codex') return blob.includes('usage_limit_exceeded') || b.includes("you've hit your usage limit");
+  if (lane === 'hermes') return hermesQuotaText(b);
+  return false;
+}
+
+export function sigRejected(lane, blob) {
+  const b = blob.toLowerCase();
+  if (lane === 'qwen') return b.includes('[api error: 400') || b.includes('no endpoints found') || b.includes('failed to parse grammar');
+  if (lane === 'codex') return blob.includes('invalid_request_error');
+  if (lane === 'hermes') return b.includes('toolset');
+  return false;
+}
+
+// Which judge reasons mean the lane never reached a trustworthy finish, as
+// opposed to finishing cleanly with nothing in it (class empty).
+const CUT_SHORT_REASONS = {
+  grok: new Set(['not_json', 'bad_stop_reason']),
+  codex: new Set(['no_terminal_event']),
+  agy: new Set(['no_terminal_event', 'bad_last_event']),
+  qwen: new Set(['not_json', 'bad_event_array', 'bad_last_event', 'not_result', 'error_message_not_string']),
+  hermes: new Set()
+};
+function isCutShort(lane, reason, rc) {
+  if (reason === 'judge_raised') return true;
+  if (reason === 'ok') return true; // text came back but the vendor exited nonzero
+  // A nonzero vendor exit no signal explains never finished on its own terms.
+  // hermes' exit 2 is the one honest vendor code for "bad args, or an empty response".
+  if (rc !== 0) return !(lane === 'hermes' && rc === 2);
+  return !!(CUT_SHORT_REASONS[lane] && CUT_SHORT_REASONS[lane].has(reason));
+}
+
+// --- refused: how many tool calls a hook or deny rule blocked ----------------
+// null means the lane gave no readable signal, which is an unknown, never 0.
+export function refusedQwen(out) {
+  let events;
+  try {
+    events = JSON.parse(out);
+  } catch {
+    return null;
+  }
+  if (!Array.isArray(events) || !events.length || !isObj(events[events.length - 1])) return null;
+  const pd = events[events.length - 1].permission_denials;
+  return Array.isArray(pd) ? pd.length : null;
+}
+
+export function refusedAgy(out) {
+  let found = false, count = 0;
+  for (const o of jsonLines(out)) {
+    found = true;
+    const e = isObj(o.step_update) && isObj(o.step_update.tool_info) ? o.step_update.tool_info.error : null;
+    if (isObj(e) && e.type === 'TOOL_ERROR' && typeof e.message === 'string') {
+      const m = e.message.toLowerCase();
+      if (m.includes('permission check failed') || m.includes('matches user-configured deny rule')) count++;
+    }
+  }
+  return found ? count : null;
+}
+
+// codex's router writes one `Rejected(` line per blocked command on stderr.
+// Each line counts. Prose such as "operation not permitted" in an answer does
+// not: it matches a model merely explaining the phrase.
+const CODEX_ROUTER_REJECTED = /codex_core::tools::router:[^\n]*Rejected\(/gi;
+export function refusedCodex(out, err) {
+  let count = (String(err || '').match(CODEX_ROUTER_REJECTED) || []).length;
+  for (const o of jsonLines(out)) {
+    // Forward-compatible only: a structural denial item, never prose.
+    if (o.type === 'item.completed' && isObj(o.item) && (o.item.type === 'command_denied' || o.item.type === 'denied')) count++;
+  }
+  return count || null;
+}
+
+// grok never reports a refusal in stdout. It lives in the session transcript,
+// ~/.grok/sessions/<cwd, percent-encoded>/<sessionId>/updates.jsonl, one
+// {"params":{"sessionId","update"}} envelope per line. Two denial shapes are
+// counted: a PreToolUse hook run with status.blocked, and grok's own deny-rule
+// engine ("Denied by permission policy"). The sessionId comes from lane output,
+// so it is held to a narrow charset before any path is built, the resolved path
+// must stay inside the sessions root, every counted line must name the same
+// session, and the read is capped.
+export const GROK_SESSION_ID = /^[A-Za-z0-9-]{8,64}$/;
+export const GROK_TRANSCRIPT_CAP = 5 * 1024 * 1024;
+// grok encodes the cwd the way Python's urllib.parse.quote(cwd, safe="") does.
+function percentEncodePath(p) {
+  return encodeURIComponent(p).replace(/[!'()*]/g, (c) => '%' + c.charCodeAt(0).toString(16).toUpperCase());
+}
+// Path-aware containment. A string-prefix test accepts a sibling such as
+// "sessions-2", or on POSIX a directory literally named "sessions\outside".
+export function isInsideRoot(root, p, pathApi = { relative, isAbsolute, sep }) {
+  const rel = pathApi.relative(root, p);
+  return !!rel && !pathApi.isAbsolute(rel) && rel !== '..' && !rel.startsWith('..' + pathApi.sep);
+}
+export function refusedGrok(out, { root = process.env.CLI_RUN_GROK_SESSIONS_ROOT || join(homedir(), '.grok', 'sessions'), cwd = process.cwd(), cap = GROK_TRANSCRIPT_CAP } = {}) {
+  let o;
+  try {
+    o = JSON.parse(out);
+  } catch {
+    return null;
+  }
+  const sid = isObj(o) ? o.sessionId : null;
+  if (typeof sid !== 'string' || !GROK_SESSION_ID.test(sid)) return null;
+  const path = join(root, percentEncodePath(cwd), sid, 'updates.jsonl');
+  let data;
+  let fd;
+  try {
+    const rootReal = realpathSync(root);
+    const pathReal = realpathSync(path);
+    if (!isInsideRoot(rootReal, pathReal)) return null;
+    if (!statSync(pathReal).isFile()) return null;
+    fd = openSync(pathReal, 'r');
+    const buf = Buffer.alloc(Math.max(0, Math.min(cap, statSync(pathReal).size)));
+    let read = 0;
+    while (read < buf.length) {
+      const n = readSync(fd, buf, read, buf.length - read, read);
+      if (!n) break;
+      read += n;
+    }
+    data = buf.subarray(0, read).toString('utf8');
+  } catch {
+    return null;
+  } finally {
+    if (fd !== undefined) try { closeSync(fd); } catch {}
+  }
+  let count = 0, matched = false;
+  for (const rec of jsonLines(data)) {
+    if (!isObj(rec.params) || rec.params.sessionId !== sid) continue;
+    matched = true;
+    const u = rec.params.update;
+    if (!isObj(u)) continue;
+    if (u.sessionUpdate === 'hook_execution' && Array.isArray(u.runs)) {
+      for (const r of u.runs) if (isObj(r) && isObj(r.status) && r.status.blocked === true) count++;
+    } else if (u.sessionUpdate === 'tool_call_update' && u.status === 'failed') {
+      let blob = '';
+      try { blob = JSON.stringify(u.content ?? '').toLowerCase(); } catch {}
+      if (blob.includes('denied by permission policy') || blob.includes('hook denied')) count++;
+    }
+  }
+  return matched ? count : null;
+}
+
+export function countRefused(lane, out, err, opts) {
+  try {
+    if (lane === 'qwen') return refusedQwen(out);
+    if (lane === 'agy') return refusedAgy(out);
+    if (lane === 'codex') return refusedCodex(out, err);
+    if (lane === 'grok') return refusedGrok(out, opts);
+    return null; // hermes: no native refusal signal
+  } catch {
+    return null;
+  }
+}
+
+// Runs after the lane has exited, so adversarial output must not make it slow:
+// the input is bounded and every repeat is bounded. Redacted before any match
+// can clip a secret.
+const DENIAL_SCAN_BYTES = 256 * 1024;
+function firstDenialText(out, err) {
+  const blob = redact(`${String(out || '').slice(0, DENIAL_SCAN_BYTES)}\n${String(err || '').slice(0, DENIAL_SCAN_BYTES)}`);
+  for (const pat of [
+    /Permission denied for command\([^)\n]{0,200}\)\. Matches user-configured deny rule\./,
+    /Denied by permission policy:[^\n"]{0,160}/,
+    /Hook denied:[^\n"]{0,160}/,
+    /denied:[^\n"]{0,160}/,
+    /[Rr]ejected:[^\n"]{0,160}/,
+    /error=exec_command failed:[^\n]{0,160}/
+  ]) {
+    const m = pat.exec(blob);
+    if (m) return m[0].trim();
+  }
+  return null;
+}
+
+// A lane often paraphrases a refusal in its own words; a short head of its
+// answer names it when no exact denial phrase matches.
+function deliverableSnippet(out, limit = 160) {
+  let o;
+  try {
+    o = JSON.parse(out);
+  } catch {
+    return redact(String(out || '').slice(0, 4096)).trim().slice(0, limit) || null;
+  }
+  if (!isObj(o)) return null;
+  for (const k of ['text', 'response', 'result']) if (typeof o[k] === 'string' && o[k].trim()) return redact(o[k].slice(0, 4096)).trim().slice(0, limit);
+  return null;
+}
+
+const FIX = {
+  auth: 'set the credential the message above names (its environment variable, or the lane\'s own login command), then rerun',
+  quota: 'switch to another lane, or wait for the reset time if the message gave one',
+  rejected: 'correct the model id, flag or request the upstream message names',
+  refused: 'adjust the hook or deny rule named above, or give this lane the tool it needs',
+  cut_short: 'rerun once; if it recurs, run without --quiet and read the lane\'s stderr on the terminal',
+  empty: 'rerun once, or use another lane',
+  timeout: 'raise --timeout, or split the brief into smaller pieces',
+  no_output: 'rerun once; if it recurs, check that the lane runs on its own outside cli-run'
+};
+
+// A one-line problem naming the concrete cause, and a one-line fix. The caller
+// relays both. Returns { problem: null, fix: null } when there is nothing to say.
+export function problemAndFix(lane, cls, { out = '', err = '', detail = '', refused = null, authoritative = null } = {}) {
+  const cause = typeof authoritative === 'string' && authoritative.trim() ? redact(authoritative).trim() : null;
+  const d = typeof detail === 'string' && detail ? redact(detail) : null;
+  const tag = `cli-run[${lane}]`;
+  const denial = () => firstDenialText(out, err) || deliverableSnippet(out) || d;
+  switch (cls) {
+    case 'auth': return { problem: `${tag} auth: ${cause || d || 'missing or invalid credentials'}`, fix: FIX.auth };
+    case 'quota': return { problem: `${tag} quota: ${cause || d || 'rate limit or credits exhausted'}`, fix: FIX.quota };
+    case 'rejected': return { problem: `${tag} rejected: ${cause || d || 'the upstream rejected the request'}`, fix: FIX.rejected };
+    case 'refused': return { problem: `${tag} refused: ${denial() || 'a hook or deny rule blocked the call'}`, fix: FIX.refused };
+    case 'cut_short': return { problem: `${tag} cut short: ${d || 'no terminal success event, cause not identifiable'}`, fix: FIX.cut_short };
+    case 'empty': return { problem: `${tag} empty: ${d || 'completed but delivered nothing'}`, fix: FIX.empty };
+    case 'timeout': return { problem: `${tag} timeout: ${d || 'exceeded the wall clock'}`, fix: FIX.timeout };
+    case 'unavailable': return { problem: `${tag} unavailable: ${d || 'binary not found on PATH'}`, fix: `install the ${lane} CLI and put it on PATH, or enable it in lanes.json` };
+    case 'no_output': return { problem: `${tag} no output: the process wrote nothing to stdout or stderr`, fix: FIX.no_output };
+    case 'ok':
+      if (Number.isInteger(refused) && refused > 0) {
+        return { problem: `${tag} ok, but ${refused} call(s) were refused: ${denial() || 'a hook or deny rule blocked part of the call'}`, fix: FIX.refused };
+      }
+      return { problem: null, fix: null };
+    default:
+      return { problem: null, fix: null };
+  }
+}
+
+// Classify a judged run. `reason`/`detail` are the judge's own (not the
+// wrapper's decorated detail). A nonzero vendor exit is never ok, whatever
+// came back. Never throws.
+export function classifyRun(lane, { rc = 0, out = '', err = '', reason = '', detail = '', text = '', refusedOpts } = {}) {
+  out = typeof out === 'string' ? out : '';
+  err = typeof err === 'string' ? err : '';
+  detail = typeof detail === 'string' ? detail : '';
+  const refused = countRefused(lane, out, err, refusedOpts);
+  let cls;
+  try {
+    if (text && rc === 0) cls = 'ok';
+    else if (!out.trim() && !err.trim()) cls = 'no_output';
+    else {
+      const blob = authoritativeBlob(lane, out, err, detail, rc);
+      if (sigAuth(lane, blob)) cls = 'auth';
+      else if (sigQuota(lane, blob)) cls = 'quota';
+      else if (sigRejected(lane, blob)) cls = 'rejected';
+      else if (Number.isInteger(refused) && refused > 0) cls = 'refused';
+      else if (isCutShort(lane, reason, rc)) cls = 'cut_short';
+      else cls = 'empty';
+    }
+  } catch {
+    cls = 'cut_short';
+  }
+  return { cls, refused };
+}
+
 // --- the process boundary --------------------------------------------------
 // The lane runs DETACHED, so it leads its own process group. On timeout (or an
 // output-buffer overrun) the whole group is killed, not just the direct child:
@@ -694,7 +1124,9 @@ function usage(msg) {
   --model / --effort pin what a lane runs with, instead of letting it inherit its
   own config. Every lane takes --model; every lane except qwen takes --effort.
   Levels are the vendor's own (agy low|medium|high, hermes none|minimal|...): an
-  unknown level is rejected by the lane, and reported as that lane's exit code.
+  unknown level is rejected by the lane, and reported by class (codex: rejected, 16).
+  Exit codes: 0 ok, 10 empty, 11 no output, 12 timeout, 13 unavailable, 14 auth,
+  15 quota, 16 rejected, 17 refused, 18 cut short. Failures print a problem and a fix line.
   auto sizes per call: below 4,000 prompt characters is medium, otherwise high; a codex audit is always high. Auto never resolves above high.
   Pin them per lane instead of per call with "defaults" in bin/lanes.json.`);
   return USAGE;
@@ -703,7 +1135,7 @@ function usage(msg) {
 // Bounded, control-character-free head of vendor stderr for the terminal.
 // Never logged: provider text can echo whatever the prompt contained.
 function stderrHead(err, n = 300) {
-  const s = String(err || '').replace(/[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]/g, '').trim();
+  const s = redact(String(err || '').replace(/[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]/g, '')).trim();
   if (!s) return '';
   return s.length > n ? s.slice(0, n) + '…' : s;
 }
@@ -877,21 +1309,21 @@ export async function main(argv) {
     effort_truncated: sizing.truncated
   });
   const enabled = cfg === null ? null : cfg.enabled;
-  if (enabled === null) {
-    console.error('cli-run: lanes.json exists but is not a valid {"enabled": [...]} file; refusing every lane until it is fixed');
-    log({ ...base, verdict: 'unavailable', rc: UNAVAILABLE, reason: 'lanes_json_malformed' });
+  const unavailable = ({ msg, reason, fix }) => {
+    console.error('cli-run: ' + msg);
+    if (!opts.quiet) console.error(`cli-run fix: ${fix}`);
+    log({ ...base, verdict: 'unavailable', class: 'unavailable', rc: UNAVAILABLE, refused: null, reason });
     return UNAVAILABLE;
+  };
+  if (enabled === null) {
+    return unavailable({ msg: 'lanes.json exists but is not a valid {"enabled": [...]} file; refusing every lane until it is fixed', reason: 'lanes_json_malformed', fix: 'fix bin/lanes.json, or delete it to enable every lane' });
   }
   if (!enabled.includes(lane)) {
-    console.error(`cli-run: ${lane} is not enabled in lanes.json`);
-    log({ ...base, verdict: 'unavailable', rc: UNAVAILABLE, reason: 'disabled' });
-    return UNAVAILABLE;
+    return unavailable({ msg: `${lane} is not enabled in lanes.json`, reason: 'disabled', fix: `add "${lane}" to "enabled" in bin/lanes.json, or use an enabled lane` });
   }
   const binary = which(lane);
   if (!binary) {
-    console.error(`cli-run: ${lane} not found on PATH`);
-    log({ ...base, verdict: 'unavailable', rc: UNAVAILABLE, reason: 'unavailable' });
-    return UNAVAILABLE;
+    return unavailable({ msg: `${lane} not found on PATH`, reason: 'unavailable', fix: `install the ${lane} CLI and put it on PATH` });
   }
 
   if (route.effort === 'auto' && !opts.quiet) console.error(`cli-run: effort auto -> ${sizing.resolved} (${sizing.basis})`);
@@ -903,52 +1335,60 @@ export async function main(argv) {
     const r = await runBounded(cmd, opts.timeout);
     const out = r.stdout || '';
     const err = r.stderr || '';
-    let verdict, reason, detail, code, text = '';
+    let verdict, reason, detail, cls, text = '', refused = null;
     if (r.interrupted) {
-      verdict = 'interrupted'; reason = 'killed'; detail = `cli-run received ${r.interrupted}; the lane's process group was killed`; code = 128 + (r.interrupted === 'SIGINT' ? 2 : 15);
+      verdict = 'interrupted'; reason = 'killed'; detail = `cli-run received ${r.interrupted}; the lane's process group was killed`; cls = 'interrupted';
     } else if (r.timedOut) {
-      verdict = 'timeout'; reason = 'timeout'; detail = `exceeded ${opts.timeout}s; process group killed`; code = TIMEOUT;
+      verdict = 'timeout'; reason = 'timeout'; detail = `exceeded ${opts.timeout}s; process group killed`; cls = 'timeout';
     } else if (r.overrun) {
-      verdict = 'no_deliverable'; reason = 'no_output'; detail = 'output exceeded the 16 MiB buffer; process group killed'; code = NO_DELIVERABLE;
+      verdict = 'no_deliverable'; reason = 'no_output'; detail = 'output exceeded the 16 MiB buffer; process group killed'; cls = 'cut_short';
     } else if (r.error) {
-      verdict = 'unavailable'; reason = 'unavailable'; detail = r.error.message; code = UNAVAILABLE;
+      verdict = 'unavailable'; reason = 'unavailable'; detail = r.error.message; cls = 'unavailable';
     } else if (r.signal || r.status === null) {
       // A lane killed by a signal has no honest exit status. Whatever it printed
       // before dying is not a deliverable; a null status must never become exit 0.
-      verdict = 'killed'; reason = 'killed'; detail = `lane killed by ${r.signal || 'unknown signal'}`; code = NO_DELIVERABLE;
+      verdict = 'killed'; reason = 'killed'; detail = `lane killed by ${r.signal || 'unknown signal'}`; cls = 'cut_short';
     } else {
-      const j = judge(lane, r.status, out, err, outFile);
-      text = j.text || '';
+      const j = safeJudge(lane, r.status, out, err, outFile);
       reason = j.reason;
       detail = j.detail;
+      ({ cls, refused } = classifyRun(lane, { rc: r.status, out, err, reason: j.reason, detail: j.detail, text: j.text || '' }));
       if (r.status !== 0) {
         // A nonzero vendor exit is a failure on the vendor's own terms, whether or
-        // not something parseable came back. Pass the code through, keep the
-        // verdict honest, and show what the vendor said on stderr.
-        verdict = 'exit_nonzero'; code = r.status;
+        // not something parseable came back. The class names why; the vendor's
+        // own code is kept as cli_rc, and its stderr head shows on the terminal.
+        verdict = 'exit_nonzero';
         if (reason === 'ok') reason = 'exit_nonzero';
         const head = stderrHead(err);
         detail = `lane exited ${r.status}` + (head ? `; stderr: ${head}` : '') + (j.reason !== 'ok' ? `; ${j.detail}` : '');
-      } else if (text) {
-        const unmet = checkContracts(opts, text, before);
+      } else if (cls === 'ok') {
+        const unmet = checkContracts(opts, j.text, before);
         if (unmet) {
-          verdict = 'no_deliverable'; reason = 'contract_unmet'; detail = unmet; code = NO_DELIVERABLE;
+          verdict = 'no_deliverable'; reason = 'contract_unmet'; detail = unmet; cls = 'empty';
         } else {
-          verdict = 'ok'; code = OK;
+          verdict = 'ok'; text = j.text;
         }
-      } else if (!out.trim() && !err.trim()) {
-        verdict = 'no_output'; reason = 'no_output'; code = NO_OUTPUT;
+      } else if (cls === 'no_output') {
+        verdict = 'no_output'; reason = 'no_output';
       } else {
-        verdict = 'no_deliverable'; code = NO_DELIVERABLE;
+        verdict = 'no_deliverable';
         const head = stderrHead(err);
         if (head) detail += `; stderr: ${head}`;
       }
     }
+    const code = cls === 'interrupted' ? 128 + (r.interrupted === 'SIGINT' ? 2 : 15) : CLASS_CODES[cls];
     if (text && code === OK) process.stdout.write(text + '\n');
     const routeNote = route.model || route.effort ? `${route.model || 'lane default'}/${route.effort || 'lane default'}` : 'lane default';
-    if (!opts.quiet) console.error(`cli-run[${lane}] ${verdict} rc=${code} ${r.seconds.toFixed(1)}s raw=${r.outBytes || 0}B route=${routeNote} :: ${detail}`);
-    // Durable log: fixed reason code and structural numbers only.
-    log({ ...base, verdict, rc: code, cli_rc: r.status, signal: r.signal || null, seconds: Math.round(r.seconds * 100) / 100, raw_bytes: r.outBytes || 0, deliverable_bytes: Buffer.byteLength(text), reason: REASONS.has(reason) ? reason : 'unknown' });
+    if (!opts.quiet) {
+      console.error(`cli-run[${lane}] ${verdict} rc=${code} class=${cls} refused=${refused === null ? 'null' : refused} ${r.seconds.toFixed(1)}s raw=${r.outBytes || 0}B route=${routeNote} :: ${redact(detail)}`);
+      let authoritative = null;
+      if (lane === 'codex' && (cls === 'quota' || cls === 'rejected')) authoritative = codexPrimaryError(out);
+      const pf = problemAndFix(lane, cls, { out, err, detail, refused, authoritative });
+      if (pf.problem) console.error('cli-run problem: ' + redact(pf.problem).slice(0, 600));
+      if (pf.fix) console.error('cli-run fix: ' + pf.fix);
+    }
+    // Durable log: fixed reason code, fixed class and structural numbers only.
+    log({ ...base, verdict, class: cls in CLASS_CODES || cls === 'interrupted' ? cls : 'unknown', rc: code, cli_rc: r.status, signal: r.signal || null, refused: Number.isInteger(refused) ? refused : null, seconds: Math.round(r.seconds * 100) / 100, raw_bytes: r.outBytes || 0, deliverable_bytes: Buffer.byteLength(text), reason: REASONS.has(reason) ? reason : 'unknown' });
     return code;
   } finally {
     rmSync(tmp, { recursive: true, force: true });

package/docs/audit-brief.md

@@ -126,3 +126,23 @@ Three findings reproduced against the 0.1.15 branch before it shipped, none of t
 - **Not yet attacked for real.** `windowsSpawnPlan`, `resolveCmdShim` and the escaping functions are unit-tested (pure string logic, runs on every CI host) and the resolved-shim path is exercised end to end on `windows-latest` through the fake-lane fixtures in `test/cli.test.js` (installed as a real npm-style `.cmd` shim). A lane can no longer reach `cmd.exe` at all (a unit test pins the refusal). The opt-in `cmd.exe` branch, used only by the installer's own `npm install -g`, is not exercised end to end through a live Windows process in this suite; its escaping is proven by exact-string unit tests only, and its arguments never include user text.
 - **A platform limit found the same way, unrelated to the spawn path itself: Windows has no OS-level signals at all.** `cli-run.mjs`'s graceful shutdown (`process.on('SIGTERM', ...)`, kill the lane's process group, then exit 143/130) is a POSIX guarantee only: `ChildProcess.kill(sig)` on Windows calls `TerminateProcess()` unconditionally for SIGTERM AND SIGINT alike, giving the target process no chance to run any handler at all, proven on `windows-latest` CI (the wrapper died as `{code: null, signal: sig}` for both; a hypothesis that SIGINT gets a real, catchable console-control event on Windows was tried first and measured false in this exact scenario, not assumed). `test/cli.test.js`'s `#13` now expects an unhandled termination for either signal on win32, and the original graceful-exit assertion elsewhere.
 - **Two narrow, individually-verified Windows skips remain, neither in the spawn path itself.** (1) A lane dying mid-run from a real POSIX signal cannot be reproduced on win32: a real Windows lane is a plain `node <script>` process, so it cannot die "by signal" any more than the product being tested can, and the only way a test fixture can even simulate one (a nested `sh -c "...; kill -TERM $$"`) puts an extra node process between cli-run.mjs and the dying shell, so cli-run.mjs observes only that node's translated exit code (measured: MSYS bash's self-kill status leaks through as a plain nonzero exit code, 3840, which this tool already handles honestly via `exit_nonzero`). (2) `weekly-audit.sh`'s watchdog (`bounded()`/`killtree()`, `pgrep -P` plus killing a backgrounded subshell's tree) relies on real bash job control this script only ever runs under on the Ubuntu box it targets; actually executing it against a genuinely hanging stub under Git Bash's job-control emulation hung past a 20s outer timeout on `windows-latest` CI, a known class of MSYS/Cygwin limitation (a `kill -KILL` not reliably reaching the underlying Windows process tree of a backgrounded subshell), not a defect in the generated script, which still renders and syntax-checks correctly.
+
+## New in 0.1.23: failure classes in cli-run
+
+`bin/cli-run.mjs` now puts every run in a closed failure class that owns the exit code (`auth` 14, `quota` 15, `rejected` 16, `refused` 17, `cut_short` 18, beside `ok` 0, `empty` 10, `no_output` 11, `timeout` 12, `unavailable` 13), counts refused tool calls per lane (including a read of grok's session transcript, whose `sessionId` comes from lane stdout), and prints redacted problem and fix lines. The durable log gains only `class` and `refused`. Threat model: a false exit 0, provider text or a secret reaching the log or the terminal, a transcript read escaping the sessions root, misclassification that sends a user the wrong way, and anything that throws or stalls.
+
+### Round 1 (pre-release, GPT-6 Astra at xhigh), fixed before shipping
+
+Every finding was reproduced as a failing case in `test/classify.test.js` against the unfixed code (all seven red), then fixed; the suite is green after. One round, by rule; the regression cases verify the fixes.
+
+| # | Sev | Finding | Fix | Test |
+|---|---|---|---|---|
+| 1 | HIGH | qwen's display detail clipped an error message at 120 characters before redaction, so a long JSON password lost its closing quote and printed its prefix | redact before every clip: judge detail, API-error text, denial text, deliverable snippet, problem cause | R1 |
+| 2 | HIGH | the JSON credential pattern ended at an escaped quote and missed unterminated values | match JSON string escapes, run an unterminated value to the end, and accept a JSON body escaped inside a string | R2 |
+| 3 | HIGH | `firstDenialText` scanned 16 MiB with an unbounded `[^)]*`: 1 MiB of unclosed `Permission denied for command(` took 15.6 s after the lane had exited | scan at most 256 KiB of stdout and of stderr, with bounded repeats | R3 |
+| 4 | MEDIUM | transcript containment was a string prefix with both separators, so on POSIX a directory literally named `sessions\outside` passed | `isInsideRoot()` via `path.relative`, rejecting `..`, absolute and cross-drive results; tested with `path.posix` and `path.win32` on every OS | R4 |
+| 5 | MEDIUM | qwen signals were searched in the clipped display detail, so a quota error after 120 characters read as `empty`, and a model name in the detail could fake a signal | classify on `qwenErrorText()`: the terminal event's full error and an API-error result only | R5 |
+| 6 | MEDIUM | hermes fell back to stdout when stderr was empty, so an answer saying "no rate limit" read as `quota` | stderr only | R6 |
+| 7 | MEDIUM | a nonzero vendor exit with a recognised empty shape (agy exit 7, `SUCCESS`, empty response) returned `empty`, contradicting the documented `cut_short` fallback | an unexplained nonzero exit is `cut_short`, except hermes' own exit 2 | R7 |
+
+Reported clean by the same audit: no false exit 0 across 675 malformed probes and five lanes; no provider text or secret fragment in probed log records; the real codex 0.153.4 fixture's non-fatal error item still succeeds; grok traversal, symlink, FIFO, session-match and read-cap guards; timeout, interrupt, overrun, killed lane, JSON contract, `--quiet` and the doctor canary. Not verified by it: native Windows execution, live vendors, filesystem races.

package/docs/catalog.md

@@ -63,6 +63,7 @@ Generated from `src/catalog.js`. Do not hand-edit; `npm run gen:catalog` rewrite
 - **cli-run lane:** yes
 - **Plans:**
   - SuperGrok (base headroom, checked 2026-09-12): https://x.ai/news/grok-build-cli
+  - SuperGrok Plus (high headroom, checked 2026-09-12): https://x.ai/pricing
   - X Premium Plus (base headroom, checked 2026-09-12): https://x.ai/news/grok-build-cli
 - **Built against:** 1.0.5
 

package/docs/part-2-intermediate.md

@@ -28,7 +28,7 @@ One driver, no second AI in the mix: the orchestrator invokes the CLIs; it never
 
 ## 3. Exit 0 is a lie on every lane
 
-Every agent CLI can report success and deliver nothing. `bin/cli-run.mjs` builds the right invocation per lane, reads that lane's **native** terminal event, and exits `10` when a run produced no deliverable, `12` on timeout, `13` when the lane is missing. Byte count is not a check either; a run can emit hundreds of kilobytes and no conclusion. One lane's own success flags lie outright (an upstream 400 reported as success), so its judge reads the two honest signals instead.
+Every agent CLI can report success and deliver nothing. `bin/cli-run.mjs` builds the right invocation per lane, reads that lane's **native** terminal event, and exits `10` when a run produced no deliverable, `12` on timeout, `13` when the lane is missing, and `14` to `18` when the lane's own error says why (auth, quota, rejected, refused, cut short), so a missing API key is never blamed on the model. Byte count is not a check either; a run can emit hundreds of kilobytes and no conclusion. One lane's own success flags lie outright (an upstream 400 reported as success), so its judge reads the two honest signals instead.
 
 Every call goes through it. "This lane is flaky" becomes a query over its log instead of an argument. `node bin/cli-run.mjs --doctor` is the first thing to run after install: enabled lanes, binaries on PATH, the route each lane is pinned to, and with `--run` a one-word canary per lane.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "model-orchestrator",
-  "version": "0.1.21",
+  "version": "0.1.23",
   "description": "Model orchestrator for AI coding agents and LLMs: Claude Code, Codex, Gemini, Grok, Qwen, Ollama. Routing rules tell your agent which model, subagent or CLI to use for each task, so small work goes to cheap tiers and fewer tokens go to frontier models. One installer, plus a CLI runner that logs every route.",
   "type": "module",
   "bin": {

package/src/catalog.js

@@ -150,6 +150,7 @@ export const AIS = [
     cliRun: true
     , plans: [
       { id: 'supergrok', name: 'SuperGrok', headroom: 'base', source: 'https://x.ai/news/grok-build-cli', checked: '2026-09-12' },
+      { id: 'supergrok-plus', name: 'SuperGrok Plus', headroom: 'high', source: 'https://x.ai/pricing', checked: '2026-09-12' },
       { id: 'x-premium-plus', name: 'X Premium Plus', headroom: 'base', source: 'https://x.ai/news/grok-build-cli', checked: '2026-09-12' }
     ]
   },

package/templates/advanced/vm/jobs/weekly-audit.sh

@@ -7,7 +7,7 @@
 #   - the previous successful report is NEVER truncated: output goes to a temp file and is
 #     renamed into place only on a clean exit; failed output is kept beside it for diagnosis
 #   - the lane runs with the strongest boundary it offers ({{AUDIT_LANE_BOUNDARY_NOTE}})
-#   - rc 10/12/13 from cli-run means no report was produced; the timer's journal shows it
+#   - any nonzero rc from cli-run (10 to 18) means no report was produced; the timer's journal shows it
 set -uo pipefail
 INSTALL_DIR={{INSTALL_DIR_SH}}
 AUDIT_LANE="{{AUDIT_LANE}}"

package/templates/intermediate/CLI-RUN.md

@@ -56,16 +56,40 @@ qwen is the lane whose own success flags lie: an upstream 400 comes back as exit
 
 ## Exit codes
 
-| Code | Meaning |
-|---|---|
-| 0 | structurally accepted non-empty response, every `--expect-*` contract met |
-| 10 | ran and produced no deliverable, or a contract was unmet, or the lane was killed by a signal, or output overran the 16 MiB buffer |
-| 11 | no output at all |
-| 12 | timed out; the lane and every descendant in its process group were killed |
-| 13 | lane unavailable: binary missing, disabled in `lanes.json`, or `lanes.json` malformed |
-| 130 / 143 | cli-run itself received SIGINT / SIGTERM; the lane's process group was killed first, then the temp dir removed |
-| 2 | usage error in cli-run itself |
-| N | the lane exited N != 0: passed through unchanged, verdict `exit_nonzero`, even when parseable text came back. The bounded head of the lane's stderr is shown on your terminal so an auth failure reads as one |
+| Code | Class | Meaning | What to do |
+|---|---|---|---|
+| 0 | `ok` | structurally accepted non-empty response, every `--expect-*` contract met | use it; if `refused=N` is above 0, read the problem line |
+| 10 | `empty` | ran and delivered nothing, or a contract was unmet | rerun once, or use another lane |
+| 11 | `no_output` | no output at all | rerun once; check the lane runs on its own |
+| 12 | `timeout` | timed out; the lane and every descendant in its process group were killed | raise `--timeout` or split the brief |
+| 13 | `unavailable` | binary missing, disabled in `lanes.json`, or `lanes.json` malformed | install or enable the lane |
+| 14 | `auth` | the lane's own error says a credential is missing or it is not logged in | set the credential; retrying cannot help |
+| 15 | `quota` | the lane's own error says usage limit, credits or rate limit | switch lanes or wait for the reset |
+| 16 | `rejected` | the upstream rejected the request: an unknown model id, a malformed request | fix the id, flag or request it names |
+| 17 | `refused` | no deliverable, and the lane reports tool calls a hook or deny rule blocked | adjust the rule, or give the lane the tool |
+| 18 | `cut_short` | no trustworthy finish: a missing or non-success terminal event, a lane killed by a signal, output past the 16 MiB buffer, or a nonzero vendor exit nothing above explains (hermes' own exit 2, bad args or an empty response, stays `empty`) | rerun once, then read the lane's stderr |
+| 130 / 143 | `interrupted` | cli-run itself received SIGINT / SIGTERM; the lane's process group was killed first, then the temp dir removed | |
+| 2 | | usage error in cli-run itself | |
+
+A nonzero vendor exit is never `ok`, even when parseable text came back; the vendor's own code is kept in the log as `cli_rc`, and the bounded head of its stderr is shown on your terminal.
+
+## Why a run failed: the class
+
+Exit 10 used to cover causes that need opposite responses. A missing API key is not a model fault, and retrying a spent quota cannot help. So every run lands in exactly one class, and on the terminal every failure (and every `ok` with refused calls) gets two more lines:
+
+```
+cli-run[qwen] exit_nonzero rc=14 class=auth refused=0 0.8s raw=212B route=lane default :: lane exited 1; subtype="error_during_execution": Missing API key ...
+cli-run problem: cli-run[qwen] auth: Missing API key ...
+cli-run fix: set the credential the message above names (its environment variable, or the lane's own login command), then rerun
+```
+
+A calling agent relays both lines and asks before fixing anything.
+
+- **Signals come from each lane's authoritative error fields only.** codex: its own `error`, `turn.failed` and error-item events plus stderr. agy: the terminal result's status and error plus stderr. qwen: the terminal event's full error text (never the clipped display line) plus stderr. hermes: stderr on a nonzero exit, never its stdout. Never the model's prose, so an answer that explains what "rate limit" means is not a quota failure. grok and hermes have no native auth signal, and none is invented.
+- **Precedence** when several are present: `auth`, `quota`, `rejected`, `refused`, `cut_short`, `empty`. A missing credential explains everything downstream of it.
+- **`refused`** counts tool calls a hook or deny rule blocked: qwen's `permission_denials`, agy's deny-rule `TOOL_ERROR` steps, one per codex router `Rejected(` line on stderr, and grok's session transcript (`~/.grok/sessions/<cwd>/<sessionId>/updates.jsonl`: hook runs with `blocked`, and "Denied by permission policy"). grok's `sessionId` comes from lane output, so it must match `^[A-Za-z0-9-]{8,64}$`, the resolved path must stay inside the sessions root, every counted line must name that session, and the read stops at 5 MiB. `null` means the lane gave no readable signal (hermes always), which is an unknown, never 0.
+- **Refused with a deliverable is still exit 0.** The deliverable exists; `refused=N` and the problem and fix lines say what was blocked.
+- **Redacted.** The status, problem and stderr lines pass through one redaction pass before they are printed: JSON credential keys (escaped quotes, unterminated values and JSON escaped inside a string included), `Authorization:` values of any scheme, bearer values, URL query credentials, and the `sk-`, `xai-`, `ghp_` and `AIza` key prefixes. Redaction runs before any clipping, so a long token is never cut into an unrecognisable fragment. Denial text is searched in at most the first 256 KiB of stdout and of stderr, with bounded patterns, so hostile output cannot stall the wrapper after the lane has exited. None of these lines is ever logged.
 
 ## The route: which model, and how hard it thinks
 
@@ -97,7 +121,7 @@ A flag beats `defaults`; `defaults` beats nothing. Each vendor spells these diff
 
 Three rules that keep this honest:
 
-- **A level `cli-run` does not recognise is not rejected here.** Levels are the vendor's, they change, and guessing the valid set would date this tool. An unknown level is refused by the lane and surfaces as that lane's own exit code and stderr.
+- **A level `cli-run` does not recognise is not rejected here.** Levels are the vendor's, they change, and guessing the valid set would date this tool. An unknown level is refused by the lane and surfaces as a class (codex reports it as `rejected`, exit 16; a lane with no rejection signal as `cut_short`, exit 18), with the lane's own exit code in the log as `cli_rc` and its stderr on your terminal.
 - **`--effort` on qwen is a usage error, not a silent drop.** A flag that vanishes leaves you believing a route that never ran.
 - **`--effort auto` is bounded.** It uses prompt size, or an audit's changed-file evidence, and resolves only medium or high. An audit is always high. Auto is a heuristic, not a measurement: name `xhigh` explicitly for security-critical or irreversible work.
 - **Values are charset-bounded** (letters, digits, and `. _ : @ / + -`, no leading dash, 64 characters). A model id becomes an argv element and, on codex, part of a TOML value; bounding it is what stops either from being escaped.
@@ -108,7 +132,7 @@ Three rules that keep this honest:
 
 ## Log
 
-`~/.ai-orchestrator/cli-run.log.jsonl`, one line per run: lane, verdict, rc, the lane's own exit code, signal, seconds, raw bytes, deliverable bytes, a 12-hex sha256 prefix of the prompt and its length, the route (`model_requested`, `effort_requested`, and `model_source` / `effort_source`, each one of `flag`, `lanes.json` or `lane_default`), auto-sizing evidence (`effort_resolved`, `effort_basis`, `effort_scope`), and `reason`: one of a fixed set of codes (`ok`, `not_json`, `bad_stop_reason`, `empty_text`, `no_terminal_event`, `bad_status`, `api_error_in_result`, `total_errors`, `contract_unmet`, `exit_nonzero`, `timeout`, `killed`, `disabled`, `lanes_json_malformed`, ...). `effort_basis` is exactly one of `explicit`, `prompt_chars`, `audit_floor`, or `none`. Never the prompt text, never a provider-supplied value, never free text: a value the log does not recognise is written as `unknown`. The human-readable detail, which may quote the provider, goes to your terminal only (and nowhere with `--quiet`). "This lane is flaky" becomes a query instead of an argument, and so does "we route audits at high effort".
+`~/.ai-orchestrator/cli-run.log.jsonl`, one line per run: lane, verdict, `class` (one of the classes above), rc, the lane's own exit code (`cli_rc`), signal, `refused` (an integer, or `null` when the lane gives no signal), seconds, raw bytes, deliverable bytes, a 12-hex sha256 prefix of the prompt and its length, the route (`model_requested`, `effort_requested`, and `model_source` / `effort_source`, each one of `flag`, `lanes.json` or `lane_default`), auto-sizing evidence (`effort_resolved`, `effort_basis`, `effort_scope`), and `reason`: one of a fixed set of codes (`ok`, `not_json`, `bad_stop_reason`, `empty_text`, `no_terminal_event`, `bad_status`, `api_error_in_result`, `total_errors`, `contract_unmet`, `exit_nonzero`, `timeout`, `killed`, `disabled`, `lanes_json_malformed`, ...). `effort_basis` is exactly one of `explicit`, `prompt_chars`, `audit_floor`, or `none`. Never the prompt text, never a provider-supplied value, never free text: a value the log does not recognise is written as `unknown`. The human-readable detail, which may quote the provider, goes to your terminal only (and nowhere with `--quiet`). "This lane is flaky" becomes a query instead of an argument, and so does "we route audits at high effort".
 
 The log records what was **requested**, on every record including a run refused before the lane started. It does not record an actual. Reporting is inconsistent: grok returns a `modelUsage` block naming a model, the other four lanes return nothing of the kind, so an `actual` field would be populated for one lane and empty for four. It would also be a provider-supplied string, and this log holds fixed codes and bounded caller-supplied values only. `model_source: "lane_default"` is the honest way to say this run inherited something invisible from here.
 
@@ -122,7 +146,7 @@ Absent: every lane enabled, nothing pinned. Present but malformed or unreadable:
 
 ## A killed lane is not a deliverable
 
-A lane that dies by signal has no honest exit status. Whatever it printed first is discarded; the run reports `killed` with exit 10.
+A lane that dies by signal has no honest exit status. Whatever it printed first is discarded; the run reports `killed`, class `cut_short`, exit 18.
 
 ## Interrupting cli-run kills the lane too