catherd-cli 0.2.1 → 1.1.0

package/plugin/skills/catherd-setup/SKILL.md

@@ -1,13 +1,13 @@
 ---
 name: catherd-setup
-description: Use when the user wants to tune catherd — which models and efforts each role may use, cost versus speed, which subscriptions to lean on, harness isolation — /catherd-setup, "set up catherd", "tune my catherd profile". Not for running a task; that is the catherd skill.
+description: Use when the user wants to tune catherd — which models and efforts each role may use, how much each role may touch, cost versus speed, which subscriptions to lean on, failover, budget, harness isolation — /catherd-setup, "set up catherd", "tune my catherd profile". Not for running a task; that is the catherd skill.
 ---
 
 # catherd setup
 
-You tune the user's catherd profile in conversation. A profile says, per role, which models and efforts it may run on; whether routing favours cost or speed; whether each vendor harness runs with the user's customizations or isolated; how many heavy commands run at once; and when to push.
+You tune the user's catherd profile in conversation. A profile says, per role, which rungs it may run on (`backend:model#effort`, in ladder order) and what it may touch (its access mode); whether routing favours cost or speed; how each backend is billed; whether Jev picks the rung; which rung stands in when a backend hits its usage limit; the run budget and timeouts; whether each vendor harness runs with the user's customizations or isolated; how many heavy commands run at once; and when to push.
 
-**You never edit a file by hand.** `profile_set` is the only writer, so this conversation and the TUI (`bunx catherd-cli`) cannot drift apart.
+**You never edit a file by hand.** `profile_set` is the only writer, so this conversation, the `catherd profile` commands and the TUI cannot drift apart.
 
 ## Rules
 
@@ -21,45 +21,55 @@ You tune the user's catherd profile in conversation. A profile says, per role, w
 Ask these, one at a time, each with its recommended answer:
 
 1. **Their order of speed, cost and quality.** Recommend cost first, the default `objective`: catherd climbs a rung when a cheap one cannot do the work, so the lanes that need speed get it anyway, and the reviewer and the verifier hold quality either way.
-2. **The subscriptions they hold:** a ChatGPT plan (Codex), a Claude plan, OpenRouter or other opencode credit. Recommend leaning on subscriptions before metered spend, and on Claude last among the workers, since it spends the same quota as this conversation.
+2. **The subscriptions they hold:** a ChatGPT plan (Codex), a Claude plan, OpenCode Go, Zen credit or API keys. They set `billing` per key (`codex`, `claude`, `claude-code`, `opencode-go`, `opencode`): `chatgpt-plan`, `claude-plan`, `subscription` or `metered`. Recommend leaning on subscriptions before metered spend, and on Claude last among the workers, since it spends the same quota as this conversation.
 3. **The kind of work they orchestrate:** front-end screens, back-end services, terminal and ops work, docs. Recommend from their own runs when `runs_summary` has any.
 
 ## 2. Read the facts before proposing
 
 In one message, call:
 
-- `catalog_query({ role: "<role>" })` for each role you will discuss: what their installed backends can run, with capabilities and scores. A model whose `installed` is false is not offered;
-- `runs_summary({})`: how each rung has done on their own runs (runs, refusals, climbs, time) and the harness cost line;
-- `profile_get()`: where they stand now.
+- `catalog_query({ role: "<role>" })` for each role you will discuss: the models that can fill it, their scored rungs, any "treat like", each rung's cost under their billing, and whether their backend's last listing offers it (`listed: false` means their account does not);
+- `runs_summary({})`: how each rung has done on their own runs (runs, refusals, climbs, time) and the harness cost line, for claude-code and opencode only: Codex reports no per-request input, so it has no harness figure;
+- `profile_get({ repo })`: where they stand now, with each role's `access` and `enforcement`. Pass the user's repo: without a name, the profile tools act on the profile this repo runs on (the one bound to it, else the active one), which `here` names; `active` is the global active profile.
 
 ## 3. Propose one decision at a time
 
 Go through these in order, and skip any the user does not care about:
 
-1. `objective`;
-2. the worker's ladder: its enabled models and efforts, and its `defaultRung`;
-3. the reviewer and the UI reviewer;
-4. the architect and the verifier, which spend Claude quota;
-5. the writer, the researcher and the artist;
-6. harness isolation, per harness (below);
-7. `lock.heavy` and `notify`.
+1. `objective`, and `jev.use` (`auto` asks Jev when a key exists; `off` routes on each lane's `Kind:` and `Difficulty:` lines);
+2. `billing`, from step 1;
+3. the worker's `rungs` and its `defaultRung`;
+4. the reviewer and the UI reviewer;
+5. the architect and the verifier, which spend Claude quota. A `claude:` rung runs as a native subagent in this session; a `claude-code:` rung runs headless through `dispatch`. Recommend native for these two;
+6. the writer, the researcher and the artist;
+7. each role's `access` (below);
+8. `failover` (below);
+9. `budget`, `timeouts` and `preflight.confirm`;
+10. harness isolation, per harness (below);
+11. `lock.heavy` and `notify`.
 
 Each proposal has three parts: the change, a worked example from their facts, and the tradeoff in their terms. For instance: "Luna high on build lanes: about 5 min slower than Sol medium, no Claude quota, climbs on 1 in 5 of your runs so far."
 
-- Offer only the models `catalog_query` lists as capable for the role.
-- An unscored model can be enabled only once it has a "treat like <scored model#effort>". That lives in the catalog override, which the TUI edits: send the user to `bunx catherd-cli` for it, then come back.
+- Offer only the rungs `catalog_query` lists as capable for the role, written `backend:model#effort`.
+- An unscored model can be enabled only once it has a "treat like <scored rung>". There is no tool for that: give the user the command to run in a terminal, `catherd catalog treat-like <rung> <scored rung>`, then come back and propose again.
 
-**Harness isolation.** For each harness they use (`codex`, `opencode`), offer `harness.<name>.isolated` with its harness line from `runs_summary` and this tradeoff: "native keeps your hooks, skills and AGENTS.md; isolated saves ~N tokens per run, but the role loses them." Recommend native. When they have no isolated runs yet, the number is the median first-turn input of their native runs: say that isolation would save some part of it, not all of it. When there are no runs at all, say there is no number yet, and recommend native until there is.
+**Access.** Each role runs `read-only`, `workspace-write` or `full`. The defaults: architect, reviewer and researcher `read-only`; worker, writer and artist `workspace-write`; verifier and UI reviewer `full`. `profile_get` says per role whether its backend holds it to that mode (`enforced`: Codex's sandbox) or only asks (`advisory`: claude-code, opencode and native subagents, where the model can still reach past it). Recommend the defaults; when the user wants a role tighter or looser, say what it can no longer do (a read-only reviewer on claude-code or opencode has no shell, so it cannot run `git diff`) and that `profile_validate` will warn about it. A `workspace-write` role also writes catherd's lock dir and the temp dir, reaches the network, binds loopback ports and talks to a local Docker socket, so it runs its own installs and tests; `roles.<role>.network: false` takes the network, loopback and Docker away from that role (Codex enforces it; claude-code drops its web tools; opencode has no sandbox to enforce it). `catherd doctor` probes each of these per backend.
+
+**Failover.** `failover` maps a rung to its stand-in when that rung's backend hits a usage limit. A stand-in must be scored and on another quota (Go and Zen bill apart; native `claude` and `claude-code` share the Claude plan). The default profile fails a Codex rung over only to a stand-in that clears the same bars at least as well: Go's GPT-6 Luna for Luna high, and Kimi K3 (its scores borrowed from Sol medium) for Sol medium. Sol high and xhigh have none, so a limit there pauses the lane rather than dropping it a tier. Say so, and offer to change it when they have no Go subscription. `null` removes an entry.
+
+**Budget and timeouts.** `budget` (`minutes`, `tokens`, `usd`) is a soft cap: from 80 % routing starts at the cheapest rung that clears the bar, and at 100 % no new role starts. `timeouts.idleMin` (15) stops a role that has gone quiet, `timeouts.wallMin` (90) one that runs too long. `preflight.confirm: true` makes `preflight` show its commands for the user to approve first.
+
+**Harness isolation.** For each harness they use (`codex`, `claude-code`, `opencode`), offer `harness.<name>.isolated` with its harness line from `runs_summary` (Codex has none: it reports no per-request input, so say there is no figure for it instead of offering one) and this tradeoff: "native keeps your hooks, skills and AGENTS.md; isolated saves ~N tokens per run, but the role loses them." Recommend native. When they have no isolated runs yet, the number is the median first-turn input of their native runs: say that isolation would save some part of it, not all of it. When there are no runs at all, say there is no number yet, and recommend native until there is.
 
 ## 4. Write it
 
-- Call `profile_set({ patch })` with only what changed; pass `name` only to edit a profile other than the active one. It validates before it writes:
-  - `saved: false` comes with `errors`. Explain each in their terms, fix the patch, and propose again.
-  - `saved: true` comes with the `diff`. Read it back to the user, one line per change.
-- Then call `profile_validate()`. It checks that every enabled role has a capable model, that the worker's ladder is non-empty for every kind, and that no unscored model is enabled without its "treat like".
+- Call `profile_set({ repo, patch })` with only what changed; pass `name` only to edit a profile other than the one this repo runs on (a name that does not exist yet starts a new profile from the default one). Lists (`rungs`, `notify`) replace, maps (`billing`, `failover`, `budget`) merge, and `null` removes a key. A key it does not know is refused with `E_INPUT_INVALID`. It validates before it writes:
+  - `saved: false` comes with `errors`, each with a `path`, a `message` and often a `fix`. Explain each in their terms, fix the patch, and propose again.
+  - `saved: true` comes with the `diff` (`path`, `before`, `after`) and any `warnings`. Read the diff back to the user, one line per change, and each warning with it.
+- Then call `profile_validate({ repo })`. `errors` block a save: the worker disabled, an enabled role with no usable rung, an unscored rung without a "treat like", a failover stand-in unscored or on the same quota, a backend catherd cannot run. `warnings` do not: an access mode other than the role's default, a model the backend's listing lacks, a stand-in that never runs, a stand-in that scores below its rung ("downgrade"), a Claude stand-in while another plan could stand in ("spends Claude quota"), a ladder whose rung scores below the one before it ("the ladder goes down"). Each has a `fix`; offer it.
 
 ## 5. Say what applies when
 
-- Codex and opencode changes, isolation included, apply at the next dispatch, even in a run already under way.
-- A Claude agent listed in `new_session_needed_for` applies from the next Claude Code session: Claude Code registers agent files when a session starts. Other Claude changes apply now.
-- Editing a profile that is not the active one writes its agent files but links none; they apply once it becomes active.
+- Codex, claude-code and opencode changes, access and isolation included, apply at the next dispatch, even in a run already under way.
+- An agent listed in `newSessionNeededFor` applies from the next Claude Code session: Claude Code reads agent files when a session starts. Other Claude changes apply now.
+- Editing a profile that is not the active one writes its agent files but links none; they apply once it becomes active (`catherd profile use <name>`), or in a repo it is bound to (`catherd profile use <name> --repo`).

package/src/adapters/access.ts

@@ -0,0 +1,63 @@
+import { existsSync, mkdtempSync, realpathSync, rmSync } from "node:fs";
+import { homedir, tmpdir } from "node:os";
+import { join } from "node:path";
+import { locksDir } from "../infra/paths.ts";
+import type { AccessShell } from "./backend.ts";
+import { runCli } from "./cli.ts";
+
+/** How long one access probe may run: an HTTPS fetch through a slow proxy included. */
+export const probeShell = { timeoutMs: 20_000 };
+
+/**
+ * A probe shell from a fresh scratch dir: `prefix` is the argv before `sh -c <script> _ <args…>` (empty for
+ * a backend whose worker shell runs unsandboxed).
+ */
+export function scratchShell(how: string, prefix: string[]): AccessShell {
+  const cwd = mkdtempSync(join(realTmpdir(), "catherd-probe-"));
+  const [bin, ...rest] = prefix.length ? prefix : ["sh"];
+  const sh = prefix.length ? ["sh"] : [];
+  return {
+    how,
+    run: (script, args) =>
+      runCli(bin as string, [...rest, ...sh, "-c", script, "_", ...args], { ...probeShell, cwd }),
+    close: () => rmSync(cwd, { recursive: true, force: true }),
+  };
+}
+
+// Spec §5: what a workspace-write worker may reach besides the repo, as each backend's own flags grant it.
+
+/** `p` by its real path when it exists (macOS: /var/folders/… is /private/var/folders/…), else as given. */
+function realOr(p: string): string {
+  try {
+    return realpathSync(p);
+  } catch {
+    return p;
+  }
+}
+
+/** The temp dir by its real path (macOS: /var/folders/… is /private/var/folders/…); sandboxes compare real paths. */
+export const realTmpdir = (): string => realOr(tmpdir());
+
+/** The directories a workspace-write worker writes besides the repo: the heavy-lock dir and the temp dir, real paths. */
+export const writableRoots = (): string[] => [realOr(locksDir()), realTmpdir()];
+
+/** The usual local Docker sockets, in the order they are tried (OrbStack and Docker Desktop link /var/run/docker.sock). */
+export const dockerSocketCandidates = (home = homedir()): string[] => [
+  "/var/run/docker.sock",
+  join(home, ".orbstack", "run", "docker.sock"),
+  join(home, ".docker", "run", "docker.sock"),
+  join(home, ".colima", "default", "docker.sock"),
+];
+
+/**
+ * The local Docker socket a worker may talk to, by its real path: the one `DOCKER_HOST` names when it is a
+ * unix socket; none when it names a remote daemon (tcp://, ssh://…), which the user's docker client uses
+ * instead of any local socket; else the first of `candidates` that exists.
+ */
+export function dockerSocket(candidates = dockerSocketCandidates()): string | null {
+  const host = process.env.DOCKER_HOST;
+  if (host?.startsWith("unix://")) return realOr(host.slice("unix://".length));
+  if (host) return null;
+  const found = candidates.find((p) => existsSync(p));
+  return found ? realOr(found) : null;
+}

package/src/adapters/all.ts

@@ -0,0 +1,8 @@
+import { claudeCodeAdapter } from "./claude-code/index.ts";
+import { codexAdapter } from "./codex/index.ts";
+import { opencodeAdapter } from "./opencode/index.ts";
+import { registerAdapter } from "./registry.ts";
+
+registerAdapter(codexAdapter);
+registerAdapter(claudeCodeAdapter);
+registerAdapter(opencodeAdapter);

package/src/adapters/backend.ts

@@ -0,0 +1,159 @@
+import type { ErrorCode } from "../domain/errors.ts";
+import type { ADAPTER_IDS, Rung } from "../domain/ids.ts";
+import type { BillingMode } from "../domain/cost.ts";
+import type { Access, ExitInfo, RunStatus, Tokens } from "../domain/record.ts";
+import type { CliResult } from "./cli.ts";
+type AdapterId = (typeof ADAPTER_IDS)[number];
+
+export interface Probe {
+  installed: boolean;
+  version: string | null;
+  versionOk: boolean;
+  /** null when the CLI offers no way to ask */
+  loggedIn: boolean | null;
+  /** how the CLI is logged in, in a word or two ("ChatGPT", "API key"), when it says */
+  login?: string;
+  /** the billing mode that login implies, when it implies one: doctor compares it with the profiles' */
+  billing?: BillingMode;
+  problems: { code: ErrorCode; message: string; fix: string }[];
+}
+
+/** A worker's shell for doctor's probes: `run` is `sh -c <script> _ <args…>`; `close` removes its scratch dir. */
+export interface AccessShell {
+  /** how it runs, for the doctor rows: "codex sandbox", "an unsandboxed shell" */
+  how: string;
+  run(script: string, args: string[]): Promise<CliResult | null>;
+  close(): void;
+}
+
+export interface DiscoveredModel {
+  id: string;
+  efforts: string[];
+  context: number | null;
+  imageIn: boolean;
+}
+
+export interface RunRequest {
+  rung: Rung;
+  access: Access;
+  /** spec §5: a workspace-write role's network and loopback grants; false only when the profile says `network: false` */
+  network?: boolean;
+  thread: string | null;
+  isolated: boolean;
+  repo: string;
+  briefPath: string;
+  replyPath: string;
+  dispatchDir: string;
+}
+
+/**
+ * `env` holds only the adapter's overrides, never PWD. Admission writes them to spec.json with PWD added
+ * and nothing else, so no credential reaches disk; the `_supervise` process builds the worker's env at
+ * spawn time with `workerEnv(process.env, spec.env, spec.cwd)`, the only place secrets are scrubbed.
+ */
+export interface SpawnPlan {
+  cmd: string;
+  args: string[];
+  env: Record<string, string>;
+  cwd: string;
+  stdinPath: string | null;
+}
+
+export interface EventDelta {
+  thread?: string;
+  tokens?: Tokens;
+  /** the input tokens of one model request, from an event that reports them per request (the harness cost) */
+  requestInput?: number;
+  costUsd?: number;
+  lastEvent?: string;
+  /**
+   * what the worker is doing, in words, when the line says (spec §3.7 `peek`): a command (`$ bun test`), a file
+   * edit (`edit src/a.ts`) or a message line
+   */
+  activity?: string;
+  failure?: string;
+  limit?: boolean;
+  tooOld?: boolean;
+  retrying?: boolean;
+  /** the stream's terminal event: the supervisor may kill a CLI that lingers after it */
+  final?: boolean;
+  /** a tool call starting (`open`) or ending: while one is open the run is busy, however quiet */
+  item?: { id: string; open: boolean };
+}
+
+export interface FinishedRun {
+  request: RunRequest;
+  eventLines: string[];
+  reply: string;
+  stderr: string;
+  exit: ExitInfo;
+  startedAtMs: number;
+}
+
+export interface Outcome {
+  status: RunStatus;
+  thread: string | null;
+  tokens: Tokens;
+  costUsd: number | null;
+  images: string[];
+  error: { code: string; message: string } | null;
+  /** The role's reply, for a CLI that streams it rather than writing the reply file (absent: the file is the reply). */
+  reply?: string;
+}
+
+/** What earlier records on the same thread already counted, so a session total becomes this run's share. */
+export interface Spent {
+  tokens: Tokens;
+  costUsd: number;
+}
+
+export interface BackendAdapter {
+  id: AdapterId;
+  minVersion: string;
+  probe(): Promise<Probe>;
+  /** The models the backend offers; in `repo` when given, for a backend whose listing depends on it. */
+  listModels(repo?: string): Promise<DiscoveredModel[]>;
+  /**
+   * Refuses a rung this backend cannot run and readies the backend's own config, before admission writes
+   * anything (spec §6.3: variants are validated before dispatch). Throws a CatherdError.
+   */
+  prepare?(req: { rung: Rung; access: Access; isolated: boolean; repo: string }): Promise<void>;
+  plan(req: RunRequest): SpawnPlan;
+  parse(line: string): EventDelta;
+  finalize(run: FinishedRun): Outcome;
+  /**
+   * Spec §6.3: the backend's own account of a finished session, for a stream that undercounts. Returns
+   * `o` refined; never throws (the caller also cuts it off after a timeout and keeps `o`).
+   */
+  settle?(o: Outcome, run: FinishedRun, prior: Spent): Promise<Outcome>;
+  enforcement: Record<Access, "enforced" | "advisory">;
+  errors: { limit: RegExp[]; tooOld: RegExp[] };
+  resume: { supported: boolean; sameAccessOnly: boolean; threadPattern: RegExp };
+  interrupt?(thread: string, cwd: string): Promise<void>;
+  /** `sinceMs`: when this run started, so what an earlier run on the thread left behind does not count */
+  isBusy?(thread: string, cwd: string, sinceMs?: number): Promise<boolean>;
+  /** Spec §4.5: this backend's own stand-in for a rung on a usage limit, when the profile names none. */
+  failoverFor?(rung: Rung, repo?: string): Rung | null;
+  /**
+   * Spec §5 and §12: a shell that runs a command the way this backend's workspace-write worker runs one,
+   * with the grants the worker gets, for doctor's access probes; a string says why it cannot be tested here.
+   */
+  accessShell?(o: { network: boolean }): Promise<AccessShell | string>;
+  /** Spec §10.3: why this backend's isolation is weak; doctor warns when a profile uses it. */
+  isolationNote?: string;
+  graceAfterFinalMs: number | null;
+}
+
+export function extractVersion(s: string): string | null {
+  return /(\d+)\.(\d+)\.(\d+)/.exec(s)?.[0] ?? null;
+}
+
+export function compareVersions(a: string, b: string): number {
+  const pa = a.split(".").map(Number);
+  const pb = b.split(".").map(Number);
+  for (let i = 0; i < 3; i++) {
+    const d = (pa[i] ?? 0) - (pb[i] ?? 0);
+    if (d !== 0) return d;
+  }
+  return 0;
+}

package/src/adapters/claude-code/events.ts

@@ -0,0 +1,86 @@
+import { type Tokens, ZERO_TOKENS } from "../../domain/record.ts";
+
+interface ClaudeResult {
+  isError: boolean;
+  text: string;
+  apiStatus: number | null;
+  tokens: Tokens;
+  costUsd: number | null;
+}
+
+export interface ClaudeFold {
+  thread: string | null;
+  result: ClaudeResult | null;
+  limit: boolean;
+  lastEvent: string | null;
+}
+
+/** A usage or rate limit, in the result text Claude Code prints. */
+export const CLAUDE_LIMIT = [/usage limit/i, /hit your (?:usage )?limit/i, /rate[ _-]?limit/i];
+/** What an older `claude` prints for a flag catherd passes (commander's messages). */
+export const CLAUDE_TOO_OLD = [
+  /unknown option '--/i,
+  /option '--[\w-]+ <[\w.]+>' argument '[^']*' is invalid/i,
+];
+
+type Event = Record<string, any>;
+
+export function parseClaudeLine(line: string): Event | null {
+  if (!line.trim().startsWith("{")) return null;
+  try {
+    const e = JSON.parse(line) as unknown;
+    return e && typeof e === "object" && !Array.isArray(e) ? (e as Event) : null;
+  } catch {
+    return null;
+  }
+}
+
+/** Spec §4.3's convention: input counts cached and freshly cached tokens too. */
+export function claudeTokens(u: Event | undefined): Tokens {
+  if (!u) return { ...ZERO_TOKENS };
+  const n = (k: string) => (typeof u[k] === "number" ? (u[k] as number) : 0);
+  const cached = n("cache_read_input_tokens");
+  return {
+    input: n("input_tokens") + n("cache_creation_input_tokens") + cached,
+    cached,
+    output: n("output_tokens"),
+  };
+}
+
+export function eventName(e: Event): string {
+  if (e.type === "system") return e.subtype === "api_retry" ? "retrying" : `system/${e.subtype ?? "?"}`;
+  if (e.type === "assistant") {
+    const c = (e.message?.content ?? []).at(-1) as Event | undefined;
+    return c?.type === "tool_use" ? `tool_use/${c.name}` : `assistant/${c?.type ?? "?"}`;
+  }
+  if (e.type === "user") return "tool_result";
+  return String(e.type ?? "?");
+}
+
+const limited = (e: Event): boolean =>
+  (e.type === "rate_limit_event" && e.rate_limit_info?.status === "rejected") ||
+  (e.type === "assistant" && e.error === "rate_limit") ||
+  (e.type === "result" &&
+    e.is_error === true &&
+    (e.api_error_status === 429 || CLAUDE_LIMIT.some((r) => r.test(String(e.result ?? "")))));
+
+/** Tokens and cost come from the final `result` event only: assistant events repeat one message's usage. */
+export function foldClaudeEvents(lines: string[]): ClaudeFold {
+  const f: ClaudeFold = { thread: null, result: null, limit: false, lastEvent: null };
+  for (const line of lines) {
+    const e = parseClaudeLine(line);
+    if (!e) continue;
+    f.lastEvent = eventName(e);
+    if (typeof e.session_id === "string" && e.session_id) f.thread ??= e.session_id;
+    if (limited(e)) f.limit = true;
+    if (e.type === "result")
+      f.result = {
+        isError: e.is_error === true,
+        text: typeof e.result === "string" ? e.result : "",
+        apiStatus: typeof e.api_error_status === "number" ? e.api_error_status : null,
+        tokens: claudeTokens(e.usage),
+        costUsd: typeof e.total_cost_usd === "number" ? e.total_cost_usd : null,
+      };
+  }
+  return f;
+}