@saadnvd1/agent-os 0.3.471 → 0.3.473

package/README.md

@@ -33,6 +33,17 @@ sign-off. A workspace's running task limit (Settings → Workspaces) caps how ma
 wait under **Queued** with their place in line and start on their own, and
 `aos task --after` holds one until another finishes. [More](#tasks)
 
+### Each kind of work on the right model
+
+Planning and gate reviews run on Opus, implementation, the orchestrator and
+scope checks on Sonnet, titles and summaries on Haiku, each editable in
+Settings → Models and per workspace. A task plans on one model and
+implements on another in the same conversation, can be resumed on any
+model, and can run on another provider (DeepSeek, GLM, any
+Anthropic-compatible endpoint) whose key stays a reference.
+**Usage → Compare** shows Claude-only and routed tasks side by side.
+[More](#model-routing)
+
 ![A task's chat: the edit, the tests, the code review and the PR it opened](screenshots/task-chat.png)
 
 ![Two queued tasks in the sidebar, one in line and one waiting on another task, with its menu open](screenshots/queued.png)
@@ -437,6 +448,95 @@ and the same actions in each row's menu; a row becomes the task's own row
 when it starts. Why it works
 this way: [docs/decisions/2026-10-10-queued-tasks.md](docs/decisions/2026-10-10-queued-tasks.md).
 
+## Model routing
+
+Each kind of work AgentOS launches an agent for runs on its own model, set in
+**Settings → Models** (⌘K → Model settings). A model is named
+`provider/model`:
+
+| Role                                                | Default                               |
+| --------------------------------------------------- | ------------------------------------- |
+| Task planning (a task's plan step)                  | `anthropic/claude-opus-5-5`           |
+| Gate review                                         | `anthropic/claude-opus-5-5`           |
+| Task implementation                                 | `anthropic/claude-sonnet-5-5`         |
+| Orchestrator                                        | `anthropic/claude-sonnet-5-5`         |
+| Scope check                                         | `anthropic/claude-sonnet-5-5`         |
+| Interactive sessions                                | `anthropic/claude-opus-5-5`           |
+| Chat & summary helpers (titles, summaries, PR text) | `anthropic/claude-haiku-4-5-20251001` |
+
+The most specific setting wins: a task's own model, then its workspace's
+(**Settings → Workspaces → Models**), then Settings, then the default.
+`AGENTOS_REVIEW_MODEL` (gate review and scope check) and
+`AGENTOS_TITLE_MODEL` (helpers) in the server's environment still replace a
+role's default where Settings names none. A
+Claude session started without a model (the orchestrator's `start_session`,
+the API) runs on the interactive role's; one you pick a model for keeps it.
+
+**Plan, then implement.** A chat task whose planning and implementation
+models differ starts with a plan step: its first turn runs on the planning
+model and writes the plan to `.agentos/plan.md` in its worktree (ignored by
+git), changing nothing else. When that turn ends, the same conversation
+switches to the implementation model (a live model switch, or a resume with
+`--model` when the provider changes) and is told to follow the plan. Turn it
+off in Settings, or per task with `plan: false`. Terminal tasks skip it.
+
+**Per task.** The task draft has a **Model** chip ("Routed by role", or one
+model); `POST /api/tasks` and the orchestrator's `start_task` take `model`, as
+`"provider/model"` for the task's own roles or a map such as
+`{"implement": "zai/glm-5.3"}`, and `plan`. Its gate review and scope check
+never follow a task's model, only Settings and its workspace, so work can't
+pick its own reviewer. **Resume on model…** on a task's row
+(or `POST /api/tasks/<id>/model`, or the orchestrator's `set_task_model`)
+moves a paused task to another model: same conversation, same worktree, and
+it keeps that model from then on. Each turn records the model, provider and
+role it ran as.
+
+**Providers.** `anthropic` is your Claude login and needs no key. Others are
+Anthropic-compatible endpoints (OpenAI-compatible ones can be defined and
+tested, but Claude Code can't run on them): a base URL, where the key lives,
+the models, and Claude Code's opus/sonnet/haiku tiers mapped to them so
+subagents and background calls stay on that provider. DeepSeek
+(`https://api.deepseek.com/anthropic`) and Z.ai GLM
+(`https://api.z.ai/api/anthropic`) ship as presets, off until given a key.
+Claude stays the default: another provider runs only where a role or a task
+picks it. If a role's provider is turned off or its key can't be read,
+helper runs, reviews and the orchestrator fall back to the role's default
+rather than fail. **Test provider** sends one tiny request and says whether it
+answered. Changing models, providers or the test runs only from this machine
+or over Tailscale, never from a paired device: a provider's URL and key
+reference decide where a key is sent.
+
+The key is never stored by AgentOS. A provider names where to read it: a
+tokenvault entry (`tv get <project> <desc>`), a macOS Keychain item
+(`security find-generic-password -s <service> -w`) or an environment
+variable named `*_API_KEY` (not `ANTHROPIC_`, `CLAUDE_` or `AGENTOS_`). It's read when an agent launches and goes only into that agent's
+environment (`ANTHROPIC_BASE_URL` with `ANTHROPIC_AUTH_TOKEN` or
+`ANTHROPIC_API_KEY`), never its command line, the database, a prompt or a
+log; a terminal task's shell reads it from the reference itself. Errors that
+might echo a key are scrubbed of it.
+
+**Usage by model.** **Usage → By model** shows tokens and estimated cost per
+role and per model in the plan's 5-hour window and the last 24 hours, from
+the price table in Settings → Models (dollars per million input, cache
+write, cache read and output tokens, editable).
+
+**How to read the comparison.** **Usage → Compare** (or ⌘K → Model
+comparison, or the sidebar menu) puts Claude-only tasks (the baseline,
+including everything from before routing) beside routed ones, per provider,
+over 7, 30 or 90 days, with a total column. A task counts under the provider
+its own planning and implementation turns ran on; its reviews and checks add
+to its cost wherever they ran. For each column: tasks, turns, tokens,
+estimated dollars (list prices; for Claude, what the API would charge),
+Claude window points per task (how far each turn moved the plan's 5-hour
+window, an estimate), agent time, CI green on the first push, review passed
+on the first try, gate failures and rework commits per task (heads checked
+after the first review), and merged or dropped. Outcomes come from the
+orchestrator's checks, so tasks it didn't run have cost but no outcome. A
+column with fewer than 10 tasks is marked a small sample: a hint, not a
+result. Parity means the routed column's outcome rows match the baseline's
+while its window points per task drop. Why it works this way:
+[docs/decisions/2026-10-11-model-routing.md](docs/decisions/2026-10-11-model-routing.md).
+
 ## New sessions
 
 ⌥N (the **+** on a phone) opens a draft: a composer in the current project, or

package/app/api/models/route.ts

@@ -0,0 +1,34 @@
+import { NextRequest, NextResponse } from "next/server";
+import {
+  effectiveModels,
+  savedModelSettings,
+  setModelSettings,
+} from "@/lib/models/config";
+import { DEFAULT_ROLE_MODELS } from "@/lib/models/roles";
+import { requireLocalTrust } from "@/lib/security/route-guard";
+
+// GET /api/models - model routing: what's saved, what's in force, and the
+// role defaults. Providers carry key references only, never a key.
+export async function GET() {
+  return NextResponse.json({
+    saved: savedModelSettings(),
+    effective: effectiveModels(),
+    defaults: DEFAULT_ROLE_MODELS,
+  });
+}
+
+// PUT /api/models {roles?, providers?, prices?, planStep?} - replaces the
+// saved settings. Only from this machine or the tailnet: a provider's base
+// URL and key reference decide where a key is sent, so a stolen device
+// token mustn't be able to point them anywhere.
+export async function PUT(request: NextRequest) {
+  const refused = requireLocalTrust(request);
+  if (refused) return refused;
+  try {
+    setModelSettings(await request.json());
+    return GET();
+  } catch (error) {
+    const message = error instanceof Error ? error.message : String(error);
+    return NextResponse.json({ error: message }, { status: 400 });
+  }
+}

package/app/api/models/test/route.ts

@@ -0,0 +1,17 @@
+import { NextRequest, NextResponse } from "next/server";
+import { testProvider } from "@/lib/models/test-provider";
+import { requireLocalTrust } from "@/lib/security/route-guard";
+
+// POST /api/models/test {provider, model?} - one tiny request to the
+// provider with its key; says whether it answered, never the key. It sends
+// a key, so only from this machine or the tailnet.
+export async function POST(request: NextRequest) {
+  const refused = requireLocalTrust(request);
+  if (refused) return refused;
+  const { provider, model } = await request.json().catch(() => ({}));
+  if (typeof provider !== "string" || !provider)
+    return NextResponse.json({ error: "Name a provider" }, { status: 400 });
+  return NextResponse.json(
+    await testProvider(provider, typeof model === "string" ? model : undefined)
+  );
+}

package/app/api/sessions/[id]/summarize/route.ts

@@ -1,3 +1,4 @@
+import { routedRun } from "@/lib/models/launch";
 import { NextRequest, NextResponse } from "next/server";
 import { isRemoteHost } from "@/lib/hosts";
 import { spawn } from "child_process";
@@ -130,8 +131,10 @@ async function getTmuxCwd(sessionName: string): Promise<string | null> {
 async function generateSummary(conversation: string): Promise<string> {
   const prompt = `Summarize this Claude Code conversation in under 300 words. Focus on: what was built, key files changed, current state, and any pending work. Be specific.`;
 
+  const routed = await routedRun("helper");
   return new Promise((resolve, reject) => {
-    const claude = spawn("claude", ["-p", prompt], {
+    const claude = spawn("claude", ["-p", prompt, "--model", routed.model], {
+      env: { ...process.env, ...routed.env },
       stdio: ["pipe", "pipe", "pipe"],
     });
 

package/app/api/tasks/[id]/model/route.ts

@@ -0,0 +1,23 @@
+import { NextRequest, NextResponse } from "next/server";
+import { resumeOnModel } from "@/lib/models/resume";
+
+// POST /api/tasks/:id/model {model, message?} - resume the task on another
+// model ("provider/model" or a map of role to one), keeping its
+// conversation and worktree.
+export async function POST(
+  request: NextRequest,
+  { params }: { params: Promise<{ id: string }> }
+) {
+  const { model, message } = await request.json().catch(() => ({}));
+  try {
+    const result = await resumeOnModel(
+      (await params).id,
+      model,
+      typeof message === "string" && message.trim() ? message : undefined
+    );
+    return NextResponse.json(result);
+  } catch (error) {
+    const message = error instanceof Error ? error.message : String(error);
+    return NextResponse.json({ error: message }, { status: 409 });
+  }
+}

package/app/api/tasks/route.ts

@@ -1,3 +1,4 @@
+import { parseOverride } from "@/lib/models/resolve";
 import { NextRequest, NextResponse } from "next/server";
 import { createTask, listTasks, movedTasks } from "@/lib/tasks";
 import { startOrQueue } from "@/lib/tasks/queue";
@@ -33,6 +34,7 @@ export async function POST(request: NextRequest) {
       baseBranch,
       after,
       view,
+      plan,
     } = await request.json();
     if (id !== undefined && !isSessionId(id)) throw new Error("Bad task id");
     const pid =
@@ -42,7 +44,9 @@ export async function POST(request: NextRequest) {
     const opts = {
       projectId: pid,
       prompt: String(prompt ?? ""),
-      model,
+      // Checked now, so a queued task can't fail on it later.
+      model: parseOverride(model) ?? undefined,
+      plan: typeof plan === "boolean" ? plan : undefined,
       name: typeof name === "string" ? name : undefined,
       hostId: typeof hostId === "string" && hostId ? hostId : undefined,
       baseBranch: typeof baseBranch === "string" ? baseBranch : undefined,

package/app/api/usage/compare/route.ts

@@ -0,0 +1,21 @@
+import { NextRequest, NextResponse } from "next/server";
+import { comparisonReport } from "@/lib/usage/compare";
+
+const DAY = 24 * 3600 * 1000;
+
+// GET /api/usage/compare?days=30 (or from=&to= in ms) - Claude-only tasks
+// against routed ones: cost and outcomes per provider, and spend per role
+// and model.
+export async function GET(request: NextRequest) {
+  const q = request.nextUrl.searchParams;
+  const now = Date.now();
+  const days = Math.min(Math.max(Number(q.get("days")) || 30, 1), 365);
+  const to = Number(q.get("to")) || now;
+  const from = Number(q.get("from")) || to - days * DAY;
+  if (from >= to)
+    return NextResponse.json(
+      { error: "from must be before to" },
+      { status: 400 }
+    );
+  return NextResponse.json(comparisonReport(from, to));
+}

package/app/api/usage/route.ts

@@ -1,17 +1,20 @@
 import { NextRequest, NextResponse } from "next/server";
 import { USAGE_RANGES, type UsageRange } from "@/lib/usage/aggregate";
-import { checkRunReport, usageReport } from "@/lib/usage/report";
+import { checkRunReport, modelUsage, usageReport } from "@/lib/usage/report";
 import { usageWindows } from "@/lib/usage/windows";
 
 // GET /api/usage?range=today|7d|30d - chat spend by day, session and
-// workspace, what the orchestrator's reviews cost by PR, and the account's
-// rolling windows
+// workspace, what the orchestrator's reviews cost by PR, the account's
+// rolling windows, and spend by model and role in the 5-hour window and the
+// last 24 hours
 export async function GET(request: NextRequest) {
   const asked = request.nextUrl.searchParams.get("range") as UsageRange | null;
   const range = asked && USAGE_RANGES.includes(asked) ? asked : "7d";
+  const windows = await usageWindows();
   return NextResponse.json({
     report: usageReport(range),
     reviews: checkRunReport(range),
-    windows: await usageWindows(),
+    windows,
+    models: modelUsage(windows.fiveHour?.resetsIn),
   });
 }

package/app/api/workspaces/[id]/models/route.ts

@@ -0,0 +1,27 @@
+import { NextRequest, NextResponse } from "next/server";
+import { setWorkspaceModels, workspaceModels } from "@/lib/models/config";
+import { getWorkspace } from "@/lib/workspaces";
+import { requireLocalTrust } from "@/lib/security/route-guard";
+
+type RouteParams = { params: Promise<{ id: string }> };
+
+// GET/PUT /api/workspaces/:id/models {roles} - the workspace's own role
+// models, over Settings'.
+export async function GET(_request: NextRequest, { params }: RouteParams) {
+  return NextResponse.json(workspaceModels((await params).id));
+}
+
+// Writes only from this machine or the tailnet, like Settings' own.
+export async function PUT(request: NextRequest, { params }: RouteParams) {
+  const refused = requireLocalTrust(request);
+  if (refused) return refused;
+  const { id } = await params;
+  if (!getWorkspace(id))
+    return NextResponse.json({ error: "Workspace not found" }, { status: 404 });
+  try {
+    return NextResponse.json(setWorkspaceModels(id, await request.json()));
+  } catch (error) {
+    const message = error instanceof Error ? error.message : String(error);
+    return NextResponse.json({ error: message }, { status: 400 });
+  }
+}

package/components/CommandPalette/useAppCommands.ts

@@ -17,6 +17,7 @@ import {
   Plus,
   Sun,
   Workflow,
+  Scale,
 } from "lucide-react";
 import type { Session } from "@/lib/db";
 import type { PaletteCommand } from "@/lib/palette/registry";
@@ -147,7 +148,24 @@ export function useAppCommands({
       group: "Go to",
       keywords: ["cost", "tokens", "limits", "spend", "window"],
       icon: Gauge,
-      run: usageUiActions.open,
+      run: () => usageUiActions.open(),
+    },
+    {
+      id: "app.model-comparison",
+      title: "Model comparison",
+      group: "Go to",
+      keywords: [
+        "models",
+        "routing",
+        "providers",
+        "before after",
+        "quality",
+        "deepseek",
+        "glm",
+        "cost",
+      ],
+      icon: Scale,
+      run: () => usageUiActions.open("compare"),
     },
     {
       id: "app.tasks",

package/components/Draft/DraftChips.tsx

@@ -3,6 +3,7 @@
 import { useEffect } from "react";
 import {
   Bot,
+  Cpu,
   FolderGit2,
   GitBranch,
   GitPullRequest,
@@ -17,6 +18,7 @@ import { resolveModelForAgent } from "@/lib/model-catalog";
 import type { GitCheck } from "@/data/git/queries";
 import { useAgentStatusQuery } from "@/data/agents";
 import { useProjectMachinesQuery } from "@/data/projects";
+import { modelChoices, useModelsQuery } from "@/data/models";
 import type { AgentProbe } from "@/lib/agents/probe";
 import { ChipHeading, ChipItem, ChipMenu, ChipToggle } from "./Chip";
 
@@ -64,6 +66,7 @@ export function DraftChips({
   const { data: localProbes } = useAgentStatusQuery();
   const probes = local ? localProbes : undefined;
   const base = draft.baseBranch ?? git?.defaultBranch ?? "main";
+  const { data: models } = useModelsQuery(draft.openPr);
   // A project elsewhere runs where it lives; a scratch chat, a task or a
   // project here can go to another machine.
   const canMove = !project || draft.openPr || projectHere;
@@ -232,6 +235,31 @@ export function DraftChips({
           onChange={(openPr) => onChange({ openPr })}
         />
       )}
+      {project && draft.openPr && local && (
+        <ChipMenu
+          icon={Cpu}
+          name="Model"
+          label={draft.taskModel ?? "Routed by role"}
+        >
+          <ChipItem
+            checked={!draft.taskModel}
+            onSelect={() => onChange({ taskModel: null })}
+            hint="Plans and implements on the role models in Settings"
+          >
+            Routed by role
+          </ChipItem>
+          {modelChoices(models?.effective).map((c) => (
+            <ChipItem
+              key={c.ref}
+              checked={c.ref === draft.taskModel}
+              onSelect={() => onChange({ taskModel: c.ref })}
+              hint={c.provider}
+            >
+              {c.label}
+            </ChipItem>
+          ))}
+        </ChipMenu>
+      )}
       {project && draft.openPr && (
         <ChipToggle
           icon={SquareTerminal}

package/components/Draft/DraftPanel.tsx

@@ -65,7 +65,7 @@ export function DraftPanel({
         const { session, queued } = await createTask.mutateAsync({
           projectId: project.id,
           prompt: text,
-          model: draft.model,
+          model: draft.taskModel ?? undefined,
           baseBranch: draft.baseBranch ?? undefined,
           hostId: draft.hostId !== project.host_id ? draft.hostId : undefined,
           view: draft.terminal ? "terminal" : undefined,

package/components/Models/ModelSelect.tsx

@@ -0,0 +1,67 @@
+"use client";
+
+import {
+  Select,
+  SelectContent,
+  SelectGroup,
+  SelectItem,
+  SelectLabel,
+  SelectTrigger,
+  SelectValue,
+} from "@/components/ui/select";
+import { modelChoices } from "@/data/models";
+import type { EffectiveModels } from "@/lib/models/config";
+
+const INHERIT = "__inherit";
+
+// A "provider/model" picker over every enabled provider's models, grouped
+// by provider. With `inheritLabel`, a first option that clears it.
+export function ModelSelect({
+  id,
+  value,
+  effective,
+  inheritLabel,
+  disabled,
+  onChange,
+}: {
+  id?: string;
+  value: string | null;
+  effective: EffectiveModels | undefined;
+  inheritLabel?: string;
+  disabled?: boolean;
+  onChange: (ref: string | null) => void;
+}) {
+  const choices = modelChoices(effective);
+  const groups = [...new Set(choices.map((c) => c.provider))];
+  // A saved model no provider lists any more still shows as itself.
+  const missing = value && !choices.some((c) => c.ref === value);
+  return (
+    <Select
+      value={value ?? INHERIT}
+      onValueChange={(v) => onChange(v === INHERIT ? null : v)}
+      disabled={disabled}
+    >
+      <SelectTrigger id={id} className="h-11 md:h-9">
+        <SelectValue />
+      </SelectTrigger>
+      <SelectContent>
+        {inheritLabel && (
+          <SelectItem value={INHERIT}>{inheritLabel}</SelectItem>
+        )}
+        {missing && <SelectItem value={value}>{value}</SelectItem>}
+        {groups.map((g) => (
+          <SelectGroup key={g}>
+            <SelectLabel>{g}</SelectLabel>
+            {choices
+              .filter((c) => c.provider === g)
+              .map((c) => (
+                <SelectItem key={c.ref} value={c.ref}>
+                  {c.label}
+                </SelectItem>
+              ))}
+          </SelectGroup>
+        ))}
+      </SelectContent>
+    </Select>
+  );
+}