@cydm/pie 2.6.12 → 2.7.1

package/README.md

@@ -83,7 +83,7 @@ Required model fields:
 
 - `id`
 - `input`
-- `cost`
+- `cost` (`input`, `output`, `cacheRead`, `cacheWrite`; optional `cacheWriteLong`)
 - `contextWindow`
 - `maxTokens`
 
@@ -104,6 +104,22 @@ pie models init --stdout
 
 For editor auto-complete and validation, you can also point the optional `"$schema"` field in your local `models.json` at your installed `@cydm/pie/models.schema.json` file.
 
+`cost` may additionally carry `cacheWriteLong` — the per-million-token rate for 1h-TTL cache writes on providers that report a TTL breakdown (Anthropic-style routes). It is optional; legacy configs without it keep working. When it is absent, long writes are priced at the Anthropic 2x-input estimate and the calculated cost is marked `cacheWriteLongEstimated` so it is not presented as a precise provider bill.
+
+### Settings
+
+`~/.pie/settings.json` (or `./.pie/settings.json` for project scope) holds CLI defaults. Relevant prompt-cache keys:
+
+```jsonc
+{
+  // Explicit Anthropic prompt-cache TTL: "none" | "short" | "long".
+  // Absent keeps the provider default (Anthropic: short/5m).
+  "promptCacheRetention": "short"
+}
+```
+
+Priority is **per-request option > CLI setting > provider default**. The setting is forwarded only on Anthropic prompt-cache routes; it never forces OpenAI-compatible `store` or another provider's cache mode. Pie does not auto-switch TTL from observed hit rates — choose `long` explicitly when the 1h write premium is worth it.
+
 ### Remote model catalogs
 
 Pie can add centrally managed models without replacing local profiles or defaults. Add a catalog to `~/.pie/models.json` and keep the virtual key in an environment variable:
@@ -335,7 +351,7 @@ Platform notes:
 - **Windows**: the helper installs to `~/.pie/helpers/computer-use/windows-bridge.exe` and needs an interactive desktop session.
 - **Linux**: the helper installs to `~/.pie/helpers/computer-use/linux-bridge` and needs an AT-SPI2 bus (a running desktop session; headless servers are not supported).
 
-Config: `~/.pie/extensions/computer-use.json` (user) or `.pie/computer-use.json` (project) with `browser_use`, `headless`, `cursor_overlay`, `managed_browser`; env overrides `PIE_COMPUTER_USE_BROWSER_USE` / `PIE_COMPUTER_USE_HEADLESS` / `PIE_COMPUTER_USE_CURSOR_OVERLAY` / `PIE_COMPUTER_USE_MANAGED_BROWSER`. Run `/computer-use` (slash command) to inspect the loaded config, and `pie doctor` for helper install status. The extension can be toggled like any other (`/settings/extensions`, identity `computer-use`).
+Config: `~/.pie/extensions/computer-use.json` (user) or `.pie/computer-use.json` (project) with `browser_use`, `headless`, `cursor_overlay`, `managed_browser`; env overrides `PIE_COMPUTER_USE_BROWSER_USE` / `PIE_COMPUTER_USE_HEADLESS` / `PIE_COMPUTER_USE_CURSOR_OVERLAY` / `PIE_COMPUTER_USE_MANAGED_BROWSER`. The engine ships as the `computer-use-axi` skill: invoke it through bash (`node <skill baseDir>/computer-use-axi-cli.js <command>`) per its SKILL.md, and run `pie doctor` for helper install status.
 
 ## Web Research
 
@@ -349,7 +365,7 @@ Pie exposes one default model-visible web tool: `web_research`. Use it for publi
 
 Pie's default tools operate from the current workspace. File tools are workspace-scoped, `~/.pie` is allowlisted for Pie configuration/session support, and shell commands return structured timeout/truncation/error details. The canonical shell tool is still named `bash` for compatibility; on Windows `shell:auto` uses PowerShell, and callers can explicitly request `bash`, `powershell`, or `cmd`. High-risk shell commands require confirmation outside YOLO mode. `/settings/yolo` relaxes filesystem sandbox directory restrictions and should only be used in trusted workspaces. `pie permissions` prints the current policy-derived safety summary.
 
-In interactive Plan mode (`/tools/plan`), workspace editing tools are unavailable. The `bash` guard accepts simple read commands and read-only pipelines such as `rg 'TODO' . | head -20`; it blocks redirects, command substitution, and write-capable subcommands. Use `plan_scratch` to write, read, or list temporary text drafts in a directory dedicated to the current session. Shell redirection to `/tmp` remains blocked in Plan mode. Leave Plan mode before changing workspace files.
+In interactive Plan mode (`/tools/plan`), workspace editing tools are unavailable. The `bash` guard accepts simple read commands and read-only pipelines such as `rg 'TODO' . | head -20`; it blocks redirects, command substitution, and write-capable subcommands. Use `plan_scratch` to write, read, or list temporary text drafts in a directory dedicated to the current session. Shell redirection to `/tmp` remains blocked in Plan mode. When the plan is ready, the agent submits the ordered steps with `plan_submit` and you choose to execute immediately (progress is tracked through `[DONE:n]` markers in responses), keep planning, or refine the plan in a prefilled editor; a plan written as plain text under a `Plan:`/`计划：` heading is still detected as a fallback. Leave Plan mode before changing workspace files.
 
 ## Code Intelligence
 
@@ -441,3 +457,69 @@ If the package version already exists on npm, bump `products/cli/package.json` f
 ## License
 
 MIT
+
+### Experimental Responses profiles
+
+In `models.json`, set a profile's `api` to `"openai-responses"`, `baseUrl` to
+its API root (for example `https://api.openai.com/v1`), and `apiKeyEnv` to your
+credential environment variable. Supply model IDs and limits in `models` as
+usual. Optional profile/model `transport` is `"sse"` (default), `"websocket"`
+(strict), or `"auto"` (WS with conservative pre-submission SSE fallback).
+Model transport overrides profile transport. API selection is explicit.
+
+A profile may declare `endpoints: { "<api>": "https://..." }` and each model
+may select `api` (otherwise it inherits the profile API). Supported keys are
+`openai-completions`, `openai-responses`, and `anthropic-messages`; endpoint URLs
+follow the same absolute HTTP(S) validation as `baseUrl`. The resolved API uses
+`endpoints[api]` when present, otherwise `baseUrl`. An explicitly different model
+API **requires** its endpoint entry; an explicit same-API selection can use
+`baseUrl`. Remote catalogs additionally require every endpoint to share the
+authenticated catalog origin. `modelOverrides` cannot change these routes.
+
+For example, Bigmodel's Chat Completions and Responses roots differ. This is a
+configuration example, not special provider logic; use distinct model IDs within
+the profile (replace the Chat model ID with your supported model):
+
+```json
+{
+  "profiles": {
+    "bigmodel": {
+      "api": "openai-completions",
+      "baseUrl": "https://open.bigmodel.cn/api/coding/paas/v4",
+      "endpoints": {
+        "openai-responses": "https://open.bigmodel.cn/api/v1"
+      },
+      "apiKeyEnv": "BIGMODEL_API_KEY",
+      "transport": "sse",
+      "models": [
+        {
+          "id": "glm-5.2", "api": "openai-completions", "input": ["text"],
+          "contextWindow": 128000, "maxTokens": 4096,
+          "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 }
+        },
+        {
+          "id": "glm-5.3", "api": "openai-responses", "input": ["text"],
+          "contextWindow": 128000, "maxTokens": 4096,
+          "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 }
+        }
+      ]
+    }
+  }
+}
+```
+
+Limits/prices above are illustrative, not provider metadata. Keep Bigmodel and
+Kimi Responses on SSE for the tested endpoints/accounts: Responses support does
+not imply usable WS sessions. Current benchmarks show continuation upload savings,
+not consistent latency improvement; SSE stays default, WS/continuation opt-in.
+
+Pie defaults to `store: false`. Optional `compat` fields are `store`,
+`supportsStore`, `supportsReasoningEffort`, `supportsEncryptedReasoning`, and
+`responsesContinuation`. Set the latter to true to enable verified same-connection
+WS delta requests. SSE remains full-context. Local history remains authoritative;
+rewind, compact, reset and model/credential changes discard or re-evaluate anchors.
+Connection loss after submission is surfaced without automatic replay.
+
+Text/images and function tools are supported; audio/video inputs fail explicitly.
+Connection reuse, delta continuation and prompt-cache hits are separate facts.
+See [transport validation and opt-in benchmarks](../../docs/maintenance/responses-transport.md).

package/dist/builtin/extensions/ask-user/index.js

@@ -1,9 +1,9 @@
 import { createRequire as __createRequire } from "node:module"; const require = __createRequire(import.meta.url);
 import {
   createAskUserCapability
-} from "../../../chunks/chunk-PPEWRUMQ.js";
-import "../../../chunks/chunk-HHNOENEH.js";
-import "../../../chunks/chunk-7B4IIBVS.js";
+} from "../../../chunks/chunk-M44CWVDZ.js";
+import "../../../chunks/chunk-NJFPBU36.js";
+import "../../../chunks/chunk-XUROMYTT.js";
 import "../../../chunks/chunk-XXGYUYXI.js";
 import "../../../chunks/chunk-LZQXQOPJ.js";
 

package/dist/builtin/extensions/plan-mode/index.js

@@ -7,9 +7,9 @@ import {
   isPlanModeSafeCommand,
   markCompletedPlanSteps,
   restoreExecutionState
-} from "../../../chunks/chunk-PPEWRUMQ.js";
-import "../../../chunks/chunk-HHNOENEH.js";
-import "../../../chunks/chunk-7B4IIBVS.js";
+} from "../../../chunks/chunk-M44CWVDZ.js";
+import "../../../chunks/chunk-NJFPBU36.js";
+import "../../../chunks/chunk-XUROMYTT.js";
 import {
   Type
 } from "../../../chunks/chunk-XXGYUYXI.js";
@@ -26,10 +26,13 @@ var EXECUTION_FLAG_KEY = "planExecution";
 var SCRATCH_METADATA_KEY = "planScratchId";
 var SCRATCH_NAME = /^[a-zA-Z0-9][a-zA-Z0-9._-]{0,63}$/;
 var MAX_SCRATCH_CHARS = 64e3;
+var MAX_PLAN_STEPS = 50;
+var MAX_PLAN_ITEM_CHARS = 500;
 function toPlanTodoItems(state) {
   return state.steps.map((step) => ({
     step: step.id,
     text: step.title,
+    detail: step.description || void 0,
     completed: step.status === "completed"
   }));
 }
@@ -37,6 +40,8 @@ function planModeExtension(ctx) {
   const capability = createPlanCapability();
   let planModeEnabled = false;
   let executingPlan = false;
+  let approvalOfferedThisTurn = false;
+  let lastOfferedPlanKey = "";
   function setExecutingPlan(value) {
     executingPlan = value;
     ctx.setSessionMetadata(EXECUTION_FLAG_KEY, value);
@@ -113,6 +118,34 @@ function planModeExtension(ctx) {
     ctx.setExecutionReminder(buildExecutionReminder(next) ?? void 0);
     return next;
   }
+  function formatPlanText(items) {
+    return items.map((item, index) => `${index + 1}. ${item.text}${item.detail ? `
+   ${item.detail}` : ""}`).join("\n");
+  }
+  function renderPlanMarkdown(goal, items) {
+    const heading = goal ? `## \u{1F4CB} ${goal} \u2014 ${items.length} steps` : `## \u{1F4CB} Plan \u2014 ${items.length} steps`;
+    const steps = items.map((item, index) => {
+      const line = `${index + 1}. \u2610 ${item.text}`;
+      return item.detail ? `${line}
+   - \u21B3 ${item.detail}` : line;
+    }).join("\n");
+    return `${heading}
+
+${steps}`;
+  }
+  function startPlanExecution(items) {
+    planModeEnabled = false;
+    ctx.setSessionMetadata(SESSION_METADATA_KEY, false);
+    ctx.setPlanMode(false);
+    setExecutingPlan(items.length > 0);
+    writeExecutionState(createExecutionStateFromPlan(items));
+    updateStatus();
+    return `Execute the approved plan:
+
+${formatPlanText(items)}
+
+Execute the steps in order, starting with step 1. After completing a step, include [DONE:n] in your response.`;
+  }
   function updateStatus() {
     const state = readExecutionState();
     if (executingPlan && state.steps.length > 0) {
@@ -134,12 +167,83 @@ function planModeExtension(ctx) {
     setExecutingPlan(false);
     writeExecutionState(null);
     ctx.setSessionMetadata(SESSION_METADATA_KEY, planModeEnabled);
+    approvalOfferedThisTurn = false;
+    lastOfferedPlanKey = "";
     applyPlanModeState(planModeEnabled, true);
     ctx.ui.notify(
       planModeEnabled ? `Plan mode enabled. Available tools: ${ctx.getActiveTools().join(", ")}` : "Plan mode disabled. Full access restored."
     );
     updateStatus();
   }
+  ctx.registerTool({
+    name: "plan_submit",
+    label: "plan_submit",
+    description: "Submit the finished plan for user approval. Only useful in Plan mode: the user chooses to execute now, keep planning, or refine the plan. Use short imperative step titles; put constraints, risks, and verification notes in each step's optional detail. Do not execute any step before the user approves.",
+    parameters: Type.Object({
+      goal: Type.Optional(Type.String({ description: "One-line summary of what the plan accomplishes" })),
+      steps: Type.Array(
+        Type.Object({
+          title: Type.String({ description: "Short imperative step title", maxLength: MAX_PLAN_ITEM_CHARS }),
+          detail: Type.Optional(Type.String({ description: "Constraints, risks, or verification notes for this step", maxLength: 2e3 }))
+        }),
+        { minItems: 1, maxItems: MAX_PLAN_STEPS }
+      )
+    }),
+    risk: { riskClass: "session_mutation", permissionScope: "session", mutatesSession: true, availableInPlanMode: true },
+    async execute(args) {
+      const reply = (text, isError = false) => ({ content: [{ type: "text", text }], isError });
+      if (!planModeEnabled) {
+        return reply("plan_submit is only available in Plan mode. Enter Plan mode with /tools/plan first, or present the plan as text if you are not planning.", true);
+      }
+      const items = [];
+      for (const step of (args.steps ?? []).slice(0, MAX_PLAN_STEPS)) {
+        const title = String(step?.title ?? "").trim().slice(0, MAX_PLAN_ITEM_CHARS);
+        if (!title) continue;
+        const detail = typeof step?.detail === "string" && step.detail.trim() ? step.detail.trim().slice(0, 2e3) : void 0;
+        items.push({ step: items.length + 1, text: title, detail, completed: false });
+      }
+      if (items.length === 0) {
+        return reply("plan_submit requires at least one step with a non-empty title.", true);
+      }
+      approvalOfferedThisTurn = true;
+      lastOfferedPlanKey = JSON.stringify(items);
+      const rendered = items.map((item) => `${item.step}. \u2610 ${item.text}${item.detail ? `
+   ${item.detail}` : ""}`).join("\n");
+      if (!ctx.hasUI) {
+        return reply(`Plan submitted (${items.length} steps) in headless mode; interactive approval is unavailable, so nothing will be executed:
+
+${rendered}`);
+      }
+      const goal = typeof args.goal === "string" && args.goal.trim() ? args.goal.trim() : void 0;
+      ctx.ui.notify(renderPlanMarkdown(goal, items), "info", { markdown: true });
+      const yoloOption = typeof ctx.setYoloMode === "function" && !ctx.yoloMode ? ["Execute in YOLO mode (auto-approve everything)"] : [];
+      const choice = await ctx.ui.select(`\u{1F4CB} Plan (${items.length} steps) \u2014 what next?`, [
+        "Execute the plan (track progress)",
+        ...yoloOption,
+        "Stay in plan mode",
+        "Refine the plan"
+      ]);
+      if (choice?.startsWith("Execute")) {
+        if (choice.includes("YOLO")) {
+          ctx.setYoloMode?.(true);
+          ctx.ui.notify("\u26A0\uFE0F YOLO mode enabled for plan execution (sandbox directory restrictions disabled).", "warning");
+        }
+        return reply(startPlanExecution(items));
+      }
+      if (choice === "Refine the plan") {
+        const refinement = await ctx.ui.editor("Refine the plan (edit the plan or add feedback):", formatPlanText(items));
+        if (refinement?.trim()) {
+          return reply(`The user refined the plan:
+
+${refinement.trim()}
+
+Update the plan accordingly and call plan_submit again with the revised steps. Do not execute anything yet.`);
+        }
+        return reply("The user closed refinement without changes. Stay in Plan mode and wait for guidance.");
+      }
+      return reply("The user chose to stay in Plan mode. Do not execute anything; wait for guidance or refine the plan when asked.");
+    }
+  });
   ctx.registerCommand({
     path: capability.commands?.[0]?.path ?? ["tools", "plan"],
     description: capability.commands?.[0]?.description ?? "Toggle plan mode (read-only exploration)",
@@ -158,23 +262,27 @@ function planModeExtension(ctx) {
     const command = typeof event.args === "object" && event.args !== null ? event.args.command : void 0;
     if (typeof command === "string" && !isPlanModeSafeCommand(command)) {
       ctx.ui.notify(
-        `Plan mode: Command blocked. Use read-only commands, or plan_scratch for temporary drafts. Use /plan to leave Plan mode.
+        `Plan mode: Command blocked. Use read-only commands or plan_scratch for temporary drafts; submit the finished plan with plan_submit. Use /tools/plan to leave Plan mode.
 Command: ${command}`,
         "error"
       );
-      return { block: true, reason: `Plan mode: command "${command}" is not a read-only shell command. Use plan_scratch for temporary drafts, or /plan to leave Plan mode.` };
+      return { block: true, reason: `Plan mode: command "${command}" is not a read-only shell command. Use plan_scratch for temporary drafts, submit the finished plan with plan_submit, or use /tools/plan to leave Plan mode.` };
     }
   });
   ctx.on("agent:start", async () => {
     const persisted = readPersistedPlanMode();
     applyPlanModeState(persisted);
     executingPlan = !persisted && readPersistedExecuting();
+    approvalOfferedThisTurn = false;
+    if (!persisted) lastOfferedPlanKey = "";
     writeExecutionState(readExecutionState());
   });
   ctx.on("session:changed", async () => {
     const enabled = readPersistedPlanMode();
     applyPlanModeState(enabled, enabled);
     executingPlan = !enabled && readPersistedExecuting();
+    approvalOfferedThisTurn = false;
+    lastOfferedPlanKey = "";
     writeExecutionState(readExecutionState());
   });
   ctx.on("turn:end", async (event) => {
@@ -193,16 +301,18 @@ Command: ${command}`,
     const state = readExecutionState();
     if (executingPlan && state.steps.length > 0) {
       if (state.steps.every((step) => step.status === "completed")) {
-        const completedList = state.steps.map((step) => `\u2713 ${step.title}`).join("\n");
-        ctx.ui.notify(`**Plan Complete!** \u2713
+        const completedList = state.steps.map((step) => `${step.id}. \u2713 ${step.title}${step.description ? `
+   - \u21B3 ${step.description}` : ""}`).join("\n");
+        ctx.ui.notify(`## \u2705 Plan complete
 
-${completedList}`, "info");
+${completedList}`, "info", { markdown: true });
         setExecutingPlan(false);
         writeExecutionState(null);
       }
       return;
     }
     if (!planModeEnabled || !ctx.hasUI) return;
+    if (approvalOfferedThisTurn) return;
     const messages = event.messages || [];
     const lastAssistant = [...messages].reverse().find(
       (message) => message.role === "assistant" && Array.isArray(message.content)
@@ -213,27 +323,26 @@ ${completedList}`, "info");
     const text = content.filter((item) => item.type === "text").map((item) => item.text).join("\n");
     const extracted = extractPlanTodoItems(text);
     if (extracted.length === 0) return;
-    const extractedState = writeExecutionState(createExecutionStateFromPlan(extracted));
-    const todoListText = extractedState.steps.map((step) => `${step.id}. \u2610 ${step.title}`).join("\n");
-    ctx.ui.notify(`**Plan Steps (${extractedState.steps.length}):**
-${todoListText}`, "info");
-    const choice = await ctx.ui.select("Plan mode - what next?", [
+    const planKey = JSON.stringify(extracted);
+    if (planKey === lastOfferedPlanKey) return;
+    lastOfferedPlanKey = planKey;
+    approvalOfferedThisTurn = true;
+    const todoListText = extracted.map((item, index) => `${index + 1}. \u2610 ${item.text}${item.detail ? `
+   - \u21B3 ${item.detail}` : ""}`).join("\n");
+    ctx.ui.notify(`## \u{1F4CB} Plan \u2014 ${extracted.length} steps
+
+${todoListText}`, "info", { markdown: true });
+    const choice = await ctx.ui.select(`\u{1F4CB} Plan (${extracted.length} steps) \u2014 what next?`, [
       "Execute the plan (track progress)",
       "Stay in plan mode",
       "Refine the plan"
     ]);
     if (choice?.startsWith("Execute")) {
-      planModeEnabled = false;
-      setExecutingPlan(extractedState.steps.length > 0);
-      ctx.setSessionMetadata(SESSION_METADATA_KEY, false);
-      ctx.setPlanMode(false);
-      updateStatus();
-      const execMessage = extractedState.steps.length > 0 ? `Execute the plan. Start with: ${extractedState.steps[0].title}. Mark completed steps with [DONE:n] in your response.` : "Execute the plan you just created.";
-      ctx.sendUserMessage(execMessage);
+      ctx.sendUserMessage(startPlanExecution(extracted));
       return;
     }
     if (choice === "Refine the plan") {
-      const refinement = await ctx.ui.editor("Refine the plan:", "");
+      const refinement = await ctx.ui.editor("Refine the plan (edit the plan or add feedback):", formatPlanText(extracted));
       if (refinement?.trim()) {
         ctx.sendUserMessage(refinement.trim());
       }

package/dist/builtin/extensions/subagent/index.js

@@ -1,15 +1,16 @@
 import { createRequire as __createRequire } from "node:module"; const require = __createRequire(import.meta.url);
 import {
-  createCliHostCapabilities
-} from "../../../chunks/chunk-AAEBH6KX.js";
+  createCliHostCapabilities,
+  formatTransportLifecycle,
+  logTransportLifecycle
+} from "../../../chunks/chunk-W74NRSRV.js";
 import {
   SUBAGENT_SESSION_RECORD_VERSION,
   createSharedFileSystemTools,
   createSubagentCapability
-} from "../../../chunks/chunk-PPEWRUMQ.js";
-import "../../../chunks/chunk-HHNOENEH.js";
-import "../../../chunks/chunk-7B4IIBVS.js";
-import "../../../chunks/chunk-5353DEEC.js";
+} from "../../../chunks/chunk-M44CWVDZ.js";
+import "../../../chunks/chunk-NJFPBU36.js";
+import "../../../chunks/chunk-XUROMYTT.js";
 import "../../../chunks/chunk-XXGYUYXI.js";
 import "../../../chunks/chunk-LZQXQOPJ.js";
 
@@ -393,6 +394,14 @@ function subagentExtension(ctx) {
     log: (message) => ctx.log(message),
     notify: ctx.hasUI ? (message, type) => ctx.ui.notify(message, type === "success" ? "info" : type) : void 0,
     debugLog: (module, message, data) => ctx.log(`[subagent:${module}] ${message}${data !== void 0 ? ` ${JSON.stringify(data)}` : ""}`),
+    // Body-free Responses transport timeline for CHILD agents: 5 of the 7
+    // incident failures happened inside subagents with no transport record.
+    onTransportLifecycle: (event) => {
+      logTransportLifecycle(event);
+      if (event.phase === "timeout" || event.cleanupOutcome === "request_abandoned") {
+        ctx.log(formatTransportLifecycle(event));
+      }
+    },
     resolveModelClass: (modelClass) => ctx.resolveModelClass?.(modelClass),
     resolveModelRef: (modelRef) => ctx.resolveModelRef?.(modelRef),
     createRuntimeTools: (mode) => {

package/dist/builtin/extensions/todo/index.js

@@ -8,9 +8,9 @@ import {
   executeManageTodoList,
   executionStateToTodos,
   restoreExecutionState
-} from "../../../chunks/chunk-PPEWRUMQ.js";
-import "../../../chunks/chunk-HHNOENEH.js";
-import "../../../chunks/chunk-7B4IIBVS.js";
+} from "../../../chunks/chunk-M44CWVDZ.js";
+import "../../../chunks/chunk-NJFPBU36.js";
+import "../../../chunks/chunk-XUROMYTT.js";
 import "../../../chunks/chunk-XXGYUYXI.js";
 import "../../../chunks/chunk-LZQXQOPJ.js";
 

package/dist/builtin/skills/computer-use-axi/SKILL.md

@@ -0,0 +1,172 @@
+---
+name: computer-use-axi
+description: "Drive desktop apps and managed CDP browser pages through the computer-use-axi CLI run via bash: list UI roots, capture bounded outlines, search/expand/inspect element refs, perform checked actions with postconditions, and wait for conditions. Use whenever a task needs real UI interaction - clicking through a native app, reading a screen, driving a desktop window, or acting on a browser page state. Read this before the first UI-control task or after any command usage error."
+---
+
+# computer-use-axi
+
+Resolve the launcher against this skill's `baseDir` and run it through the
+host's shell tool (axi-style; on Windows use `shell:auto`/PowerShell, not
+bash.exe):
+
+```bash
+node <baseDir>/computer-use-axi-cli.js <command> [options]
+```
+
+Desktop control is a last resort — prefer API/CLI/web tools when they suffice.
+First use on macOS installs the `pie-computer-use.app` helper; the CLI then
+asks you to grant it Accessibility and Screen Recording permissions.
+
+## How it runs
+
+The CLI is a short-lived client. A local **broker** owns the execution engine,
+native helper handles, observation state, resource scheduling, `@o` output
+refs, and image artifacts. The broker starts on demand, is shared by every Pie
+session on this desktop, and exits after 30 minutes with no in-flight or
+unconfirmed work. It is not a system service and opens no network listener
+(Unix socket on macOS/Linux, per-user named pipe on Windows).
+
+```bash
+node <baseDir>/computer-use-axi-cli.js status   # broker pid/generation/queues
+node <baseDir>/computer-use-axi-cli.js stop     # safe shutdown, owns only its processes
+```
+
+A broker restart creates a new generation. State ids from the old generation
+are reported as expired; they are never restored or replayed — run `roots` and
+`observe` again.
+
+`stop` is a safe drain, not a hard kill. While a native action is still
+settling, the stopping broker keeps its endpoint bound and rejects new
+connections with `broker_draining`; a bootstrapping broker waits for that
+drain instead of starting a second execution authority. Retry shortly after a
+`broker_draining` rejection rather than racing another broker.
+
+## Current guidance lives in the CLI
+
+`--help` prints the full command surface; each command's output carries its
+own next-step guidance. This file covers the flow semantics.
+
+## The loop
+
+```bash
+node <baseDir>/computer-use-axi-cli.js roots                    # list @r roots
+node <baseDir>/computer-use-axi-cli.js observe --root @r1       # outline + stateId
+node <baseDir>/computer-use-axi-cli.js search --state S1 --text Save
+node <baseDir>/computer-use-axi-cli.js act --state S1 --actions '[{"action":"press","ref":"@e12"}]'
+```
+
+1. `roots` finds controllable windows/pages and mints `@r` refs.
+2. `observe` captures one root and returns a folded outline plus a `stateId`.
+   `--mode semantic` is cheapest (no OCR/images), `fused` is the default,
+   `visual` forces image evidence.
+3. `search` / `expand` / `inspect` query that saved state (`--state S1`) —
+   no re-observation needed. Search needs at least one of `--text`, `--role`,
+   `--capability`; refine broad queries instead of paging.
+4. `act` performs checked actions against the same state and returns the
+   successor state with a new `stateId` — use it directly for follow-ups.
+   Observe again only after uncertain external changes or eviction errors.
+
+## Refs and state
+
+- `@rN` root refs come from `roots`; they stay stable per window identity.
+- `@eN` element refs + the owning `stateId` come from `observe`/`search`;
+  every state-consuming command takes `--state`.
+- States are stale-checked by epoch: acting twice from one stateId fails the
+  second time by design — consume the successor stateId the act output names.
+- `@oN` refs continue long truncated output: `read --ref @o1 --offset N`
+  (byte offsets; no `--state` needed for `@o`). The broker allocates these.
+- `pictureOnly` nodes have no semantic element: coordinate actions only, and
+  only from an image-bearing (visual) state.
+
+## Image evidence
+
+Visual observations do not print base64. The broker writes the original
+screenshot to a private file (`0700` directory, `0600` file under the Pie
+config directory) and the text result names its absolute path, dimensions,
+sha256, broker generation, and owning `stateId`.
+
+When the result includes an `Image evidence:` line, call the normal
+`read_file` tool on that absolute path before using coordinates. The image
+belongs only to the returned `stateId`: after acting, use the successor
+image/state, not the old one. Artifacts are evicted with their state; if the
+file is gone, observe again instead of guessing coordinates. The broker never
+scales or transforms the bytes it stores.
+
+## Owners and concurrency
+
+The Pie host passes an opaque owner identity into the shell; separate sessions
+have separate observations, `@o` refs, and artifact directories. Manual CLI
+use without that environment resolves to a stable `standalone` namespace.
+Owners are **not** an OS security boundary — every client is the same local
+user. Two owners acting on the same app/page share one resource lane and one
+epoch counter, so the second write is rejected as stale rather than
+overlapping. `status` reports only an owner **count** and non-reversible
+fingerprints; it never prints another session's owner id.
+
+## Acting and batching
+
+```bash
+node <baseDir>/computer-use-axi-cli.js act --state S1 --actions '[
+  {"action":"click","ref":"@e18"},
+  {"action":"typeText","text":"hello"}
+]' --expect '{"text":"hello"}'
+```
+
+- Batch steps only when the second does not need to inspect the first's
+  result; steps run sequentially in one transaction.
+- Attach `--expect` (JSON condition: `text`/`role`/`value`/`ref`/`scope-ref`/
+  `until`/`timeout-ms`) when an action has an observable completion signal —
+  but only within the same root: a modal/sheet or a closing window is a NEW
+  root, so expect cannot see it; run the action without `--expect` and read
+  the returned root note instead.
+- Focus-following works only inside one act transaction: after a click into
+  an editable region, later `typeText`/`keypress` steps in the SAME
+  `--actions` array may omit `ref`. A standalone keypress (for example
+  `["cmd","w"]`) must carry `ref` (the window or a specific element).
+- On failure the output still names the successor state — inspect it instead
+  of blindly replaying.
+- Acting requires the platform helper: on macOS first use installs
+  `pie-computer-use.app` and asks for Accessibility + Screen Recording
+  grants. If a helper/setup error appears, run
+  `node <baseDir>/scripts/setup-helper.mjs` once, grant the permissions, and
+  retry.
+
+## Uncertain outcomes
+
+If a command returns `Outcome unknown` (the client timed out or lost the
+broker connection after dispatch), the native action may already have run.
+Do **not** replay it. The broker keeps that resource lane occupied until the
+native helper confirms completion; run `observe` to establish a successor
+state and decide from there. Reusing the same `--request-id` retrieves the
+original result (or `in-flight`/`outcome-unknown`); it never dispatches again.
+
+## Browser pages
+
+```bash
+node <baseDir>/computer-use-axi-cli.js browser launch --url https://example.com
+node <baseDir>/computer-use-axi-cli.js browser navigate --state S1 --url https://other.example
+node <baseDir>/computer-use-axi-cli.js browser eval --state S1 --expr 'document.title'
+```
+
+Browser states use the same outline/action/condition contracts as desktop
+states. Native browser windows are ordinary desktop UI — use `roots`/
+`observe`/`act` on them. Browser control may need enabling via the
+computer-use config (`browser_use`).
+
+## Test-only backend
+
+The deterministic fake desktop backend used by the broker test suite is not
+shipped in release bundles. In a source checkout it activates only when both
+`PIE_COMPUTER_USE_FAKE_BACKEND=1` and
+`PIE_COMPUTER_USE_ENABLE_TEST_COMMANDS=1` are set; otherwise the setting is
+ignored with a warning and the real platform backend is used. Never rely on it
+for real UI work.
+
+## Output bounds and exit codes
+
+Model-facing output is capped (~48 KiB / 2 000 lines); oversized results
+return a preview plus an `@o` continuation ref. Exit codes: `0` ok, `1` tool
+or broker error (message on stderr), `2` usage error, `3` broker
+version/generation mismatch (run `stop`, then retry). `--json` wraps
+`{ content, details, isError }` for programmatic use; `--request-id` makes a
+retry idempotent.