humanish 0.111.0 → 0.111.1

package/CHANGELOG.md

@@ -8,6 +8,22 @@ The Unreleased section holds the full notes for the next version until it is tag
 
 ## Unreleased
 
+## 0.111.1: E2B URLs in study URLs, fan-out dry runs verify, zero-data-retention cache holds (2026-10-06)
+
+humanish 0.111.1 fixes a study URL refusal and two verify problems that an adopter met on 0.111.0,
+and the input cost of long computer-use sessions on zero-data-retention OpenAI organizations. A
+study URL may carry an E2B app URL in a query parameter or fragment, which 0.110.2 through 0.111.0
+refused as a credential. A computer-use dry run with more than one participant against an E2B app
+URL verifies `share_ready`, and verify finds a hex-encoded secret or E2B URL after `=`. On a
+zero-data-retention organization, a participant past about 64,000 tokens of carried conversation
+keeps the provider's prompt cache, and an OpenAI 404 for a stored item switches it to carrying its
+own conversation where 0.111.0 stopped it. A Codex participant's wait over 30 seconds is shortened
+to 30 seconds, a terminal session past 512 KiB of output keeps its verdict, and `humanish doctor`
+in a terminal study's sandbox reports the egress placeholder as no key. `execution.egressAllow` on
+a route other than `terminal` now warns, and 0.112.0 refuses it there.
+
+[Release notes](https://github.com/danielgwilson/humanish/releases/tag/v0.111.1)
+
 ## 0.111.0: StudyConfig has the keys of a v3 study file (2026-10-05)
 
 humanish 0.111.0 is the first breaking release under the compatibility policy. It breaks library

package/CONTEXT.md

@@ -33,7 +33,7 @@ bundle and output fields keep older spellings; the second table maps them.
 | participant desktop | The desktop one computer-use or shared-world participant drives: a substrate plus the route's concerns, such as subject provisioning, comms and the participant's plan. It lives in the route, so `src/substrates/` holds only provider primitives ([ADR 0004](docs/decisions/0004-participant-desktops-stay-in-the-route.md)).                                                                                                                                                                                                       | `ParticipantDesktop` in `src/routes/computer-use/participant-desktop.ts`; `createE2BParticipantDesktop` in `src/routes/computer-use/e2b-desktop/desktop.ts`; the local VM's in `src/routes/computer-use/local-vm.ts`                                                                                                                                                                                        |
 | receipt             | A line in a run's `sandbox-receipts.ndjson` that records a sandbox's id, provider, the participant or subject it belongs to (`laneId`), and its time limit as soon as the sandbox is created, before any work runs in it. `humanish reclaim` kills sandboxes by these ids after an interrupted run.                                                                                                                                                                                                                                   | `SandboxReceipt` in `src/run/sandbox-receipts.ts`; `reclaimRunSandboxes` in `src/run/reclaim.ts`                                                                                                                                                                                                                                                                                                            |
 | owner tags          | The E2B metadata every sandbox humanish creates carries: `tool: humanish`, the run's `runId`, and `runKey`, a digest of the run directory's inode and creation time. The run journals them in an owner line before its first create. `humanish reclaim` lists sandboxes that match all three, which finds one whose id never reached a receipt.                                                                                                                                                                                       | `sandboxOwnerTags` in `src/run/sandbox-creates.ts`; `findE2BSandboxesByTags` in `src/substrates/e2b/sandbox.ts`                                                                                                                                                                                                                                                                                             |
-| topology            | Whether participants share one instance of the subject. `route: shared-world` gives one for all; every other route gives each participant its own (`per-lane-worlds`). `run.json` records it as `sharedWorld.topology`.                                                                                                                                                                                                                                                                                                               | `declaresSharedWorld` in `src/study/study-fields.ts`                                                                                                                                                                                                                                                                                                                                                        |
+| topology            | Whether participants share one instance of the subject. `route: shared-world` gives one for all; every other route gives each participant its own (`per-lane-worlds`). `run.json` records it as `sharedWorld.topology`.                                                                                                                                                                                                                                                                                                               | `route` in `StudyConfig` (`src/study/types.ts`); `isSharedWorldComposition` in `src/study/routing.ts`                                                                                                                                                                                                                                                                                                       |
 | plane               | The one instance of the subject that a shared-world study's participants use together. A `provisioned` plane is a `clone` or `local-tree` subject that humanish serves in a sandbox, with a declared subject state. An `external-public` plane is a deployment already running at `subject.appUrl`, whose owner the study names in `subject.publicTarget.owner`. Each plane has at least two participants.                                                                                                                            | `SharedWorldPlane` in `src/study/plan-types.ts`; `src/routes/shared-world/`                                                                                                                                                                                                                                                                                                                                 |
 | stream              | One live or recorded view of a participant, such as a browser or a terminal, in `run.json` and the Observer.                                                                                                                                                                                                                                                                                                                                                                                                                          | `RunStream` in `src/run/streams.ts`                                                                                                                                                                                                                                                                                                                                                                         |
 | Observer            | The single-file review page built from `observer/observer-data.json` (`humanish.observer-data.v1`).                                                                                                                                                                                                                                                                                                                                                                                                                                   | `observer/` (the page), `src/observer/` (data, render, serve)                                                                                                                                                                                                                                                                                                                                               |

package/dist/actors/codex/restricted-participant-policy.d.ts

@@ -9,6 +9,8 @@ export declare const PARTICIPANT_LIMITS: Readonly<{
     narration: 2000;
     output: number;
     actions: 4;
+    /** The longest wait one action lasts; a longer one is shortened to it. */
+    waitMs: 30000;
     requestMs: 180000;
     cleanupMs: 5000;
 }>;

package/dist/actors/codex/restricted-participant-policy.js

@@ -1,5 +1,5 @@
 import { z } from "zod";
-import { browserControlActionSchema, browserOnlyControlActionSchema, validateBrowserControlAction, } from "../../browser-control/protocol.js";
+import { BROWSER_CONTROL_LIMITS, browserControlActionSchema, browserOnlyControlActionSchema, validateBrowserControlAction, } from "../../browser-control/protocol.js";
 import { validClosingReport } from "../computer-use/loop.js";
 /** The declared profile. A participant replaces cliVersion with its detected CLI release, or with
  * this host's newest qualified release before its first launch. */
@@ -21,6 +21,8 @@ export const PARTICIPANT_LIMITS = Object.freeze({
     narration: 2000,
     output: 256 * 1024,
     actions: 4,
+    /** The longest wait one action lasts; a longer one is shortened to it. */
+    waitMs: BROWSER_CONTROL_LIMITS.waitMs,
     requestMs: 180_000,
     cleanupMs: 5000,
 });
@@ -49,15 +51,41 @@ export const PARTICIPANT_FINAL_SCHEMA = {
         },
     },
 };
+/**
+ * Shorten each wait longer than one browser-control request may wait. A failed tool call ends the
+ * native run, so a wait that asks too much would otherwise end the whole session.
+ */
+function shortenWaits(value) {
+    const actions = value?.actions;
+    if (typeof value !== "object" || !Array.isArray(actions))
+        return { value, shortened: [] };
+    const shortened = [];
+    const limit = PARTICIPANT_LIMITS.waitMs;
+    const next = actions.map((action, index) => {
+        const wait = action;
+        if (wait?.kind !== "wait" ||
+            typeof wait.ms !== "number" ||
+            !Number.isFinite(wait.ms) ||
+            wait.ms <= limit)
+            return action;
+        shortened.push({ index, requestedMs: wait.ms, ms: limit });
+        return { ...wait, ms: limit };
+    });
+    return shortened.length === 0
+        ? { value, shortened }
+        : { value: { ...value, actions: next }, shortened };
+}
 /** humanish tool arguments, validated before the shared executor sees a batch. */
 export function parseParticipantTool(value, speechEnabled = false) {
     if (Buffer.byteLength(JSON.stringify(value) ?? "") > PARTICIPANT_LIMITS.output)
         throw new Error("invalid_response");
-    const v = toolInput(speechEnabled).parse(value);
+    const { value: admitted, shortened } = shortenWaits(value);
+    const v = toolInput(speechEnabled).parse(admitted);
     if (Buffer.byteLength(v.narration) > PARTICIPANT_LIMITS.narration)
         throw new Error("invalid_response");
     return {
         actions: v.actions.map(validateBrowserControlAction),
+        ...(shortened.length === 0 ? {} : { shortenedWaits: shortened }),
         pendingSafetyChecks: [],
         done: false,
         ...(v.narration ? { message: v.narration } : {}),

package/dist/actors/codex/restricted-participant.js

@@ -39,7 +39,7 @@ function deferred() {
     return { promise, resolve, reject };
 }
 // prose-check: model prompt (Codex reads this tool description, not a person)
-const toolDescription = (speechEnabled) => `Act on the participant's browser through humanish. Submit one to four UI actions${speechEnabled ? ", including speak when you need to reply aloud," : ""} and a short public comment; never private reasoning. humanish returns a JSON STRING with execution acknowledgments${speechEnabled ? ", speech heard from the actual participant speaker sink," : ""} and a fresh screenshot. In Code Mode use: const r = JSON.parse(await tools.humanish_ui({narration: "...", actions: [...]})); text({acknowledgments: r.acknowledgments${speechEnabled ? ", heardSpeech: r.heardSpeech" : ""}, contextHint: r.contextHint, closing: r.closing}); image(r.imageUrl). Call serially and inspect each returned screenshot${speechEnabled ? " and heardSpeech array" : ""} before deciding what to do next.${speechEnabled ? " Use speak only after the visible UI shows that you joined the call and your microphone is unmuted; it sends audio into that call." : ""} Acknowledged input does not prove an application outcome. If closing is true, stop calling tools and give your final account.`;
+const toolDescription = (speechEnabled) => `Act on the participant's browser through humanish. Submit one to four UI actions${speechEnabled ? ", including speak when you need to reply aloud," : ""} and a short public comment; never private reasoning. humanish returns a JSON STRING with execution acknowledgments${speechEnabled ? ", speech heard from the actual participant speaker sink," : ""} and a fresh screenshot. In Code Mode use: const r = JSON.parse(await tools.humanish_ui({narration: "...", actions: [...]})); text({acknowledgments: r.acknowledgments${speechEnabled ? ", heardSpeech: r.heardSpeech" : ""}, contextHint: r.contextHint, closing: r.closing}); image(r.imageUrl). Call serially and inspect each returned screenshot${speechEnabled ? " and heardSpeech array" : ""} before deciding what to do next. A wait action lasts at most ${L.waitMs} ms; a longer wait is shortened to ${L.waitMs} ms, so wait longer with another call.${speechEnabled ? " Use speak only after the visible UI shows that you joined the call and your microphone is unmuted; it sends audio into that call." : ""} Acknowledged input does not prove an application outcome. If closing is true, stop calling tools and give your final account.`;
 const participantInstructions = (prompt, speechEnabled) => `${prompt}\n\nYou are the study participant throughout this conversation, including its closing account. Use only the supplied screenshots and humanish_ui tool to interact. The tool returns a JSON string: parse it, inspect acknowledgments${speechEnabled ? " and heardSpeech captured from the actual participant speaker sink" : ""}, and display imageUrl with Code Mode image(). Do not print the image data URL as text. Keep your persona and earlier observations throughout the session.${speechEnabled ? " A speak action plays into the participant microphone; use it only after the visible UI shows that you joined the call and the microphone is unmuted." : ""} Speak publicly about your experience, never reveal private reasoning. Completed inputs do not prove application outcomes; verify on the next screenshot. When the task ends, return only the required final JSON with outcome, summary and frictionReports. Report observed confusion and recovered mistakes as well as blockers. Do not invent observations.`;
 /** Turns and failures from the native run, handed in order to the next provider request. */
 class ParticipantEvents {

package/dist/actors/computer-use/executor-diagnostic.d.ts

@@ -0,0 +1,15 @@
+/** The driver steps a diagnostic can name. */
+export declare const CUA_DIAGNOSTIC_STEPS: readonly ["owner_window_check", "cdp_session", "frame_tree", "isolated_world", "focus_probe", "insert_text", "cdp_detach", "text_port", "channel", "channel_write"];
+export type CuaDiagnosticStep = (typeof CUA_DIAGNOSTIC_STEPS)[number];
+/** The kinds of failure a diagnostic can name. */
+export declare const CUA_DIAGNOSTIC_CATEGORIES: readonly ["target_closed", "detached", "navigation", "timeout", "protocol_error", "channel_closed", "unknown"];
+export type CuaDiagnosticCategory = (typeof CUA_DIAGNOSTIC_CATEGORIES)[number];
+export interface CuaExecutorDiagnostic {
+    readonly step: CuaDiagnosticStep;
+    readonly category: CuaDiagnosticCategory;
+}
+/** The diagnostic for `error`, thrown during `step`: the step and a category, never its text. */
+export declare function classifyExecutorFailure(step: CuaDiagnosticStep, error: unknown): CuaExecutorDiagnostic;
+export declare function isCuaExecutorDiagnostic(value: unknown): value is CuaExecutorDiagnostic;
+/** The fixed sentence for a trace or a failure record, built only from the two words. */
+export declare function describeExecutorDiagnostic(diagnostic: CuaExecutorDiagnostic): string;

package/dist/actors/computer-use/executor-diagnostic.js

@@ -0,0 +1,105 @@
+// The diagnostic an executor attaches to a failure its driver did not declare: which step failed
+// and what kind of failure it was, as two fixed words. It carries no error text. A browser error
+// message can hold a form value, page text, a cookie pair or a file path, and redaction on the
+// host cannot be relied on to catch all of them, so the message never leaves the classifier.
+/** The driver steps a diagnostic can name. */
+export const CUA_DIAGNOSTIC_STEPS = Object.freeze([
+    "owner_window_check",
+    "cdp_session",
+    "frame_tree",
+    "isolated_world",
+    "focus_probe",
+    "insert_text",
+    "cdp_detach",
+    "text_port",
+    "channel",
+    "channel_write",
+]);
+/** The kinds of failure a diagnostic can name. */
+export const CUA_DIAGNOSTIC_CATEGORIES = Object.freeze([
+    "target_closed",
+    "detached",
+    "navigation",
+    "timeout",
+    "protocol_error",
+    "channel_closed",
+    "unknown",
+]);
+const STEP_LABELS = Object.freeze({
+    owner_window_check: "checking the owner window",
+    cdp_session: "opening a DevTools session (newCDPSession)",
+    frame_tree: "reading the frame tree (Page.getFrameTree)",
+    isolated_world: "creating the isolated world (Page.createIsolatedWorld)",
+    focus_probe: "probing focus (Runtime.evaluate)",
+    insert_text: "inserting text (Input.insertText)",
+    cdp_detach: "detaching the DevTools session (CDPSession.detach)",
+    text_port: "running a text port step",
+    channel: "reading the browser-control channel",
+    channel_write: "writing to the browser-control channel",
+});
+const CATEGORY_SENTENCES = Object.freeze({
+    target_closed: "the page, its browser context or the browser closed",
+    detached: "the DevTools session or the frame detached",
+    navigation: "the page navigated or its document was replaced",
+    timeout: "the call timed out",
+    protocol_error: "the browser refused a DevTools protocol call",
+    channel_closed: "the channel to the guest closed",
+    unknown: "the error matched no known kind",
+});
+/** Error classes that name their category on their own. */
+const ERROR_CLASSES = Object.freeze({
+    TargetClosedError: "target_closed",
+    TimeoutError: "timeout",
+    ProtocolError: "protocol_error",
+});
+/** Node system error codes a broken channel or socket reports. */
+const SYSTEM_CODES = Object.freeze({
+    EPIPE: "channel_closed",
+    ECONNRESET: "channel_closed",
+    ERR_STREAM_DESTROYED: "channel_closed",
+    ERR_STREAM_PREMATURE_CLOSE: "channel_closed",
+    ERR_STREAM_WRITE_AFTER_END: "channel_closed",
+    ETIMEDOUT: "timeout",
+});
+/**
+ * Known Playwright and Chromium message shapes, checked in order; the first match names the
+ * category. The message is only tested here, never copied.
+ */
+const MESSAGE_SHAPES = Object.freeze([
+    [/detached|No session with given id/i, "detached"],
+    [
+        /\b(?:target|page|context|browser|session)\b[^.]*\b(?:closed|crashed|disconnected)\b/i,
+        "target_closed",
+    ],
+    [/navigat|Execution context was destroyed|Cannot find context with specified id/i, "navigation"],
+    [/timed? ?out|timeout/i, "timeout"],
+    [/\bProtocol error\b/i, "protocol_error"],
+]);
+function categoryOf(error) {
+    if (!(error instanceof Error))
+        return "unknown";
+    if (Object.hasOwn(ERROR_CLASSES, error.name))
+        return ERROR_CLASSES[error.name];
+    const code = error.code;
+    if (typeof code === "string" && Object.hasOwn(SYSTEM_CODES, code))
+        return SYSTEM_CODES[code];
+    const message = typeof error.message === "string" ? error.message : "";
+    return MESSAGE_SHAPES.find(([shape]) => shape.test(message))?.[1] ?? "unknown";
+}
+/** The diagnostic for `error`, thrown during `step`: the step and a category, never its text. */
+export function classifyExecutorFailure(step, error) {
+    return { step, category: categoryOf(error) };
+}
+export function isCuaExecutorDiagnostic(value) {
+    if (typeof value !== "object" || value === null)
+        return false;
+    const { step, category } = value;
+    return (Object.keys(value).length === 2 &&
+        CUA_DIAGNOSTIC_STEPS.includes(step) &&
+        CUA_DIAGNOSTIC_CATEGORIES.includes(category));
+}
+/** The fixed sentence for a trace or a failure record, built only from the two words. */
+export function describeExecutorDiagnostic(diagnostic) {
+    const { step, category } = diagnostic;
+    return `${step}, ${category}: ${CATEGORY_SENTENCES[category]} while ${STEP_LABELS[step]}`;
+}

package/dist/actors/computer-use/executor-error.d.ts

@@ -1,3 +1,4 @@
+import { type CuaExecutorDiagnostic } from "./executor-diagnostic.js";
 declare const messages: Readonly<{
     executor_closed: "Desktop session is closed.";
     executor_not_ready: "Desktop executor is not ready.";
@@ -14,17 +15,31 @@ declare const messages: Readonly<{
 }>;
 export type CuaExecutorErrorCode = keyof typeof messages;
 export type CuaExecutorDisposition = "not_dispatched" | "outcome_uncertain";
+/**
+ * Why an executor refused an action before dispatch, when the participant can act on it. Each is
+ * a fixed word: `extra_tab` means the local browser types only while the study's tab is its only
+ * tab, and another tab is open.
+ */
+export declare const CUA_REJECTION_REASONS: readonly ["extra_tab"];
+export type CuaRejectionReason = (typeof CUA_REJECTION_REASONS)[number];
 export declare function isCuaExecutorErrorCode(value: unknown): value is CuaExecutorErrorCode;
 /**
  * An executor's bounded failure declaration. No backend message, cause, action payload or
- * transport details belong here. `not_dispatched` requires evidence that dispatch never
- * began; a lost acknowledgement or partial action is `outcome_uncertain`, never a retry.
+ * transport details belong here; `diagnostic` names a failed step and a category in fixed words.
+ * `not_dispatched` requires evidence that dispatch never began; a lost acknowledgement or partial
+ * action is `outcome_uncertain`, never a retry.
+ * `reason` is allowed only on an `action_rejected` / `not_dispatched` refusal.
  * Import from the same installation as the loop: names and lookalike objects do not qualify.
  */
 export declare class ComputerUseExecutorError extends Error {
     readonly code: CuaExecutorErrorCode;
     readonly disposition: CuaExecutorDisposition;
-    constructor(code: CuaExecutorErrorCode, disposition: CuaExecutorDisposition);
+    readonly reason?: CuaRejectionReason;
+    readonly diagnostic?: CuaExecutorDiagnostic;
+    constructor(code: CuaExecutorErrorCode, disposition: CuaExecutorDisposition, details?: {
+        reason?: CuaRejectionReason | undefined;
+        diagnostic?: CuaExecutorDiagnostic | undefined;
+    });
 }
 /** A forged prototype is not an executor declaration. */
 export declare function isComputerUseExecutorError(error: unknown): error is ComputerUseExecutorError;

package/dist/actors/computer-use/executor-error.js

@@ -1,3 +1,4 @@
+import { isCuaExecutorDiagnostic } from "./executor-diagnostic.js";
 const messages = Object.freeze({
     executor_closed: "Desktop session is closed.",
     executor_not_ready: "Desktop executor is not ready.",
@@ -12,32 +13,57 @@ const messages = Object.freeze({
     execution_failed: "Desktop executor could not complete the request.",
     cancelled: "Desktop executor request was cancelled.",
 });
+/**
+ * Why an executor refused an action before dispatch, when the participant can act on it. Each is
+ * a fixed word: `extra_tab` means the local browser types only while the study's tab is its only
+ * tab, and another tab is open.
+ */
+export const CUA_REJECTION_REASONS = Object.freeze(["extra_tab"]);
+function isCuaRejectionReason(value) {
+    return CUA_REJECTION_REASONS.includes(value);
+}
 export function isCuaExecutorErrorCode(value) {
     return typeof value === "string" && Object.hasOwn(messages, value);
 }
 const executorErrors = new WeakSet();
 /**
  * An executor's bounded failure declaration. No backend message, cause, action payload or
- * transport details belong here. `not_dispatched` requires evidence that dispatch never
- * began; a lost acknowledgement or partial action is `outcome_uncertain`, never a retry.
+ * transport details belong here; `diagnostic` names a failed step and a category in fixed words.
+ * `not_dispatched` requires evidence that dispatch never began; a lost acknowledgement or partial
+ * action is `outcome_uncertain`, never a retry.
+ * `reason` is allowed only on an `action_rejected` / `not_dispatched` refusal.
  * Import from the same installation as the loop: names and lookalike objects do not qualify.
  */
 export class ComputerUseExecutorError extends Error {
     code;
     disposition;
-    constructor(code, disposition) {
+    reason;
+    diagnostic;
+    constructor(code, disposition, details = {}) {
+        const { reason, diagnostic } = details;
         if (!isCuaExecutorErrorCode(code) ||
-            (disposition !== "not_dispatched" && disposition !== "outcome_uncertain")) {
+            (disposition !== "not_dispatched" && disposition !== "outcome_uncertain") ||
+            (reason !== undefined &&
+                (!isCuaRejectionReason(reason) ||
+                    code !== "action_rejected" ||
+                    disposition !== "not_dispatched")) ||
+            (diagnostic !== undefined && !isCuaExecutorDiagnostic(diagnostic))) {
             throw new TypeError("Invalid desktop executor error declaration.");
         }
         super(messages[code]);
         this.name = "ComputerUseExecutorError";
         this.code = code;
         this.disposition = disposition;
+        if (reason !== undefined)
+            this.reason = reason;
+        if (diagnostic !== undefined)
+            this.diagnostic = Object.freeze({ step: diagnostic.step, category: diagnostic.category });
         // Keep the values used in durable diagnostics finite even for JavaScript callers.
         Object.defineProperties(this, {
             code: { writable: false, configurable: false },
             disposition: { writable: false, configurable: false },
+            reason: { writable: false, configurable: false },
+            diagnostic: { writable: false, configurable: false },
         });
         executorErrors.add(this);
     }

package/dist/actors/computer-use/loop/actions.d.ts

@@ -1,5 +1,5 @@
 import type { LoopSession } from "./session.js";
-import type { CuaAction, CuaTurnRequest } from "./types.js";
+import type { CuaAction, CuaTurnRequest, ShortenedWait } from "./types.js";
 /** A public-safe one-line action label. Never includes raw typed text. */
 export declare function describeCuaAction(action: CuaAction): string;
 /**
@@ -15,7 +15,10 @@ export declare function isIdleTurn(actions: readonly CuaAction[]): boolean;
 export interface ActionBatch {
     /** Host input acknowledgements for the provider's next request. */
     readonly execution: NonNullable<CuaTurnRequest["previousExecution"]>;
-    /** Guidance for the next request, set when a pre-dispatch rejection stopped the batch. */
+    /**
+     * Guidance for the next request, set when the provider shortened a wait or a pre-dispatch
+     * rejection stopped the batch.
+     */
     readonly hint: string | undefined;
 }
 /**
@@ -23,4 +26,4 @@ export interface ActionBatch {
  * leaves the desktop usable; a failed desktop command skips its action; any other failure ends
  * the session.
  */
-export declare function runActionBatch(session: LoopSession, actions: readonly CuaAction[]): Promise<ActionBatch>;
+export declare function runActionBatch(session: LoopSession, actions: readonly CuaAction[], shortenedWaits?: readonly ShortenedWait[]): Promise<ActionBatch>;

package/dist/actors/computer-use/loop/actions.js

@@ -1,6 +1,6 @@
 import { classifyCuaAction } from "../../affordance.js";
 import { commandFailureInfo, isCommandExitError } from "../../../substrates/command-failure.js";
-import { ComputerUseExecutorError, isComputerUseExecutorError } from "../executor-error.js";
+import { ComputerUseExecutorError, isComputerUseExecutorError, } from "../executor-error.js";
 import { CuaAbortError, CuaDeadlineError, CuaStallError, raceCallBound, raceSessionDeadline, requestScope, } from "./race.js";
 import { notice } from "./trace.js";
 // The participant's actions: their public labels, what counts as a material action, and how one
@@ -101,21 +101,39 @@ export function isIdleTurn(actions) {
 function isMaterialAttempt(action, status) {
     return !isIdleAction(action) && status !== "skipped" && status !== "not_dispatched";
 }
-function rejectedActionHint(title) {
-    return `Your action (${title}) was rejected before dispatch. No input from that action or the rest of its batch was sent. Choose your next action from the fresh screenshot; do not assume the rejected action succeeded.`;
+/** What the participant can do about a rejection whose reason the executor named. */
+const REJECTION_REMEDIES = Object.freeze({
+    extra_tab: "It was rejected because more than one browser tab is open: typing works only while the study's original tab is the only tab. Switch to the other tab and close it, keep the study's original tab open, then type again.",
+});
+function rejectedActionHint(title, reason) {
+    const remedy = reason === undefined ? "" : ` ${REJECTION_REMEDIES[reason]}`;
+    return `Your action (${title}) was rejected before dispatch. No input from that action or the rest of its batch was sent.${remedy} Choose your next action from the fresh screenshot; do not assume the rejected action succeeded.`;
+}
+function shortenedWaitHint(wait) {
+    return `Your wait of ${wait.requestedMs}ms was shortened to ${wait.ms}ms, the longest one wait action lasts. To keep waiting, send another wait.`;
+}
+/** Record a wait the provider shortened, and return the participant's note about it. */
+function recordShortenedWait(session, title, wait) {
+    session.trace.record("notice", () => notice("warn", "wait shortened", `action: ${title}; requested: ${wait.requestedMs}ms; waited: ${wait.ms}ms, the longest one wait action lasts; the session continued`));
+    return shortenedWaitHint(wait);
 }
 /**
  * Dispatch one turn's actions in order. A declared pre-dispatch rejection stops the batch and
  * leaves the desktop usable; a failed desktop command skips its action; any other failure ends
  * the session.
  */
-export async function runActionBatch(session, actions) {
+export async function runActionBatch(session, actions, shortenedWaits = []) {
     const execution = { actions: [] };
     const { actionHistory, trace } = session;
+    const notes = [];
+    const hint = () => (notes.length === 0 ? undefined : notes.join(" "));
     for (const [index, action] of actions.entries()) {
         if (session.signal?.aborted)
             throw new CuaAbortError();
         const title = describeCuaAction(action);
+        const shortened = shortenedWaits.find((wait) => wait.index === index);
+        if (shortened !== undefined)
+            notes.push(recordShortenedWait(session, title, shortened));
         actionHistory.lastActionTitle = title;
         actionHistory.recentActionTitles.push(title);
         if (actionHistory.recentActionTitles.length > RECENT_ACTION_TITLES)
@@ -147,8 +165,10 @@ export async function runActionBatch(session, actions) {
                 for (let later = index + 1; later < actions.length; later++) {
                     execution.actions.push({ index: later, status: "not_dispatched" });
                 }
-                trace.record("notice", () => notice("warn", "action rejected before dispatch", session.redactNarration(`action: ${title}; code: action_rejected; disposition: not_dispatched; remaining batch actions not dispatched: ${actions.length - index - 1}`)));
-                return { execution, hint: rejectedActionHint(title) };
+                const { reason } = error;
+                trace.record("notice", () => notice("warn", "action rejected before dispatch", session.redactNarration(`action: ${title}; code: action_rejected; disposition: not_dispatched;${reason === undefined ? "" : ` reason: ${reason};`} remaining batch actions not dispatched: ${actions.length - index - 1}`)));
+                notes.push(rejectedActionHint(title, reason));
+                return { execution, hint: hint() };
             }
             if (isComputerUseExecutorError(error) || !isCommandExitError(error))
                 throw error;
@@ -171,7 +191,7 @@ export async function runActionBatch(session, actions) {
                 : {}),
         }));
     }
-    return { execution, hint: undefined };
+    return { execution, hint: hint() };
 }
 function countAttempt(session, action, title, status) {
     if (!isMaterialAttempt(action, status))

package/dist/actors/computer-use/loop/ending.js

@@ -1,3 +1,4 @@
+import { describeExecutorDiagnostic } from "../executor-diagnostic.js";
 import { isComputerUseExecutorError } from "../executor-error.js";
 import { isComputerUsePromptRefusedError, isComputerUseProviderError } from "../provider-error.js";
 import { CuaAbortError, CuaDeadlineError } from "./race.js";
@@ -278,17 +279,24 @@ export function stopForError(session, error) {
         };
     }
     if (isComputerUseExecutorError(error)) {
+        // The cause of a failure the driver did not declare, such as a closed browser target.
+        const cause = error.diagnostic === undefined
+            ? undefined
+            : `cause: ${describeExecutorDiagnostic(error.diagnostic)}`;
         const detail = () => [
             `phase: ${redact(session.phase)}`,
             `code: ${error.code}`,
             `disposition: ${error.disposition}`,
+            cause,
             lastAction(),
         ]
             .filter(Boolean)
             .join("; ");
         return {
             completionReason: "harness_error",
-            reason: `desktop executor error: ${error.code}; disposition: ${error.disposition}`,
+            reason: [`desktop executor error: ${error.code}`, `disposition: ${error.disposition}`, cause]
+                .filter(Boolean)
+                .join("; "),
             evidence: {
                 kind: "notice",
                 body: () => ({ ...notice("error", "desktop executor error", detail()), ...screenshot }),

package/dist/actors/computer-use/loop/types.d.ts

@@ -55,6 +55,15 @@ export type CuaAction = {
 } | {
     kind: "screenshot";
 };
+/** A wait action whose duration the provider shortened before the loop dispatched it. */
+export interface ShortenedWait {
+    /** The action's index in its turn. */
+    index: number;
+    /** What the participant asked for. */
+    requestedMs: number;
+    /** What the action waits: the `ms` it carries. */
+    ms: number;
+}
 /** A captured desktop state: the (optional) frame plus a coarse signature for progress. */
 export interface CuaObservation {
     /**
@@ -162,6 +171,11 @@ export interface CuaTurn {
     message?: string;
     /** Actions to perform this turn. Empty means the model is done. */
     actions: CuaAction[];
+    /**
+     * Waits the provider shortened to the longest wait its desktop accepts, by index into
+     * `actions`. The loop records each one and tells the participant on its next request.
+     */
+    shortenedWaits?: ReadonlyArray<ShortenedWait>;
     /** Safety checks the provider flagged this turn. Non-empty pauses the run. */
     pendingSafetyChecks: CuaSafetyCheck[];
     /** Token accounting for this turn, if available. */

package/dist/actors/computer-use/loop.d.ts

@@ -1,5 +1,5 @@
 import type { CuaLoopOptions, CuaLoopResult, LoopRunOptions } from "./loop/types.js";
-export type { CuaAction, CuaExecutor, CuaLiveMetadata, CuaLoopOptions, CuaLoopResult, CuaObservation, CuaProvider, CuaSafetyCheck, CuaSpendGate, CuaTurn, CuaTurnRequest, } from "./loop/types.js";
+export type { CuaAction, CuaExecutor, CuaLiveMetadata, CuaLoopOptions, CuaLoopResult, CuaObservation, CuaProvider, CuaSafetyCheck, CuaSpendGate, CuaTurn, CuaTurnRequest, ShortenedWait, } from "./loop/types.js";
 export { describeCuaAction } from "./loop/actions.js";
 export { stableProgressKey } from "./loop/backstop.js";
 export { validClosingReport } from "./loop/debrief.js";

package/dist/actors/computer-use/loop.js

@@ -121,7 +121,7 @@ async function runTurns(session, conversation) {
             await observer.observeFinalTasks(turnNumber);
             return ended;
         }
-        const batch = await runActionBatch(session, turn.actions);
+        const batch = await runActionBatch(session, turn.actions, turn.shortenedWaits);
         conversation.previousExecution = batch.execution;
         const checkpoint = await observer.checkpoint(turnNumber);
         if ("stop" in checkpoint)

package/dist/actors/computer-use/openai-context.d.ts

@@ -1,4 +1,4 @@
-/** The estimated input a carried conversation may reach before its oldest turns are summarized. */
+/** The estimated input a carried conversation may reach before it is cut. */
 export declare const CONTEXT_TOKEN_BUDGET = 64000;
 /** What the next request would carry, for the trace. */
 export interface CarriedContextSize {
@@ -14,7 +14,11 @@ export declare class CarriedConversation {
     /** The latest accepted reply's output items, which the next request answers. */
     private pending;
     private readonly notes;
-    private notesDropped;
+    /**
+     * The turns whose note lines were dropped to keep the note under its cap. Lines are dropped in
+     * turn order from just after the head, so they are one run of consecutive turns.
+     */
+    private elided;
     private collapsedTurns;
     constructor(budget?: number);
     /**
@@ -27,6 +31,8 @@ export declare class CarriedConversation {
     /** The carried context's size, for the trace. */
     size(): CarriedContextSize;
     private trim;
+    private capNote;
+    private noteText;
     private noteItem;
 }
 /**