humanish 0.129.0 → 0.131.0

package/docs/architecture/browser-control.md

@@ -104,15 +104,17 @@ title, text, fractional scroll position and heard speech; those remain
 runtime-only under the existing loop contract. Arbitrary `appState` is refused
 because v1 has no closed schema for it. The protocol does not truncate strings,
 round coordinates, or silently drop unsupported state. A participant may ask for a
-longer wait than one request carries. The computer-use loop shortens it to the study's
-`actor.maxWaitMs` (by default two minutes, or 30 seconds on a desktop with speech, whose heard
-speech reaches the participant only with a screenshot), records a `wait shortened` notice with both
-durations when it does, and sends the rest as consecutive waits of at most 30 seconds. The Codex
-participant's `humanish_ui` schema leaves a wait's length open for that reason; the wire still
-refuses a single wait over 30 seconds. A wait that names no duration, as every OpenAI computer-use
-wait does, reaches the desktop with a length too: the study's `actor.idleWaitMs` (5 seconds by
-default) when the participant's turn only waits or takes screenshots, and 500 ms after an action in
-the same turn.
+longer wait than one request carries. In a turn with another action, the computer-use loop
+shortens it to the study's `actor.maxWaitMs` (by default two minutes, or 30 seconds on a desktop
+with speech, whose heard speech reaches the participant only with a screenshot), records a `wait
+shortened` notice with both durations when it does, and sends the rest as consecutive waits of at
+most 30 seconds. The Codex participant's `humanish_ui` schema leaves a wait's length open for that
+reason; the wire still refuses a single wait over 30 seconds. A wait that names no duration, as
+every OpenAI computer-use wait does, reaches the desktop with a length too: 500 ms after an action
+in the same turn. A turn that only waits or takes screenshots sends no wait to the desktop: the
+loop observes the screen every `actor.idleWaitMs` until it changes or the wait's time passes
+(`src/actors/computer-use/loop/watch.ts`), so a waiting participant costs one observe request per
+look.
 
 Click, double-click, move, scroll and drag may carry `heldKeys`, which the
 OpenAI provider maps from its computer tool's `keys`. The Codex participant's

package/docs/contracts/run-bundle.md

@@ -52,8 +52,8 @@ a run id or `latest`: it returns run.json with every trace file and live file jo
 as below, for v1, v2, v3 and v4 bundles, with each participant's per-request lists, and throws an Error named `StreamTraceRefusedError` for a
 file that does not match its index. Inside humanish, readers load a stream through
 `loadStreamTrace(runPaths, stream)`. A v1 stream returns as it is.
-An indexed stream's file is read under the 32 MiB per-file limit (analysis uses its 16 MiB
-source limit for each trace) and refused unless its size, sha256, item count and screenshot count
+An indexed stream's file is read under the 32 MiB per-file limit, by analysis as by every other
+reader, and refused unless its size, sha256, item count and screenshot count
 match the index. A trace or live file whose index records more than the limit is refused before it
 is read, and the refusal says the file is larger than humanish reads of one file and that
 restoring a copy does not help. A trace file whose index is under the limit and whose bytes are

package/docs/contracts/schemas.md

@@ -263,8 +263,10 @@ The rest of this section names keys by their v3 spelling:
   the end, a turn that requests no action does not end the session: the trace records a
   `participant ended before its scheduled end` notice, the next request says the time left, and
   the turn counts toward the idle backstop. Before the end, a turn that only waits or takes
-  screenshots counts toward neither backstop streak (`advanceBackstop`'s `waitingIsTheTask`). The
-  plan entry carries `endAfterMs`, and `study check` prints a `windows` row;
+  screenshots counts toward neither backstop streak (`advanceBackstop`'s `waitingIsTheTask`). A
+  watch that reaches the end stops there with no further look, and a backstop streak that trips on
+  a turn reaching the end gives way to the end. The plan entry carries `endAfterMs`, and `study
+  check` prints a `windows` row;
 - `participants[].maxSessionMs` (computer-use only, also on `count:` groups): the longest the
   participant's session lasts, in whole milliseconds from 1 to 86,400,000, counted from the start
   of its own session. `windDownMs` applies to it as to `endAfterMs`. Refused: a value outside that
@@ -282,7 +284,9 @@ The rest of this section names keys by their v3 spelling:
   `cap=<duration>`;
 - `participants[].idleWaitMs` (computer-use and shared-world, also on `count:` groups): this
   participant's `actor.idleWaitMs`, in the same range and refused above an explicit
-  `actor.maxWaitMs` (`waitLimitValidationReason`). Its trace records it in `waitSettings`;
+  `actor.maxWaitMs` (`waitLimitValidationReason`): how long its wait with no duration lasts in a
+  turn that only waits, and how often the loop looks at its screen while it watches such a turn.
+  Its trace records it in `waitSettings`;
 - `actor.clock.timeZone` (computer-use and shared-world): a tz database time zone name, such as `Europe/London`. Each turn's context hint carries
   the local time in it, and a scheduled end or session cap is given in it (UTC without it). On a hosted desktop the
   participant's browser launches with `TZ` set to it (`openParticipantSurface` in
@@ -1067,6 +1071,35 @@ Core-owned fields:
   click-like `ui_action` items, structured `coord` (`x`/`y`). Both are additive:
   absent on older bundles and non-stamping producers, and absence means
   "timing/position unrecorded", never zero
+- optional `items[].watch` (`ActorWatch` in `src/actors/contract.ts`), on the
+  computer-use `ui_action` of the wait during which a watch ended. A turn whose
+  actions are only waits and screenshots, and that asks for wait time, sends no
+  wait to the desktop: the loop looks at the screen every `waitSettings.idleWaitMs`
+  and requests the next turn when a change holds still, speech arrives, a
+  `stopWhen` rule, `dwell.when` or task `success` rule matches a look, the
+  participant's wind-down or end arrives, or the watch's time passes
+  (`src/actors/computer-use/loop/watch.ts`). A change of more than 24 tiles of
+  16 pixels counts once it holds still at a look; one of 24 tiles or fewer once
+  it has held still for more than 65 s with no tile within 4 tiles of it ticking
+  or moving with a region of more than 24 tiles
+  (`src/evidence/frame-change.ts`). The first watch lasts the turn's planned
+  length: the named waits' lengths, together up to `actor.maxWaitMs` (else
+  600000), and `idleWaitMs` for each wait that names none. Each further one in a row that ended
+  at its limit, or that a change of 24 tiles or fewer woke, lasts twice as long,
+  up to the larger of its planned length and that cap. `limitMs` is how long it
+  could last; `askedMs` is what the turn's waits asked for, the same sum with no
+  cap, so it is above `limitMs` when the named waits passed the cap and below it
+  when the watch grew; `wake` is `changed`, `speech`,
+  `condition`, `limit`, `clock` (wind-down or end), `stalled` (a look got no
+  frame; the next observation is a fresh one), or `stopped` (the study budget or
+  the session's time ran out); `looks` counts its looks; `area` (`x`, `y`,
+  `width`, `height` in pixels) bounds the change that woke it and is absent for
+  other wakes and for a change still moving at the last look. A `notice` titled
+  `watching the screen`, `lifecycle: started`, opens each watch. The turn's
+  waits share the watch in order, in proportion to their planned lengths, and
+  waits after the one carrying `watch` are titled `wait 0ms`. The look that
+  ended a watch is the turn's `turn-NN` screenshot; other looks are not kept.
+  Additive: absent on plain waits, other producers and older bundles
 - optional `items[].page` and `finalPage` (`{ origin, path }`, `TracePage` in
   `src/actors/contract.ts`): where a computer-use participant's browser was. The loop puts `page`
   on the `screenshot` item of each observation that read a URL: `turn-00-start` is the page before
@@ -1121,13 +1154,20 @@ Core-owned fields:
   `stopCause: provider_output_limit`.
 - optional `waitSettings`: the wait lengths a computer-use participant's session
   applied, in milliseconds, defaults included. `maxWaitMs` is the longest one
-  wait action lasts (`actor.maxWaitMs`); a longer wait is shortened to it.
+  wait action lasts (`actor.maxWaitMs`, else 120000, or 30000 on a desktop with
+  speech); in a turn with another action a longer wait is shortened to it.
   `idleWaitMs` is how long a wait that names no duration lasts in a turn that
   only waits or takes screenshots (`participants[].idleWaitMs`, else
-  `actor.idleWaitMs`), and `settleWaitMs` how
-  long such a wait lasts after an action in the same turn. Each `ui_action` for
-  a wait is titled with the length it lasted. Absent on other routes and on
-  earlier bundles
+  `actor.idleWaitMs`), and the time between looks at the screen while the loop
+  watches such a turn, at most 30000 on a desktop with speech. `settleWaitMs` is
+  how long a wait that names no duration lasts after an action in the same turn,
+  and the time to a watch's quick look after a large change. A watch's length
+  grows to `actor.maxWaitMs`, else 600000, which `waitSettings` does not record;
+  each watch's `watch.limitMs` does. The named waits of a turn that only waits
+  are watched for their lengths, together up to that cap, past `maxWaitMs`. A wait sent to the desktop has a
+  `ui_action` titled with its planned length, which a stalled wait keeps. A
+  watched wait's `ui_action` is titled with the part of the watch it covered.
+  Absent on other routes and on earlier bundles
 - optional `conversation`: how the first-party OpenAI computer-use provider
   carried the conversation between requests. `mode` is `threaded` (the server
   kept it through `previous_response_id`) or `explicit_context` (the server kept

package/docs/contracts/study-analysis.md

@@ -183,7 +183,19 @@ stopped, an operator can remove the empty lock directory and retry.
 
 Each request's packet admits up to 16 participants, 800 evidence items, 40 PNG
 captures, 160 KiB of text and 20 MiB of images. Individual source files, image
-dimensions and result sizes have separate limits. Count and text budgets are
+dimensions and result sizes have separate limits. Analysis reads run.json up to
+16 MiB and each participant's trace up to 32 MiB, as verify and the Observer do.
+It reads the traces one at a time and keeps of each only what selection can
+reach: the participant's share of its request's 800 evidence items, worked out
+from run.json before any trace is read, and the first 80 captures selection may
+read. Beside the one trace it is reading, analysis holds at most 800 entries for
+each request and 80 for each participant; their size grows with each trace
+item's text. A run.json past 16 MiB is refused as `ANALYSIS_SOURCE_TOO_LARGE`, and
+a run.json or trace without the fields analysis reads as `ANALYSIS_SOURCE_INVALID`;
+each message names the file, says why, and says no request was sent. Analysis
+verifies the run before it reads a trace, so a trace that does not match its
+index in run.json, or whose index records more than 32 MiB, fails verify and
+`humanish analyze` reports `ANALYSIS_VERIFY_FAILED`. Count and text budgets are
 distributed across included participants, with unused capacity from short
 sessions available to longer ones. Reclaimed capture slots go first to eligible
 participants with fewer admitted images, including those deferred by their

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "humanish",
-  "version": "0.129.0",
+  "version": "0.131.0",
   "description": "Synthetic user research for apps, CLIs, and agent-facing product flows.",
   "keywords": [
     "cli",