humanish 0.120.1 → 0.122.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +30 -0
- package/dist/actors/computer-use/actor.d.ts +3 -0
- package/dist/actors/computer-use/actor.js +1 -0
- package/dist/actors/computer-use/loop/backstop.d.ts +6 -1
- package/dist/actors/computer-use/loop/backstop.js +13 -1
- package/dist/actors/computer-use/loop/clock.d.ts +20 -0
- package/dist/actors/computer-use/loop/clock.js +64 -0
- package/dist/actors/computer-use/loop/ending.d.ts +1 -1
- package/dist/actors/computer-use/loop/session.d.ts +1 -1
- package/dist/actors/computer-use/loop/session.js +2 -1
- package/dist/actors/computer-use/loop/types.d.ts +16 -0
- package/dist/actors/computer-use/loop.d.ts +1 -1
- package/dist/actors/computer-use/loop.js +43 -8
- package/dist/actors/computer-use/wait.d.ts +3 -2
- package/dist/actors/computer-use/wait.js +5 -4
- package/dist/actors/contract.d.ts +3 -3
- package/dist/actors/contract.js +3 -0
- package/dist/actors/stop-cause.js +1 -0
- package/dist/cli/commands/study-format.js +25 -4
- package/dist/routes/computer-use/bundle.d.ts +3 -1
- package/dist/routes/computer-use/bundle.js +2 -1
- package/dist/routes/computer-use/diagnostics.d.ts +1 -1
- package/dist/routes/computer-use/diagnostics.js +4 -0
- package/dist/routes/computer-use/fanout-bundle.d.ts +2 -3
- package/dist/routes/computer-use/fanout-bundle.js +22 -14
- package/dist/routes/computer-use/live-phase.d.ts +1 -2
- package/dist/routes/computer-use/live-phase.js +2 -4
- package/dist/routes/computer-use/participant-deps.d.ts +1 -1
- package/dist/routes/computer-use/participant-deps.js +3 -1
- package/dist/routes/computer-use/participant-execution.d.ts +9 -14
- package/dist/routes/computer-use/participant-execution.js +102 -89
- package/dist/routes/computer-use/participant-model.js +17 -0
- package/dist/routes/computer-use/participant-runs.d.ts +11 -1
- package/dist/routes/computer-use/participant-runs.js +31 -4
- package/dist/routes/computer-use/plan.js +8 -1
- package/dist/routes/computer-use/result.js +57 -9
- package/dist/routes/computer-use/setup.d.ts +1 -1
- package/dist/routes/computer-use/single-bundle.d.ts +3 -0
- package/dist/routes/computer-use/single-bundle.js +3 -2
- package/dist/routes/computer-use/types.d.ts +33 -7
- package/dist/routes/shared-world/provisioned.js +2 -2
- package/dist/run/host-suspension.d.ts +56 -0
- package/dist/run/host-suspension.js +106 -0
- package/dist/run/judge.d.ts +1 -1
- package/dist/run/outcomes.js +3 -1
- package/dist/run/run.d.ts +15 -2
- package/dist/run/run.js +15 -8
- package/dist/run/status.d.ts +8 -2
- package/dist/run/status.js +11 -8
- package/dist/study/arrivals.d.ts +52 -5
- package/dist/study/arrivals.js +105 -15
- package/dist/study/keys.d.ts +6 -0
- package/dist/study/keys.js +4 -0
- package/dist/study/migrate/v2.js +8 -5
- package/dist/study/parse/actors.d.ts +1 -1
- package/dist/study/parse/actors.js +33 -4
- package/dist/study/plan-participants.d.ts +10 -1
- package/dist/study/plan-participants.js +10 -1
- package/dist/study/preflight.js +11 -1
- package/dist/study/study-deps.d.ts +6 -0
- package/dist/study/types.d.ts +26 -0
- package/dist/study/validation.d.ts +3 -2
- package/dist/study/validation.js +22 -4
- package/dist/tui-app.js +35 -35
- package/docs/contracts/run-bundle.md +20 -1
- package/docs/contracts/schemas.md +24 -3
- package/package.json +1 -1
|
@@ -172,13 +172,32 @@ Every surface that shows whether a run passed calls `runDisplay`
|
|
|
172
172
|
`outcome` line. A run
|
|
173
173
|
shows as `passed` only when its verdict is `pass` and its `ok` is not false.
|
|
174
174
|
|
|
175
|
+
## Host suspensions
|
|
176
|
+
|
|
177
|
+
The run's status cadence, every 5 s, is also its heartbeat. A wall-clock gap of 30 s or more
|
|
178
|
+
between two ticks, or between the last tick and a bundle write, is a host suspension: the machine
|
|
179
|
+
slept, or the process was stopped, and every participant loop on it stopped too. The wall clock
|
|
180
|
+
counts suspended time on macOS and Linux; Node's timers and `performance.now()` count it on macOS
|
|
181
|
+
only. Every bundle write records each suspension as a `host.suspended` warn event, `at` its start,
|
|
182
|
+
whose message names its start, end, length and offset into the run (`src/run/host-suspension.ts`).
|
|
183
|
+
|
|
184
|
+
On a computer-use run, a participant that failed in the harness or at its session time limit, and
|
|
185
|
+
was in flight across a suspension, likely failed because of it. So did a participant skipped after
|
|
186
|
+
such a failure (the pipeline gate). When any did, the outcome's first failure has
|
|
187
|
+
`kind: run` and one sentence, such as "The host was suspended for 6m 5s at +1m; the 34
|
|
188
|
+
participant failures after it are likely its effect.", ahead of each participant's own failure.
|
|
189
|
+
`review.summary` starts with the same sentence, the attributed participants share one gap line,
|
|
190
|
+
and the study result's `hostSuspension` holds the sentence, the participant ids and the
|
|
191
|
+
suspensions. Participant records, streams and traces stay as they ended. The other routes record
|
|
192
|
+
the events only.
|
|
193
|
+
|
|
175
194
|
## Recorded session stop causes
|
|
176
195
|
|
|
177
196
|
An actor trace may include `stopCause` alongside its unchanged `status`,
|
|
178
197
|
`completionReason` and verbatim `reason`. Computer-use sessions distinguish
|
|
179
198
|
`provider_output_limit`, `provider_token_limit`, `time_limit`, `spend_limit`,
|
|
180
199
|
`study_spend_limit`, `adapter_limit`, `provider_incomplete`, `provider_status`,
|
|
181
|
-
`provider_refused_prompt`, `harness_aborted`, and `
|
|
200
|
+
`provider_refused_prompt`, `harness_aborted`, `usage_unreported`, and `scheduled_end`. Absence means the route or
|
|
182
201
|
recording did not retain this precise field; it does not mean the participant
|
|
183
202
|
finished.
|
|
184
203
|
|
|
@@ -227,6 +227,25 @@ The rest of this section names keys by their v3 spelling:
|
|
|
227
227
|
and the most participants at once with every session at its full budget, and the provisioned
|
|
228
228
|
shared world's subject sandbox serves until the last session can end, which the plan checks
|
|
229
229
|
against the sandbox ceiling. A rerun starts its selected participants together;
|
|
230
|
+
- `participants[].endAfterMs` and `windDownMs` (computer-use only): when the participant's
|
|
231
|
+
session ends, in whole milliseconds on the `startAfterMs` clock and after its start, and how
|
|
232
|
+
long before that each turn tells it the end time and the minutes left (default 600,000).
|
|
233
|
+
Refused: an end at or before the start, a `windDownMs` without `endAfterMs`, an end on
|
|
234
|
+
`shared-world`, and a window whose sandbox deadline passes the sandbox ceiling. The
|
|
235
|
+
participant's session budget is its window plus 5 minutes (`participantSessionMs` in
|
|
236
|
+
`src/study/arrivals.ts`) in place of `execution.timeoutMs`, and its sandbox deadline adds the
|
|
237
|
+
route's headroom to that. At the end the loop requests the closing report (`debrief.trigger:
|
|
238
|
+
scheduled_end`) and the session ends `goal_satisfied` with `stopCause: scheduled_end`. Before
|
|
239
|
+
the end, a turn that requests no action does not end the session: the trace records a
|
|
240
|
+
`participant ended before its scheduled end` notice, the next request says the time left, and
|
|
241
|
+
the turn counts toward the idle backstop. Before the end, a turn that only waits or takes
|
|
242
|
+
screenshots counts toward neither backstop streak (`advanceBackstop`'s `waitingIsTheTask`). The
|
|
243
|
+
plan entry carries `endAfterMs`, and `study check` prints a `windows` row;
|
|
244
|
+
- `participants[].idleWaitMs` (computer-use and shared-world, also on `count:` groups): this
|
|
245
|
+
participant's `actor.idleWaitMs`, in the same range and refused above an explicit
|
|
246
|
+
`actor.maxWaitMs` (`waitLimitValidationReason`). Its trace records it in `waitSettings`;
|
|
247
|
+
- `actor.clock.timeZone` (computer-use): a tz database time zone name, such as `Europe/London`. Each turn's context hint carries
|
|
248
|
+
the local time in it, and a scheduled end is given in it (UTC without it);
|
|
230
249
|
- `execution.concurrency` (computer-use E2B routes, including shared-world): a
|
|
231
250
|
cap on participants in flight at once. When omitted, every participant runs
|
|
232
251
|
simultaneously: both routes resolve it when they plan, from the final participant count
|
|
@@ -460,8 +479,9 @@ length 1 at N=1; per-participant status/session/sandbox/subject), and `laneSumma
|
|
|
460
479
|
mirror the first participant and `subject.commit` is unanimity-gated across participants
|
|
461
480
|
(omitted with a divergence warning when participants resolve different commits). At N=1
|
|
462
481
|
the run bundle is byte-stable with the pre-fan-out output; only the result
|
|
463
|
-
projection changed. A fan-out run records a `cua-lab.fanout.plan` bundle event
|
|
464
|
-
|
|
482
|
+
projection changed. A fan-out run records a `cua-lab.fanout.plan` bundle event, and
|
|
483
|
+
each participant's outcome event (`cua-lab.session.<completionReason>`, `cua-lab.session.error`
|
|
484
|
+
or `cua-lab.session.blocked`) carries its reason;
|
|
465
485
|
`ok = observer.ok ∧ every participant ok ∧ no failed adapter or declared-scorer verdict`,
|
|
466
486
|
where a dry-run participant is ok as synthetic and a live participant must pass
|
|
467
487
|
`participantPassed` (`src/run/judge.ts`): not skipped, status `passed`, no harness
|
|
@@ -1013,7 +1033,8 @@ Core-owned fields:
|
|
|
1013
1033
|
applied, in milliseconds, defaults included. `maxWaitMs` is the longest one
|
|
1014
1034
|
wait action lasts (`actor.maxWaitMs`); a longer wait is shortened to it.
|
|
1015
1035
|
`idleWaitMs` is how long a wait that names no duration lasts in a turn that
|
|
1016
|
-
only waits or takes screenshots (`
|
|
1036
|
+
only waits or takes screenshots (`participants[].idleWaitMs`, else
|
|
1037
|
+
`actor.idleWaitMs`), and `settleWaitMs` how
|
|
1017
1038
|
long such a wait lasts after an action in the same turn. Each `ui_action` for
|
|
1018
1039
|
a wait is titled with the length it lasted. Absent on other routes and on
|
|
1019
1040
|
earlier bundles
|