humanish 0.127.0 → 0.127.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -8,6 +8,17 @@ The Unreleased section holds the full notes for the next version until it is tag
8
8
 
9
9
  ## Unreleased
10
10
 
11
+ ## 0.127.1: Clock-end closing reports declare their outcome; skipped participants named as skipped (2026-10-13)
12
+
13
+ humanish 0.127.1 fixes two things the 0.127.0 release smoke found in scheduled studies. An OpenAI
14
+ participant asked for its closing report at a scheduled end or session cap now says how the task
15
+ went in a structured `outcome` field, so one that reached its goal counts in `reachedGoal` where
16
+ 0.127.0 counted it in `ranUntilEnd`. A participant the run skips before its session starts now
17
+ shows its skip reason on its run line, and the summary counts it apart from those that took
18
+ part.
19
+
20
+ [Release notes](https://github.com/danielgwilson/humanish/releases/tag/v0.127.1)
21
+
11
22
  ## 0.127.0: Clock-end passes counted by what participants declared, a scaled wind-down, the concurrency cap in the plan (2026-10-13)
12
23
 
13
24
  humanish 0.127.0 makes long scheduled studies report what happened. A participant that passes at
@@ -23,10 +23,17 @@ export declare const participantImpressionsSchema: z.ZodArray<z.ZodObject<{
23
23
  text: z.ZodString;
24
24
  }, z.core.$strict>>;
25
25
  /**
26
- * A closing report. A report without impressions is still valid: third-party provider ports and
26
+ * A closing report. `outcome` is how the participant says the task went: `reached` when it is
27
+ * finished, `blocked` when something in the app stopped it, `not_reached` for any other reason.
28
+ * A report without an outcome or impressions is still valid: third-party provider ports and
27
29
  * stored traces do not have them.
28
30
  */
29
31
  export declare const closingReportSchema: z.ZodObject<{
32
+ outcome: z.ZodOptional<z.ZodEnum<{
33
+ blocked: "blocked";
34
+ not_reached: "not_reached";
35
+ reached: "reached";
36
+ }>>;
30
37
  summary: z.ZodString;
31
38
  frictionReports: z.ZodArray<z.ZodString>;
32
39
  impressions: z.ZodOptional<z.ZodArray<z.ZodObject<{
@@ -32,10 +32,13 @@ const impressionSchema = z.strictObject({
32
32
  /** The impressions a participant gave; an empty list means it had none. */
33
33
  export const participantImpressionsSchema = z.array(impressionSchema).max(LIMITS.impressions);
34
34
  /**
35
- * A closing report. A report without impressions is still valid: third-party provider ports and
35
+ * A closing report. `outcome` is how the participant says the task went: `reached` when it is
36
+ * finished, `blocked` when something in the app stopped it, `not_reached` for any other reason.
37
+ * A report without an outcome or impressions is still valid: third-party provider ports and
36
38
  * stored traces do not have them.
37
39
  */
38
40
  export const closingReportSchema = z.strictObject({
41
+ outcome: z.enum(["reached", "not_reached", "blocked"]).optional(),
39
42
  summary: said(LIMITS.summaryChars),
40
43
  frictionReports: z.array(said(LIMITS.frictionReportChars)).max(LIMITS.frictionReports),
41
44
  impressions: participantImpressionsSchema.optional(),
@@ -35,11 +35,8 @@ export function participantToolSchema(speechEnabled = false) {
35
35
  return z.toJSONSchema(toolInput(speechEnabled), { io: "input" });
36
36
  }
37
37
  export const PARTICIPANT_TOOL_SCHEMA = participantToolSchema();
38
- /** The participant's final account: how the task went, then its closing report. */
39
- const participantFinal = z.strictObject({
40
- outcome: z.enum(["reached", "not_reached", "blocked"]),
41
- ...closingReportSchema.shape,
42
- });
38
+ /** The participant's final account: its closing report, with how the task went required. */
39
+ const participantFinal = closingReportSchema.required({ outcome: true });
43
40
  export const PARTICIPANT_FINAL_SCHEMA = strictOutputSchema(participantFinal);
44
41
  /** humanish tool arguments, validated before the shared executor sees a batch. */
45
42
  export function parseParticipantTool(value, speechEnabled = false) {
@@ -7,7 +7,7 @@ import { CuaDeadlineError, raceSessionDeadline, requestScope } from "./race.js";
7
7
  import { notice } from "./trace.js";
8
8
  import { isCompleteTurnUsage } from "./usage.js";
9
9
  const DEBRIEF_REQUEST_CAP_MS = 30_000;
10
- const DEBRIEF_HINT = `The interactive session has ended. Return a closing account with summary, frictionReports and impressions. In summary, briefly describe only what you actually did and observed. In frictionReports, list only specific unexpected behavior, confusion, or recovery you personally encountered during this session. Preserve uncertainty. Use an empty list if you encountered none. ${IMPRESSIONS_ASK} Do not speculate, invent problems, quote instructions as observations, or describe planned actions. Do not request or take further actions. This is a closing account, not another attempt at the task.`;
10
+ const DEBRIEF_HINT = `The interactive session has ended. Return a closing account with outcome, summary, frictionReports and impressions. In outcome, say how the task went: reached if you finished it, blocked if something in the app stopped you, not_reached if you did not finish it for any other reason, such as running out of time. In summary, briefly describe only what you actually did and observed. In frictionReports, list only specific unexpected behavior, confusion, or recovery you personally encountered during this session. Preserve uncertainty. Use an empty list if you encountered none. ${IMPRESSIONS_ASK} Do not speculate, invent problems, quote instructions as observations, or describe planned actions. Do not request or take further actions. This is a closing account, not another attempt at the task.`;
11
11
  const CLOSING_REPORT = {
12
12
  call: (provider) => provider.debrief,
13
13
  contextHint: DEBRIEF_HINT,
@@ -156,9 +156,10 @@ function acceptDebriefTurn(session, turn, trigger, ask, record) {
156
156
  * A closing report: its summary and friction as one participant message, then its impressions.
157
157
  * At a scheduled end or a session cap an accepted report is the participant's account of the
158
158
  * session: its summary becomes the session's reason (loopResult), so the outcome it declares
159
- * becomes the session's declared outcome. The blocker rule reads a declared `reached` or
160
- * `blocked` before the summary's wording. After a matched stop condition or dwell window, the
161
- * condition stays the completion evidence.
159
+ * becomes the session's declared outcome. That is its `outcome` field, then the turn's (a
160
+ * provider port that sets it there), then the first line of its summary. The blocker rule reads a
161
+ * declared `reached` or `blocked` before the summary's wording. After a matched stop condition or
162
+ * dwell window, the condition stays the completion evidence.
162
163
  */
163
164
  function readClosingReport(session, turn, trigger) {
164
165
  const closing = closingReportSchema.safeParse(turn.closingReport);
@@ -182,7 +183,7 @@ function readClosingReport(session, turn, trigger) {
182
183
  session.trace.bump("messages");
183
184
  session.impressions = recordImpressions(session, closing.data.impressions);
184
185
  if (trigger.kind === "scheduled_end" || trigger.kind === "session_cap")
185
- session.declaredOutcome = declaredOutcomeOf(turn, closing.data.summary);
186
+ session.declaredOutcome = declaredOutcomeOf(turn, closing.data, closing.data.summary);
186
187
  return {
187
188
  status: "completed",
188
189
  detail: "one read-only report; no additional desktop actions; original stop and task outcomes preserved",
@@ -1,4 +1,4 @@
1
- import type { ActorCompletionReason, ActorStopCause, ParticipantDeclaredOutcome } from "../../contract.js";
1
+ import type { ActorCompletionReason, ActorStopCause, ParticipantClosingReport, ParticipantDeclaredOutcome } from "../../contract.js";
2
2
  import type { StopConditionMatch } from "../../stop-conditions.js";
3
3
  import type { LoopSession } from "./session.js";
4
4
  import { type Evidence } from "./trace.js";
@@ -29,10 +29,11 @@ export interface Stop {
29
29
  */
30
30
  export declare function declaredOutcomeFromClosingLine(message: string | undefined): ParticipantDeclaredOutcome | undefined;
31
31
  /**
32
- * The participant's own word for how it ended: the turn's schema field first, then the closing
33
- * line of `text`, its last message or its closing report's summary.
32
+ * The participant's own word for how it ended: its closing report's `outcome`, then the turn's
33
+ * (Codex and provider ports that set it there), then the closing line of `text`, its last message
34
+ * or its closing report's summary. A self-ending turn and a closing request read it in this order.
34
35
  */
35
- export declare function declaredOutcomeOf(turn: CuaTurn, text: string | undefined): ParticipantDeclaredOutcome | undefined;
36
+ export declare function declaredOutcomeOf(turn: CuaTurn, report: ParticipantClosingReport | undefined, text: string | undefined): ParticipantDeclaredOutcome | undefined;
36
37
  /**
37
38
  * The participant reported a natural endpoint. "not_reached" is a participant who stopped without
38
39
  * finishing: gave_up, which tallies as abandoned. "blocked" keeps goal_satisfied here (the actor
@@ -30,11 +30,12 @@ export function declaredOutcomeFromClosingLine(message) {
30
30
  return undefined;
31
31
  }
32
32
  /**
33
- * The participant's own word for how it ended: the turn's schema field first, then the closing
34
- * line of `text`, its last message or its closing report's summary.
33
+ * The participant's own word for how it ended: its closing report's `outcome`, then the turn's
34
+ * (Codex and provider ports that set it there), then the closing line of `text`, its last message
35
+ * or its closing report's summary. A self-ending turn and a closing request read it in this order.
35
36
  */
36
- export function declaredOutcomeOf(turn, text) {
37
- return turn.outcome ?? declaredOutcomeFromClosingLine(text);
37
+ export function declaredOutcomeOf(turn, report, text) {
38
+ return report?.outcome ?? turn.outcome ?? declaredOutcomeFromClosingLine(text);
38
39
  }
39
40
  /**
40
41
  * The participant reported a natural endpoint. "not_reached" is a participant who stopped without
@@ -126,10 +126,10 @@ async function runTurns(session, conversation) {
126
126
  continue;
127
127
  }
128
128
  if (turn.done || turn.actions.length === 0) {
129
+ const report = closingReportSchema.safeParse(turn.closingReport).data;
129
130
  // The declared outcome is kept even if redacting the participant's summary fails.
130
- session.declaredOutcome = declaredOutcomeOf(turn, turn.message);
131
+ session.declaredOutcome = declaredOutcomeOf(turn, report, turn.message);
131
132
  const ended = stops.participantEnded(turn, session.declaredOutcome, (text) => session.redactNarration(text));
132
- const report = closingReportSchema.safeParse(turn.closingReport).data;
133
133
  // A provider that can ask for impressions alone does so after the session, like a debrief.
134
134
  if (report?.impressions !== undefined || session.provider.requestImpressions === undefined) {
135
135
  session.impressions =
@@ -25,10 +25,11 @@ export declare const ACTOR_STATUSES: readonly ["passed", "abandoned", "incomplet
25
25
  export type ActorStatus = (typeof ACTOR_STATUSES)[number];
26
26
  /**
27
27
  * What the participant said happened, in a field rather than a paragraph. Providers whose
28
- * reply is schema-constrained (the local-agent routes) fill it on their final turn; a free-text
29
- * provider leaves it absent and the route falls back to reading the closing message. `reached`:
30
- * the task is finished. `blocked`: something in the app stopped the participant. `not_reached`:
31
- * the participant stopped for another reason (gave up, ran out of ideas).
28
+ * reply is schema-constrained (the local-agent routes) fill it on their final turn, and a closing
29
+ * report carries it as `outcome`; a free-text turn leaves it absent and the route falls back to
30
+ * reading the closing message. `reached`: the task is finished. `blocked`: something in the app
31
+ * stopped the participant. `not_reached`: the participant stopped for another reason (gave up,
32
+ * ran out of ideas).
32
33
  */
33
34
  export type ParticipantDeclaredOutcome = "reached" | "not_reached" | "blocked";
34
35
  /** Statuses that describe what happened to a participant rather than a harness malfunction. Verify
@@ -372,8 +373,11 @@ export interface ActorTrace {
372
373
  reason: string;
373
374
  /** Absent if no request was made; false means token accounting is incomplete. */
374
375
  usageReported?: boolean;
375
- /** The report's impressions are kept in `impressions`, with the messages that quote them. */
376
- report?: Omit<ParticipantClosingReport, "impressions">;
376
+ /**
377
+ * The report's impressions are kept in `impressions`, with the messages that quote them, and
378
+ * the outcome it declared at a clock end in `declaredOutcome`.
379
+ */
380
+ report?: Omit<ParticipantClosingReport, "impressions" | "outcome">;
377
381
  /** Links the readable projection so it is not heuristically classified a second time. */
378
382
  messageId?: string;
379
383
  };
@@ -81,10 +81,21 @@ function oneLine(text) {
81
81
  const line = text.replace(/\s+/g, " ").trim();
82
82
  return line.length > REASON_CHARS ? `${line.slice(0, REASON_CHARS - 1).trimEnd()}…` : line;
83
83
  }
84
- function participantLine({ id, status, session, diagnostics }) {
84
+ /**
85
+ * A participant's status, how it ended and its closing message. A participant the run skipped
86
+ * before its session started has no ending to classify, so its line ends on the reason the run
87
+ * recorded. That reason is the harness's own sentence, so it prints whole.
88
+ */
89
+ function participantLine(participant) {
90
+ const { id, status, session, diagnostics, skippedReason } = participant;
91
+ const ending = skippedReason !== undefined
92
+ ? [skippedReason]
93
+ : diagnostics
94
+ ? [formatCuaDiagnostics(diagnostics)]
95
+ : [];
85
96
  return [
86
97
  `participant ${id}: ${participantStatus(status)}${session ? ` (${session.completionReason})` : ""}`,
87
- ...(diagnostics ? [formatCuaDiagnostics(diagnostics)] : []),
98
+ ...ending,
88
99
  ...(session ? [oneLine(session.reason)] : []),
89
100
  ].join(" · ");
90
101
  }
@@ -37,14 +37,18 @@ function fanoutPlanEvents(args) {
37
37
  }
38
38
  /**
39
39
  * The run's summary in plain sentences: who took part, what happened to them with the denominator
40
- * attached, and that the gaps list the participants who did not pass.
40
+ * attached, and that the gaps list the participants who did not pass. A participant the run
41
+ * skipped before its session started never used the app, so it is counted in a sentence of its
42
+ * own.
41
43
  */
42
44
  function fanoutSummary(args, facts) {
43
45
  const count = args.specs.length;
44
46
  const people = plural(count, "participant");
45
47
  const rerun = args.rerun === undefined ? "" : `Rerun from ${args.rerun.sourceRunId}: `;
48
+ // The live summary is written once, before any participant starts, so it names the plan: by
49
+ // the time it is read a participant may still be waiting, or skipped.
46
50
  if (args.inProgress === true) {
47
- return `${people} ${count === 1 ? "is" : "are"} each using their own copy of the app. The run is still going, so nothing here is final.`;
51
+ return `${people} ${count === 1 ? "is planned, in its own copy" : "are planned, each in their own copy"} of the app. The run is still going, so nothing here is final.`;
48
52
  }
49
53
  if (args.dryRun) {
50
54
  return `${rerun}Dry run: ${people} would each use their own copy of ${subjectName(publicSafeAppUrlLabel(args.appUrl), args.aggregateSubject)}. No desktops were launched and $0 was spent.`;
@@ -52,7 +56,20 @@ function fanoutSummary(args, facts) {
52
56
  const tasks = facts.tasks === undefined ? "" : ` Tasks: ${facts.tasks}.`;
53
57
  // A host suspension comes first: it may explain everything after it.
54
58
  const host = args.hostSuspension === undefined ? "" : `${args.hostSuspension.summary} `;
55
- return `${host}${rerun}${people} took part, each in their own copy of the app: ${facts.outcomes}${gapsListClause(count - facts.passed, count)}.${tasks}`;
59
+ const tookPart = count - facts.skipped;
60
+ const outcomes = facts.outcomes ?? `0/${tookPart} finished a session`;
61
+ const gaps = gapsListClause(tookPart - facts.passed, tookPart);
62
+ const sentences = [];
63
+ // A run where every participant was skipped has no one to describe in this sentence.
64
+ if (tookPart > 0)
65
+ sentences.push(`${plural(tookPart, "participant")} took part, ${tookPart === 1 ? "in its own copy" : "each in their own copy"} of the app: ${outcomes}${gaps}.`);
66
+ // The gaps are pointed at once: here only when the sentence before did not.
67
+ const why = gaps === "" ? "; the gaps say why" : "";
68
+ if (facts.skipped === 1)
69
+ sentences.push(`1 participant was skipped before its session started${why}.`);
70
+ if (facts.skipped > 1)
71
+ sentences.push(`${facts.skipped} participants were skipped before their sessions started${why}.`);
72
+ return `${host}${rerun}${sentences.join(" ")}${tasks}`;
56
73
  }
57
74
  /**
58
75
  * The review's gaps: one line for the participants a host suspension likely failed, then a line
@@ -75,6 +92,7 @@ function fanoutReview(args, streams, judgment) {
75
92
  // The judge's verdict: live fan-out must prove every participant (judgeParticipants).
76
93
  const verdict = args.verdict;
77
94
  const passedParticipants = judgment.participants.filter((participant) => participant.judgedStatus === "passed").length;
95
+ const skippedParticipants = (outcomes ?? []).filter((outcome) => outcome?.skippedReason !== undefined).length;
78
96
  // What happened to the participants, with the denominator attached. The verdict above has to
79
97
  // collapse the run to one word; this does not (docs/principles/three-roles.md).
80
98
  const terminalOutcomes = (outcomes ?? []).filter((outcome) => outcome?.session?.status !== undefined);
@@ -103,9 +121,10 @@ function fanoutReview(args, streams, judgment) {
103
121
  // sandbox, so the summary carries its digest, which verify accepts in a shared run.
104
122
  summary: fanoutSummary(args, {
105
123
  passed: passedParticipants,
106
- outcomes: participants === undefined
107
- ? `0/${args.specs.length} finished a session`
108
- : formatParticipantOutcomes(participants, participantEndings),
124
+ skipped: skippedParticipants,
125
+ ...(participants === undefined
126
+ ? {}
127
+ : { outcomes: formatParticipantOutcomes(participants, participantEndings) }),
109
128
  ...(runTasks === undefined ? {} : { tasks: formatRunTaskFunnel(runTasks) }),
110
129
  }),
111
130
  gaps: args.inProgress === true
@@ -352,14 +352,18 @@ this paragraph is the record of that decision.
352
352
 
353
353
  `ComputerUseProvider.debrief` is an optional read-only request after a structured
354
354
  `stopWhen` or dwell stop. It returns a `ComputerUseTurn` with `closingReport` containing
355
- `summary`, `frictionReports` and `impressions`; an empty friction or impressions list is valid,
356
- and a report without `impressions` (an older provider) is still accepted. The loop rejects
355
+ `outcome`, `summary`, `frictionReports` and `impressions`; an empty friction or impressions list
356
+ is valid, and a report without `outcome` or `impressions` (an older provider) is still accepted.
357
+ After a scheduled end or session cap, `outcome` (`reached`, `not_reached` or `blocked`) is the
358
+ trace's `declaredOutcome`, read before the first line of `summary`. The loop rejects
357
359
  actions, pending safety checks, and invalid report shapes. It redacts accepted
358
360
  reports, records them in `ActorTrace.debrief`, and projects one readable message
359
361
  without invoking action or communication callbacks. Each impression becomes its own message
360
362
  and goes to `ActorTrace.impressions`; a session without them records why in the same field.
361
363
  A turn that ends the session itself may carry `closingReport` too, and the loop keeps its
362
- impressions the same way without a second request.
364
+ impressions the same way without a second request. Its `declaredOutcome` is the report's
365
+ `outcome`, then the turn's `outcome`, then the first line of its message, the order a closing
366
+ report at a scheduled end is read in.
363
367
 
364
368
  `ComputerUseProvider.requestImpressions` is the optional request after the participant ended
365
369
  the session itself without impressions. It asks for impressions only, returns them as
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "humanish",
3
- "version": "0.127.0",
3
+ "version": "0.127.1",
4
4
  "description": "Synthetic user research for apps, CLIs, and agent-facing product flows.",
5
5
  "keywords": [
6
6
  "cli",